Meeting minutes
as a reminder, we'll use IRC-based queue management in this meeting:
https://
Anssi: to suggest agenda topics, use Agenda+ label, e.g.:
<christianliebel> Present
Anssi: welcome to the new participants to the group:
… Fidel Tian and Brendan Ittelson from Zoom
… Andrew Nolan from Microsoft
… Ali Spivak from Google
… Mike Ozburn from Opn.li
… Jiwoong Yoo and Geunhyung Kim from SEAK
… Ryan Roemer from Nearform
… Andrew Comminos from Meta
… welcome all to the WebML Community Group!
WebMCP
Repository: webmachinelearning/webmcp
Protecting tools from 3P scripts in the top-level context
Anssi: issue #159
<gb> Issue 159 Protecting tools from 3P scripts in the top-level context (by yoavweiss) [Agenda+]
Anssi: I added this issue in an agenda update to discuss WebMCP iframe design before we go to the next issue about WebExtensions API
… Dominic and the Google team have spent some time exploring this
… problem statement: "an iframe may have access to information that the top-level document is not privy to"
… Dominic proposed a postMessage() style safety primitive as a solution
… Yoav seems to agree a postMessage() like model can work
… Johann suggests exposing tools to the user via iframe should not expose them to the embedding site automatically
Anssi: I believe the group agrees this issue is important
[ Dominic sharing slides about iframe design ]
Dominic: open questions from the slides: 1) default exposedTo behavior? 2) Should a parent be able to constrain a child's exposedTo[] list?
Benjamin: this matches closely my thinking in one of the previous issues, I like the shape of this, a few open questions on how to make it work exactly
Johann: I think the two open questions are the biggest ones to look into, not exposed should be the default
… not exposed to the 1P page and not to the agent
… worried about bank.com offering tools to make transactions, these could be attack vectors
… you don't need to call the tool yourself, just make it convincing for prompt injection attack
… close enough to the SOP violation
… that would lead to less adoption of this technology
Dominic: I hope the site with the tools can take more care in who to embed, maybe too much to ask?
Johann: why not go with safe defaults?
Dominic: agents as an extension to a user, the same attack is available if you can trick the user to actuate the same things
Johann: if you expose a tool, should it be also callable to the embedder?
Benjamin: the code examples online will frequently include unsafe defaults, this would not be too helpful
Johann: on the 2nd open question, there could be rug pull attacks, you're a helpful tool and then over time poison the context of the parent
… so the agent would need safety mechanisms that prevent this
Dominic: constrains on exposedTo to fix that?
Johann: I want 1P to be in control
MarkF: different from postMessage, define this at registration time vs. tool calling time
Dominic: in the prototype I expose tool caller origin so during the execution you see who is calling
Johann: unless you use * wildcard
Dominic: more ergonomics for allow update would be good
Yoav: this is great in terms of the model, and how to control different frames in the frame tree
… agree on the open question with Johann, use safe defaults always
… expected same origin exposure to be part of defaults
… constraining childs for expose list, no immediate use case for this
… we should make sure we have affordances to build it in the future
… Johann's use cases are about restricting the tools, not disallow childA's tools from childB
Yoav: default should be do nothing?
… exposed to agents but not other origins
<kush> +1
Johann: from security point of view, wildcard considerations different for a new API
Yoav: for impersonation the parent constraints the child, the exposeTo is opt-in, if I impersonate I opt-in to do that?
Johann: the child needs to opt-in to parent
Dominic: the child should do due diligence to check who is embedding it
Kushal: the behavour Johann wants, what if there's no value set in array
… what if someone copies code found online and puts it into an iframe
… all the agents will mix tools, iframe or embedder has no control over those
Johann: I'm worries about intentional attack, embed site to cause confusion
Benjamin: ancestor to be considered
<domfarolino> +1
<domfarolino> +1
<kush> +1
<brwalder> +1
RESOLUTION: Spec a registration-time origin exposure via registerTool() exposedTo with Permission Policy.
WebExtensions API to enumerate and invoke tools
Anssi: issue #74
<gb> Issue 74 WebExtensions API to enumerate and invoke tools (by reillyeon) [Agenda+]
Anssi: [ reorder the agenda to discuss this issue later today ]
Cross-document tool response mechanism
Anssi: issue #135
<gb> Issue 135 Cross-document tool response mechanism (by domfarolino) [Agenda+] [declarative]
Anssi: this declarative API related issue was spun off from the mothership declarative PR #76, now merged
<gb> MERGED Pull Request 76 Declarative API Explainer (by domfarolino)
Anssi: the key question posed in this issue:
… "Should the agent receive a response from the next page, after a declarative <form> tool navigates?"
Dominic: considering a more general approach for this, we have looked into what BrandonW proposed, <context> element style mechanism
Alex: would love to see the details on the <context> element
… would it be better to tag this "HTML list" with this <context> tag, rather than create a separate way to display this information?
… this model context, declarative read, would condense information
Dominic: this is sound, the downside is we push more work on the developer
… we want to be token efficient, could introduce a few attributes
… this may be annoying to the developers, they need an extra step with JSON on the page, maybe many pages already have JSON-LD for this?
… we need to talk to real sites and see how they feel about this
Alex: agree, declarative read API has a way for the model to read from the page, subset of information for the model to read, need to hear from practitioners and sites how this would work, iterate on the design
Kushal: avoiding developers to need duplicate data, we'd just take what is put in the DOM, hard to standardize how to extract data from the DOM
Kushal: if there's information that goes in there, it is additional context, we learn from implementation experience, when to do DOM scraping and not
Brandon: to Kushal's point, for in-page agents or iframe agents, they are not going to have access to the DOM, using the mechanism we're proposing, we expose through iframes, that's how they get any context from other iframes
… it is important to have a mechanism for read operations that does not rely on the DOM
… even if that means duplicating some information on the DOM
… in our testing, sometimes the most efficient workflow is not easily derived from human-readable DOM
… example, ordering from a restaurant
… a menu fits into LLM's context window
… better to consume the entire menu rather that the DOM that may have multiple pages
… human readable is not the same as LLM readable
MarkF: I was thinking if we want to provide signal to developer when to populate <context>
… form submitted via tool calling, we might target navigation via extra navigation, something in HTTP or document itself that signals this page load is consumer by agent
Dominic: header could be good for this
Alex: I haven't personally found benefit in form submission information, other than error messages
… does not need to be tied to tool call, these are multi-step processes
… I like the idea of <context> element, curious to see if we should add this level of complexity
Brandon: it sounds like we're not all aligned on whether a response to form submission is useful
… <context> has use beyond that, used in declarative read operation, could be used in form submission results if right attributes added
… propose we resolve to look into <context> element without committing to form submission response with it
<Alexn> +1
<brwalder> +1
<domfarolino> +1
RESOLUTION: Explore <context> element without committing to form submission response with it. (issue #135)
WebExtensions API to enumerate and invoke tools
Anssi: issue #74
<gb> Issue 74 WebExtensions API to enumerate and invoke tools (by reillyeon) [Agenda+]
Anssi: Simeon from WECG asked in the issue:
… - Should extensions also be able to expose their own WebMCP capabilities to the browser?
… - Should extensions also be able to expose their own WebMCP capabilities to websites?
Anssi: WebExtensions Community Group (WECG) F2F took place in London last week
Anssi: a lot of discussion about WebMCP <-> WebExtensions interaction took place at the F2F
Anssi: I don't want to speak on behalf of WECG
… I will also look into scheduling a joint discussion with the interested WECG folks
Dominic: we will also meet with Extensions people at Google today
Writing Assistance APIs
Repository: webmachinelearning/writing-assistance-apis
Summarizer preference parameter
Anssi: PR #102 to address issue #96
<gb> Issue 96 New parameter to support model tiering in Summarizer (by jingyun19)
<gb> Pull Request 102 Add clarifications to Summarizer preference parameter (by jaewon078) [Agenda+]
Anssi: Jaewon Lee from Google shared the PR for review with the group (thanks!)
… this PR adds the following note to the spec for conflicting constraints and routing fallbacks:
… "When resolving the underlying model, the implementation should prioritize hard functional constraints over the "preference" hint. For example, if the requested options require specific capabilities (such as a requested language in "expectedInputLanguages") that are only supported by a model that does not align with the requested "preference", the implementation should select the model capable of completing the task."
… I see no concerns raised in the PR review, and adequate review time has been provided
… any additional comments from the group?
… we're good to merge this PR after addressing the few review comments, thank you for this contribution, Jaewon!
Jaewon: thank you for explaining the change, we addressed the group's feedback with this PR
RESOLUTION: Add clarifications to Summarizer preference parameter per PR #102.