– DRAFT –
WebDriver Meeting May

<jgraham> RRSAgent: make minutes

Autofill Testing

github: w3c/webdriver#1797

https://www.w3.org/wiki/WebDriver/2024-05-BiDi

Yoav: We've been looking at the autofill space recently at shopify. There are a lot of issues with interop here and and how to ahndle this
… it would be great if we could have a way to automate this through testing with webdriver and with WPT
… working with Christian we have a test suite for the various issues that we have come against
… we have put forward a webdriver PR on how to solve this
… sadym has pointed out that we are now working on bidi which we have moved acorss to
… and we have a simple API here where we save the data and then we have a way to trigger it

so... does this solution seem sensible?

theindra: I think Yoav has pointed out most of this
… in the spec we have spent a lot of time trying to make this work
… so for merchants there is a lot of work going into this
… so automated tests would be good here

jgraham: I don't know a lot about autofill and I've reached out to mozilla here on how to solve it
… so from wpt point of view there is no spec and was browser defined

<whimboo> https://html.spec.whatwg.org/multipage/form-control-infrastructure.html#autofill

Yoav: there is something in the html spec and unfortunlaty it's not as precise as it should be
… there is a browser feature component here ad the UI can be different
… some of the tests are implementing the current spec but where we want the spec to be

jgraham: from my point of view... i've seen autofill issues that look like browser issues and people get confused
… so agree it's boundray breaking here

gsnedders: from our side that we, apple, do ignore what the html calls field type
… and there are a lot fo sites that try disable autofill
… and we use heuristics to figure out what need this. We, apple, have issues with the spec being overly strict in the spec. We don't see the major issue in the webdriver at the moment but we need to look at this closer

Yoav: so... what you're saying is this has become a little bit of an adversary issue
… do you know why sites do this?

gsnedders: some companies believe this is good security which is not the actual truth for them
… I don't know enough of the why but I know it exists

Yoav: I see why uou dont want to solve that as too strict but it would be good to solve this for sites that are not being adversarial.

theindra: There is a spec for turning off the spec for turning off "disabling"

gsnedders: yes and we ignore that sometimes as it causes "breakages" which is better for the users not the developer

Yoav: I see where that is can make sense e.g. passwords being blocked from autofilled
… it would be good to solve this for the 80% rather than for the perfect

gsnedders: where is this implemented in chromium

orkon: We, google, would need to discuss this with the autofill team. we know that this already exists in CDP
… the current PR isn't necessarily in line with how browsers are working, like heuristics mentioned by gsnedders
… it might be better to fill in the form and then get the browser to save that form and use it in a later tests

Yoav: is the issue with the rigidity of the "save" API.

orkon: there are times where we can have multiple credit card fields
… and how would we do that in an API version?

Yoav: I see what you're saying as there could be real world examples with payment forms.

orkon: yes. there are payment forms across different iframes

Yoav: this is useful. It would be good if we can have these details in the PR

unicode graphemes in `keyDown` events

github: web-platform-tests/wpt#46019

sadym: classic doesnt specify if we should follow grapheme clusters. There is a placeholder issue
… I would like to make a decision here and either update tests or remove it?

gsnedders: this feels a little nebulous here. Grapheme clusters can vary a lot

<sadym> https://www.unicode.org/versions/Unicode9.0.0/ch03.pdf#G2213

jgraham: this is defined in unicode but the fundamental issue is how much do we want to care about sendKey to map to 1 key
… there was once a proposal once about adding IME support

<gsnedders> https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries — which defines a bunch of different algorithms to generating grapheme clusters, but also: "These algorithms can be adapted to produce tailored grapheme clusters for specific locales or other customizations, such as the contractions used in collation tailoring tables."

jgraham: the question here is "as a user" would you be able to do this as keypress or via composition events

sadym: from the user point of view. we have an api in classic sendkeys which doesn't exist in bidi
… which takes the string and splits it and does the event handling

jgraham: yes. do we want to model this as 1 key press or composition? I think for simplicity we want to have this as 1 key press. This is what the IME. stuff would ggive us if we implemented it

sadym: so the next question. do we want graphemes or any number of symbols

jgraham: I don't think we want to allow people to type any number of chars here
… e.g. how would abc be handled in a form? one keypress doesnt make sense but a b c

sadym: so the summary is we want to support graphemes?

jgraham: yes that makes sense but we need gsnedders to give an opinion here

gsnedders: I think we do what ever appkit does here

gsnedders: I don't want a scenario where webdriver sends keystrokes that safari cant handle

ACTION: gsnedders to get back to sadym on what happens in this scenario with appkit

Clarify the logic to compute the initiator type of a request

github: w3c/webdriver-bidi#698

orkon: we have an network initator which has a lot of TODOs
… and we have been working throuh use cases in puppeteer
… and we have been looking at the entire chain of the network
… the current spec is missing the URL to show where the request has been started from
… the ideal solution here would be to specify the initiator if possible. I know Chrome doesn't support it but we will look into how this could work
… so the next question is would we want to align with the fetch spec ?
… it is feasible to keep track of the line number and position if another document initates the request

jgraham: the spec here didn't have it as we didn't know what clients might have
… and we didn't know if things in the fetch spec are exposed... and I think we need to go investigate this a lot more
… and we can take a proposal here to start from fethc spec. If there are things missing from the spec we need to figure that out
… and if we need to add that to the fetch spec accordingly. so if we can write this up as a proposal so we can go figure out if this is something we can implement and then get back to you

orkon: I think for the most part I think that is fine but where in comes to the parser state that is not documented anywhere
… and we know that if things are initated from a css file we have no way of knowing hwere it started

jgraham: for the parser stuff... we can see if the can expose them as we know devtools can show that data
… it seems possible but we don't have this specified anywhere. we can say we will do a best effort
… so we can avoid rewriting CSS and html spec to keep state

orkon: ok... I will go write a proposal on this then

AT-Driver: SafariDriver & Glass Panes;

github: w3c/at-driver#75

jugglinmike: we have been working with aria folks for automating screenreaders
… for safari we are working on a customer voice and keypresses
… so we use webdriver to set the browser state
… but we've hit a safari glass pane
… which has lead to use looking to specify higher level intents
… so we want to speak to folks at apple to help us to solve this security protection
… and are other browsers looking to implement it

gsnedders: we are aware that the glass pane does prevent some usecases
… what I will say is that you need to raise an issue on feedback assistant so we can try look at this.

<jgraham> RRSAgent: make minutes

<jgraham> RRSAgent: make logs public

<jugglinmike> AutomatedTester: The main reason the glass pane exists is to work with a historic lack of profiles

jgraham: we don't have plans to change our security model in Firefox

<gsnedders> it's not just profiles; it's also the risk of WebDriver directly being able to exfiltrate data from a session if the user starts treating it as their real browser

<gsnedders> jugglinmike: also please send me the feedback ID once you've filed!

<jugglinmike> will do gsnedders

<jgraham> RRSAgent: make minutes