W3C

– DRAFT –
Automation summit Berlin

24 August 2022

Attendees

Present
AlexRudenko, AutomatedTester, cb, diemol, foolip, gsnedders, hablich, mathiasbynens, Meysam, PujaJagani, sadym, Sasha, simonstewart, thiagowfx, titusfortner, whimboo
Regrets
-
Chair
David Burns
Scribe
AutomatedTester, David Burns, foolip

Meeting minutes

<ss> RRSAgent: start meeting

<foolip> RRSAgent: this is WebDriver BiDi Automation Summit Berlin

<simonstewart> I am impressed at the number of people in this room who feel safe to not have their laptops plugged in....

<simonstewart> Who's scribing?

I am when we start officially

<hablich> Anybody a link to the Edge roadmap?

<simonstewart> I'd love to see the Edge Bidi roadmap

<hablich> Maybe it was posted in the Zoom call? can somebody share it here too?

<mathiasbynens> everything is linked here: https://www.w3.org/wiki/WebDriver/2022-08-BiDi-Automation-Summit-Berlin

<hablich> thanks

mathiasbynens (IRC): These slides show the road map that Chrome see ... and to start a discussion
… THere are a group of users from implementators to end users
… and here is the WPT.fyi page to show that things are moving forward
… but we need to make sure that this group has alignm,ent to make sure that users are put forward
… here is the chrome milestones
… and here is our roadmap
… [talks through slide "Chrome's Current Roadmap"]
… this is based off the Puppeteer use cases
… We don't see this as full roadmap or the order we think everyone should work on
… what about the other items?

Slideset: https://docs.google.com/presentation/d/18SJv0K1_yeF9h6Gg42El-iMtL3uGiZRYgqmrScCqCuE/edit#slide=id.p

<gsnedders> (those slides aren't currently public)

<foolip> This is https://github.com/GoogleChromeLabs/chromium-bidi/milestones?direction=asc&sort=title&state=open

<hablich> Slides are public now, oopsie

simonstewart: step 7 in your roadmap would be to reformulate webdriver http on this...

mathiasbynens (IRC): is that a goal

simonstewart (IRC): yes that has always been a goal

<simonstewart> It should be possible to do everything from webdriver classic in webdriver bidi

gsnedders: webdriver classic implementations are built to similar things that are in CDP

<mathiasbynens> slides link: https://docs.google.com/presentation/d/18SJv0K1_yeF9h6Gg42El-iMtL3uGiZRYgqmrScCqCuE/edit

<simonstewart> Thank you

foolip (IRC): do we want to rewrite the spec or are we wanting to copy behaviour.

simonstewart (IRC): I don't think we want to rewrite the spec but we want to make sure we have the same behavior

whimboo: we are missing a lot of tests in the webdriver classi which makes things harder to make sure we have the same behaviours

https://docs.google.com/document/d/1azpI_HHYDew5UzB_r-b9J-OsWRAP1E3g2bstAPs7yF8/edit?pli=1#heading=h.yplbn92py38w.

Natasha: I wanted to introduce myself, I work on devtools and automation
… this the doc that Brandon did
… The first part is integrate with Bidi into Edge
… after building something things Brandon added that we should do a refactor for EdgeDriver over Bidi
… and how this will impact future tests

foolip: have you given more thought on how you will ship bidi?

<foolip> that was me first, then hablich

<hablich> I was talking, I am in the off

gsnedders: THere is nothing in there that is specific to edge in this code?

Natasha: There is no deviation from Chromium to Edge

hablich (IRC): I have question around perf metrics. Is there an attempt to make this cross browser or is there more introspection

ack

sadym (IRC): are the priorities in the doc in order?

Natasha: yes but 2 is the most important part to this group

sadym (IRC): what was the process on how Brandon created this list?

Natasha: great question, I will get Brandon to respond

<whimboo> Firefox status: https://wiki.mozilla.org/WebDriver/RemoteProtocol/WebDriver_BiDi

whimboo: I don't have fancy lides to share
… I have just shared where we are so far
… we have been implementing what users have wanted
… [discusses a few items in the doc]
… the next milestone is working on Network Interception
… we don't have anything further planned as we are waiting for the outcome of this meeting

foolip: my question is about network events ? Do you think webdriver bidi should have how to implement a HAR?

whimboo: good question, we have only done the network interception. It's a big question
… and we're collecting the events only

<foolip> Looks like https://github.com/w3c/webdriver-bidi/issues/196 is the spec change that is needed for this.

<gsnedders> gsnedders sharing https://matrix.org/_matrix/media/r0/download/mozilla.org/774c24c6adccccd812b410301c19d3eedf64a92c/Untitled%202.001.png

gsnedders: For Safari, Apple does not comment on future products and releases

diemol (IRC): is there anyway to get help to Safari from the Selenium project? Like asking Igalia to help?

gsnedders: I am not sure I am the right person to answer. There are 2 potential questions that needs answer do we need to have a protocol along side or on top of protocols will impact how we do things in webkit to safari

AutomatedTester: If Apple would like support from the Selenium project or other would like support do let us know

Christian Bromann: I am currently working on wdio and the next release will have this in it
… I am hoping to get the cddl package from Saucelabs so we can autogenerate things better on the fly

<foolip> gsnedders: What was slide 2? Or 1?

<cb> https://www.npmjs.com/package/webdriver/v/8.0.0-alpha.249

<foolip> gsnedders: :D

whimboo: which commands does it implement?

<mathiasbynens> gsnedders: missed opportunity to make slide 1 say "One more thing…" with slide 2 being the punchline

Christian Bromann: you send and receive objects only at the moment
… it does the lowest level comms

foolip (IRC): are you trying to support webdriver "classic" users or are you going be having a bidi mode

Christian Bromann: the transport layer won't be visibile to the user. As features get added they will slowly move over to the other users

simonstewart (IRC): This is likely how the Selenium client is also going to do it

Puja Jagani: THis is what we have currently written and moving things forward
… this has come from the discussions with Mozilla
… the first implementation was done by Alex (p0deje) from the selenium project
… and we're working on Java next
… [describes handshake]
… [describes event handling API at a high level]
… this document is a living doc as we move forward through things
… I know there is some discussion about supporting CDP and BIDI in the future
… and this is our starting point

<diemol> Link to Puja's doc https://docs.google.com/document/d/1dCd8Y2PYaR5mOGSmNTwllEHNmFqegfoGkP-TCKvPzSU/edit?usp=sharing

titusfortner (IRC): I haven't read the doc as much as I would have liked to
… I am not sure if this is in the doc using `execute_cdp_command`
… and we don't want people using that
… and we need to keep plans for what we do and have plan for docs

Puja Jagani: we have always told people not to use CDP rawly (is this a word?) but this is more about the public APIs
… it's covered but I will add more specific prose in my doc for that

hablich (IRC): for puppeteer we plan to move things to bidi unless we need to have a specific item in cdp

titusfortner (IRC): my issue is that there are docs on other sites using low level APIs when it's not the right way

simonstewart (IRC): historical note: back in the day we did only http
… then we added execute cdp and never got anything back
… and then we moved on to how we do CDP
… Network Intercept was one of the more popular ones
… and Selenium could deprecate and remove the `execute_cdp_command` as there are much better ways to do it

foolip (IRC): we are going to have an interesting time ahead as clients will have to do feature detection and then go from there

simonstewart (IRC): it's fun times ahead... we could use exception handling as flow control 🤮
… we have prior art in the client on how to do that

https://docs.google.com/presentation/d/18SJv0K1_yeF9h6Gg42El-iMtL3uGiZRYgqmrScCqCuE/edit#slide=id.g146bd9eddb0_1_23

Alex Rudenko: I work on Puppeteer at Chrome
… we are going to adding bidi alongside with cdp
… [discusses code example]
… we are going to keep the client API the same as much as possible

whimboo: Will you use Bidi directly or have upgrade session process?

Alex Rudenko: not sure

sadym (IRC): we haven't settled on how we are going to be doing that

Alex Rudenko: we will try support everything as we move forward

foolip (IRC): do you have anything in particular with how the session is done?

whimboo: currently we have the drivers from classic to get things into the correct state and then get the connection

Alex Rudenko: we aren't sure how this will be in the spec like do we start the browser and then send capabilities...

whimboo: there are some PRs from jgraham that hopefully solves this

simonstewart (IRC): we need to think about how people use scalable selenium systems like our SaaS providers (BrowserStack, Sauce, et al) and Selenium grid
… we need to have a standardised way to make all the permutations on clients easier to update

foolip (IRC): [missed question]

simonstewart (IRC): [describes running things locally with chrome and then moving to a grid to scale up]

Alex Rudenko: [describes 2 ways to start sessions in Puppeteer]

<simonstewart> A consistent mechanism for starting a session means that local ends don't need to adopt a lot of additional complexity

<gsnedders> <br dur="15m">

<AutomatedTester> s/[missing question]/if anything needs to change to support what Simon asked for, if we're not talking about the hub/grid URLs that you can connect to, and then upgrade to BiDi./

RRSAgent (IRC): make minutes

rrsagent: make minutes

<hablich> For people locally at the wework: Please ping me via mail in case you want attend today's dinner!

WebDriver BiDi adoption by Selenium

sadym (IRC): I want to see how we can move these things forward
… what are the missing missing parts, collab process and priority of use cases

sadym (IRC): missing parts?

Titusfortner: We have connection starts but that's it

Puja Jagani: that is correct.

Titusfortner: We have logging which is purely specific to Firefox
… but I see that Edge has some of that from the roadmaps
… at the moment everything is missing except logging

sadym (IRC): what is required to get adoption?

Puja Jagani: The roadmaps give us a direction but it would be good to know what is being impl,mented by each group as we move forward

whimboo: There are 2 points: 1 ) we currently see support for firefox, it would be good to add the same for Chrome
… and future APIs, navigation and script execution are next and nearly complete

Puja Jagani: we only did Firefox because that's the only one that we know had that features ready
… we are going to need to do that for CHrome and then having a mechanism to send the info to Selenium TLC group for next steps
… as things are implemented we need to have a way to know what and when things are ready

diemol (IRC): for me it's really hard to understand the use cases for the spec
… do we have a list of use cases that we can then track against

mathiasbynens (IRC): I'm not sure it exists but that's how we have approached it for Chrome
… but it would good to track them

simonstewart (IRC): There used to be a doc/issue that had this but I can't find it

<cb> https://github.com/w3c/webdriver-bidi/blob/master/explainer.md#goals

<gsnedders> [various discussion about where we originally had goals recorded, trying to find it]

simonstewart (IRC): the link shared has a link to the goals

mathiasbynens (IRC): It looks like we have diverged from what we wanted and what we have implemented

simonstewart (IRC): [discusses items in the list at high level]
… I don't think we have diverged too far
… the item that's not in there are devtools support [did not catch the rest]

diemol (IRC): I am trying to follow the spec and since I don't fully understand it...
… is there a way we can have this in a way that maps user cases to prose changes
… I think this will allow client implementors to follow

hablich (IRC): from a list of use cases to what is implemented?

[various of how use cases could be written]

diemol (IRC): like we have the original plans from 2019 but no way of knowing where we are on that list
… it would be good to make sure we can track it

hablich (IRC): I can see this being useful for Selenium
… are there others that could benefit

Christian Bromann: I've been following but it's a lot work to follow

<mathiasbynens> https://wpt.fyi/results/webdriver/tests/bidi?label=experimental&label=master&aligned&q=webdriver

mathiasbynens (IRC): we have the wpt.fyi
… it's good to track things across browsers

AutomatedTester: Does wpt.fyi have sufficient linking to use cases for non-browser folks to understand

diemol (IRC): it would be good to have something in the spec to help us

mathiasbynens (IRC): I think we can use features in bikeshed to help do this

diemol (IRC): It would be good to help end users know what is possible

titusfortner (IRC): it would be good to see how end users know things in selenium/puppeteer terms

Sam Sneddon [:gsnedders]: another way to do this is to have the explainer linked to the parts of the spec that supports that feature

<sadym> https://github.com/GoogleChromeLabs/chromium-bidi/milestone/2

sadym (IRC): here is a link to our milestone 2
… in the milestone we show the command that people need to complete this tasks
… and we have links to the relevant WPT
… and we also have items for how far other implementors are
… and it has a link to a example script

<sadym> https://github.com/GoogleChromeLabs/chromium-bidi/blob/main/examples/cross-browser-simplified.py

mathiasbynens (IRC): we can see about moving some of these use cases from the Chrome repo to the W3C to help people understand where things are

sadym (IRC): [discusses python example linked above]

sadym (IRC): we'd be happy to move share this in a wider context

hablich (IRC): Assuming browsers are going to implement it super fast for Selenium to catch up?

simonstewart (IRC): it depends on the binding

AutomatedTester: [desribe issues with async/sync puthon ]

diemol (IRC): we want to release things as soon as possible but we want to release in tandom

foolip (IRC): we need to link and make that better but a lot of the work that has been fulfilled is purely scaffolding
… it would be also good to help get things prioritised

Prioritising Use Cases

<foolip> https://www.w3.org/wiki/WebDriver/2022-08-BiDi-Automation-Summit-Berlin#Agenda updated by simonstewart

simonstewart (IRC): This list is the thing that I think a lot of people are already expected

simonstewart (IRC): I think things like DOM events is already there

Alex Rudenko: allows you to install a listens and then go from there

simonstewart (IRC): to us that is "bootstrapping" as it allows you install a script and do things

foolip: [discusses bootstrapping in sandboxing with sadym]

simonstewart (IRC): you probably want those bootstrapped scripts to run before a page is loaded
… bootstrapping, dom events and logging is the main 3 for selenium
… video recording is handled out of band

<simonstewart> Request interception is also a high priority item for Selenium users

Christian Bromann: I would allow us to implement listeners so we can compete properly against tools like cypress

foolip: is bootstrapping worth it in or out of sandbox

simonstewart (IRC): what does a sandbox mean here?

sadym: [unaudible]

diemol (IRC): one of the highest priorities from Selenium is "Network interception"
… this is the most requested and hardest to use
… from Selenium that is the highest
… we do get a lot of people asking for video recording and, as simon saidm we offer that in docker containers
… but people want that in the borwser

<simonstewart> Sandbox in this case is the JS execution context

whimboo: we need to see about getting network logging done first and then we need to work on the interception next
… jgraham has a PR for this already

<foolip> James's PR is https://github.com/w3c/webdriver-bidi/pull/204

simonstewart (IRC): do we want events and interception as 2 different groups of work
… [describes immutability and mutability of these events]
… [describes how it workswith Alex Rudenko in modules]

whimboo: we need to start somewhere and we're starting with event logging of network events

sadym: how would we impl.ement this in the clients?

simonstewart (IRC): local ends already paper over the cracks between implementation
… we can try cdp and fail and try bidi

sadym: what about clients that do cdp but not classic they wouldn't necessarily be able to do it
… if we deprioritise classic items then it will take non selenium clients longer to move across

<simonstewart> The Actions API is https://w3c.github.io/webdriver/#actions

<foolip> James's PR for an input module is https://github.com/w3c/webdriver-bidi/pull/175

<foolip> scribe foolip

<foolip> RRSAgent: scripe foolip

diemol: On the different use cases, wondering if the next topic (reformulating) is more important? Reaching out to framework implementers which aren't Selenium. They caught up with WebDriver in a year or two. Catching up with BiDi would take more time. Important to discuss if we'll have classic working over BiDi?

diemol: People like to use new languages, and they've capturing a good audience. WDYT?

simonstewart: I'd be happy to discuss that

foolip: In terms of the order, I think we have two things. (1) is adding new capabilities to a framework like Selenium, which leads to new things like logging and network intercept first

foolip: (2) is supporting new browsers in Puppeteer using BiDi, which brings along "everything" first

gsnedders: While there's a lot of benefit in having a lot of classic defined in terms of BiDi, or having everything classic be doable over BiDi, from our PoV we'd much rather see some more immediate advantage above and beyond recreating classic over a new protocol.

simonstewart: So a market differentaition of the spec?

gsnedders: right

simonstewart: Talk about reformulation and migration. Is it worth having a deprecated WebDriver module that allows to execute a classic command over BiDi? And we remove things from it as we add things to BiDi? That'd solve supporting both at the same time.

simonstewart: Is that mad from an implementation PoV? It's the opposite of the CDP escape hatch in chromedriver.

whimboo: The difference to chromedriver is that we implement BiDi in the browser itself. While WebDriver classic is in geckodriver.

whimboo: Marionette and RemoteAgent are separate components, might not be easy.

sadym: Same in Chrome, we have different modules for chromedriver and bidi.

simonstewart: boils down to defining a classic command in bidi

<gsnedders> simonstewart (IRC): I presume you're imagining something like `{"url": "/session/{session id}/element/{element id}/click", "method": "POST", "body": {}}`?

foolip: think the issue is that it would amount to reimplementing the classic command, not merely adding glue code

Someone: one can share code

<simonstewart> gsnedders: yes, exactly like that

hablich: Would we be able to deprecate the classic escape hatch ever? If not, it would be a strange situation.

hablich: We're now talking about Selenium and supporting them. And for Puppeteer we have cross-browser goals. Other frameworks too. How does this all influence our roadmap? Depends on whether we want to move CDP-based frameworks first.

simonstewart: How much does it matter? Selenium has a mechanism to move some functionality to BiDi. Puppeteer also has some mechanism. Do we need to move everything in classic to get started?

hablich: I also don't know. What about Cypress?

simonstewart: Cypress works in a very different way, they inject script.

cb: they also use CDP for some things

AutomatedTester: As a result Cypress can't navigate across origins, etc.

Reformulating WebDriver Classic on WebDriver BiDi

AlexRudenko: I'm not very familiar with classic. How big is the difference between classic and what's now in BiDi? Seems like the only big difference is input events.

simonstewart: There's also element locators.

AlexRudenko: We can implement those with script evaluation.

simonstewart: going through the spec, there's also capabilities, modifying timeouts (for script/navigation/etc.), element states (mostly doable with JS), interaction, cookies (nice to have more control)

simonstewart: actions API is missing, handling user prompts, screen capture and printing.

simonstewart: there's also the weird concept of "is the element displayed?"

AlexRudenko: I know CDP capabilities quite well, and for a lot of things there's no other way than to evaluate JS. Do we need to be as granular with BiDi as in classic?

simonstewart: With BiDi we wanted to avoid embedding too much complexity in the framework. More things that can go wrong.

simonstewart: e.g. finding elements is fine, but what about Shadow DOM?

simonstewart: But there's nothing to stop us from starting with JS and moving to a module as needed.

AlexRudenko: On classic vs new use cases, if we don't have basic support, every new user of BiDi will need to use classic as well?

simonstewart: yes, or CDP

AlexRudenko: but CDP is not cross browser

simonstewart: Selenium has a mechanism for upgrading to BiDi. Puppeteer also, in that you could try BiDi and fall back to CDP. New frameworks, I'd be inclined to not worry about that.

whimboo: We can collect the most important thing from both sides, and maybe they match.

sadym: My main concern on adopting BiDi in Puppeteer, if we can execute JS then the main things left are input, screenshots, printing. So I'd like to prioritize those, but not reformulate everything classic.

<Zakim> gsnedders, you wanted to ask whether someone should (soonish) look at WebDriver Classic commands and create a mapping to BiDi, so we can see where the gaps are

simonstewart: Input is the most important.

sadym: absolutealty

sadym: Simon, you'd like to avoid injecting JS?

simonstewart: long term yes, would be nice to only need one connection to a browser.

AutomatedTester: we should also consider SaaS providers, injecting JS can add latency

AutomatedTester: For example a "get attributes" command (scribe is not familiar with this)

AutomatedTester: Want to avoid "inject this JS" for every single binding/language

AutomatedTester: large amounts of JS at least.

simonstewart: In a simple case it's just `document.getElementById()` but it gets more complicated.

simonstewart: there are even implementations in curl

diemol: and bash

AlexRudenko: Puppeteer is of course biased, since it's JS-only.

AlexRudenko: It gives you some flexibility, we can wait on some things in JS, like selectors matching, next animation frame, etc.

AlexRudenko: But if we want Puppeteer to work on BiDi, we'll definitely need sandbox scripts, bootstrap scripts, and bindings (installing a function that if called results in a callback in the protocol)

AlexRudenko: Without that it'll be a challenge to use BiDi in Puppeteer. But I can see that a lot of JS could slow things down.

simonstewart: Those are useful features, having them would be great.

simonstewart: Something you might want is the ability to wait for a command to return a non-falsy value.

AlexRudenko: That's implemented by injecting JS now.

AlexRudenko: This is done by returning a Promise, can then wait for anything.

AutomatedTester: that's the end of the agenda

foolip: what were the 3 things for Selenium?

diemol: (1) network interception (2) video recording (lower priority)

diemol: prioritizing new use cases will leave basic use cases behind.

whimboo: and (3) bootstrap scripts, via cb

cb: I'd say intercept is more important than plain network events

simonstewart: there's also DOM events, things like windows/documents being created, and DOM mutation

<cb> I would have another topic I would like to discuss if we have time

simonstewart: a chatty protocol isn't good for testing service providers. classic is pretty good for this.

simonstewart: BiDi could be very chatty with network intercept etc. etc.

AutomatedTester: agree with simonstewart. logging all events will produce a lot of traffic going back to a single machine

AutomatedTester: we have logging filtering for this

diemol: on prioritizing DOM events, it's very useful, but I don't see that happening in the GitHub issues. Might be an opportunity for DevRel to share what's happening in BiDi.

diemol: Can avoid a blocking wait in your tests, just have a listener.

cb: One thing I've tried to avoid is things like stale element exceptions. With the click command we try to move the element into the viewport, but that's sometimes difficult with fixed positioning. Are there things we can do to avoid stale element exceptions?

cb: stale element exceptions and the element not receiving a click are the two most common errors I see

cb: better error messages that would help us frameworks act on it would be helpful

cb: give framework implementers more utilities to be robust. protocol can't do everything, but can give more information.

simonstewart: stale element exception is for when the element is no longer connected to the DOM

https://w3c.github.io/webdriver/#dfn-stale-element-reference

AutomatedTester: Deviating too much from what we already have can get us in trouble.

AutomatedTester: My concern is we need to be deliberate when we deviate, mental models might need to change.

foolip: are there spec or chromedriver bugs for these two issues?

cb: Don't want to point fingers, but there are various reasons that this can happen. I can't tell whose fault it is.

<hablich> Is it a bug report like this: https://bugs.chromium.org/p/chromedriver/issues/detail?id=3944&q=stale%20element%20exception&can=2 ?

cb: One idea is can we move the element to the center of the viewport instead ot the top?

cb: Giving framworks more ability to act could be providing the ID of the element that overlays the element you tried to click.

simonstewart: this is https://w3c.github.io/webdriver/#dfn-element-click-intercepted

simonstewart: we can add data to exceptions, and we could add an element ID here

foolip: is there a spec issue?

cb: I'll take the action item.

ACTION: cb to write a spec issue

whimboo: I have a list of 7 items now.

(will paste soon)

whimboo: from Puppeteer I have (will paste)

AlexRudenko: sounds right, order maybe tweak a bit. Is there something about frame tree updates?

whimboo: that's the browingContext.contextCreated event

<whimboo> Selenium... (full message at https://matrix.org/_matrix/media/r0/download/matrix.org/UkIvRmvJHJhHDbbUiOksQBox)

Selenium

* logging

* Boot strap scripts (add bindings)

* network events and interception

* dom events (window/document created, DOM mutation)

* stale elements

* element interactability (eg fixed elements or overlays, in view)

* video recording

Puppeteer

* Boot strap scripts (add bindings)

* Sandbox scripts

* input

* screenshot

* printing

AlexRudenko: Is there a way in BiDi to intercept a browsing context creation?

sadym: is this different from bootstrap scripts?

AlexRudenko: this is about configuring the new context before it's created

AlexRudenko: maybe this depends on how we define bootstrap scripts

foolip: this feature is where the iframe process is paused and the CDP client can do things before continuing

simonstewart: what kinds of things?

AlexRudenko: configure request intercept, for example.

whimboo: is this a new feature?

AlexRudenko: it's new in Puppeteer, but old in CDP

AlexRudenko: whether this is important depends on other APIs, if other APIs are declarative.

simonstewart: let's add it to the list / file a new issue

foolip: the implication of this is other APIs need to be on the style "if this happens, do that" like the webdriver classic alert API

whimboo: does this affect only events? for bidi we can subscribe to all browsing context. new iframes automatically inherit the same set of events.

whimboo: I can only see a problem when opening a new tab/window and you want some new setup that only affects that window.

AlexRudenko: Example of declarative API challenge: I want all of the events from all first party iframes.

AlexRudenko: This might be a convoluted example.

simonstewart: With network intercept, we want filters to be specified of specific requests, to reduce chattiness.

foolip: I think this is a different tradeoff than being able to stop a browsing context before it's doing anything

simonstewart: some back channel use cases: (1) manipulate for cookies for any domain

simonstewart: so this is the login use case

simonstewart: webdriver classic can only set cookies for the current domain

whimboo: the problem users have is they're not logged in, they navigate, and the page checks `navigator.webdriver` and redirect away from the login page

simonstewart: (2) ability to manipulate the browser cache, clearing it during a cache

AutomatedTester: clear the cache?

AutomatedTester: there are various forms of storage. bfcache? storage?

simonstewart: the network cache

foolip: is this for test reliability reasons?

AutomatedTester: private mode between tabs?

simonstewart: no, but half way through the test, clear everything, expect to have to log in again

AlexRudenko: Two more features

AlexRudenko: (1) testing browser extensions, end-to-end testing of those

AlexRudenko: (2) getting accessibility information from the browser (some support in Puppeteer)

whimboo: on (1) we do that already, can switch to the "chrome" context to interact with the Firefox UI.

whimboo: this is also used for testing web extensions. security implications of course.

whimboo: (2) a11y, we haven't implemented them in Firefox, but there are two new webdriver classic end points

https://w3c.github.io/webdriver/#get-computed-role

https://w3c.github.io/webdriver/#get-computed-label

AutomatedTester: Bocoup are strong advocates

AutomatedTester: Firefox has internal support. Interactions will check if element is accessible (via the a11y tree)

AutomatedTester: might have been removed

whimboo: We don't have these two endpoints implemented.

foolip: Alex, do you mean something more advanced than the two end points

AlexRudenko: yes, querying the a11y tree with something like CSS selectors

AlexRudenko: finding an element by an a11y property

<gsnedders> AOM

<cb> We have an issue to fetch by a11y label: https://github.com/w3c/webdriver/issues/1441

AutomatedTester: has AOM been implemented anywhere?

<cb> https://github.com/WICG/aom

foolip: AOM as it was a few years ago didn't get implemented. There's now aria* properties on Element and ElementInternals, might not have to do with the a11y tree though.

mathiasbynens: this is probably orthogonal to querying by a11y tree. in Puppeteer we can query by a11y property.

foolip: so do support the Puppeteer support for the a11y selector engine, BiDi would need some additional bits?

AlexRudenko: yes.

<cb> here is the bocoup presentation from TPAC Lyon I think: https://bocoup.github.io/presentation-aria-and-webdriver

<AutomatedTester> ack next

AutomatedTester: How do those Puppeteer APIs impact running tests? Historically enabling things like that would make run times go up a lot.

mathiasbynens: We went through different phases implementing this. Started with slow and wrong, but added CDP-level support to use the accessible tree. There's some overhead, but it's manageable.

mathiasbynens: how common is usage? I don't have numbers, but we hear from partners that it's helpful for projects where class names are randomized.

mathiasbynens: but the button label and accessible name do not changes as much

mathiasbynens: we also have a recorder in devtools, and there we use the accessible names and labels, to make the scripts more robust.

<mathiasbynens> in CDP we added https://chromedevtools.github.io/devtools-protocol/tot/Accessibility/#method-queryAXTree to allow us to query the a11y tree, for e.g. selecting the button with the accessible label “Search”

AutomatedTester: are we good on the agenda?

whimboo: (1) cookies manipulation (2) browser cache (3) HTTP basic auth

whimboo: going through the order again

simonstewart: I think we use network intercept in Selenium to do auth.

AlexRudenko: browser cache is also important for request intercept, because then you usually want the cache to be disabled.

simonstewart: so disabling, not clearing

whimboo: where does a11y fit on the prio list?

AlexRudenko: far down the list

Updated list:

Selenium

* logging

* Boot strap scripts (add bindings)

* network events and interception

* cookie manipulation (not same domain)

* manage the browser cache (clearing of network, local storage)

* user prompts from classic / HTTP authentication (via network events)

* dom events (window/document created, DOM mutation)

* stale elements

* element interactability (eg fixed elements or overlays, in view)

* video recording

Puppeteer

* Boot strap scripts (add bindings)

* Sandbox scripts

* network events and interception

* input

* screenshot

* printing

whimboo: intersection of these lists are bootstrap scripts, network intercept

AutomatedTester: now taking a break, 15 mins

hablich: now that we have rough prio, how do we get to something with broader buy-in, for people who aren't attending right now to also weigh in?

AutomatedTester: I would suggest we post the list to the mailing list and say this is the list, please let us know if you have issues.

simonstewart: I'm fine with that

sadym: what about a PR in the GitHub repo and discuss there?

AutomatedTester: that would also work

mathiasbynens: This list of priorities is great. There are two views into any list which are (1) what's missing from the spec and (2) end-user scenarios. Seems like people are open to milestones expressed as end-user scenarios. Can we commit to doing this?

AutomatedTester: I'll take silence as a tacit yes.

whimboo: I want to second this. We have a list of APIs, can create scenarios/examples for bootstrap scripts and network intercept, for example.

mathiasbynens: So use the list and for each give an example? Other option would be an end-user scenario that might cover multiple.

mathiasbynens: If people are open to it, we can propose a list of scenarios, for folks to review.

AutomatedTester: brilliant idea, action item for mathiasbynens

whimboo: I also like the steps you have for each milestone, pretty helpful

AutomatedTester: anything the WG can do to simplify the lives of people who don't work on browsers, I'll be wholeheartedly in favor of

mathiasbynens: yes, but to be clear it's not just about improving communication, but also for our own clarity, agreement within this group.

whimboo: What should we do with the other items we have? Should we define milestones for them as well?

sadym: We can, but deprioritize

simonstewart: my preference would be define top 3, and when one is done, add another

mathiasbynens: a rolling window approach

whimboo: we all have different implementation speeds

diemol: what I mentioned before might apply to all browsers, if there's something we can do to speed up implementation, we can look into that

AutomatedTester: So let's define a top 3, as issues.

<mathiasbynens> AI(mathias): file GitHub issue with proposed e2e milestones

mathiasbynens: exact implementation aside, what matters is that the information is there.

TPAC

AutomatedTester: I'll be happy to chair remotely. I'll get an agenda page up where we can narrow down agenda.

gsnedders: we have 2 days for BTT

ACTION: David to create agenda

Autumn/Winter F2F

hablich: we'd be looking to organize either in Berlin or Munich, hosting at a Google office.

AutomatedTester: For time, North Americans probably don't want to travel on Halloween etc.

AutomatedTester: Maybe late Oct/Nov timeframe

[super dense discussion about when thanksgivings are]

There's also BlinkOn to consider

https://groups.google.com/a/chromium.org/g/blink-dev/c/16LQXKmAD4s/m/MH9AqBD2AAAJ

November 15-17, 2022

<gsnedders> There's also the WebKit Contributors Meeting to consider, not that I know if we've actually announced anything yet… but that's trying to arrange things to not clash with BlinkOn for the sake of Igalia people involved in both.

<simonstewart> Thanks for the link to BlinkOn. I couldn't find it for the life of me!

AutomatedTester: wrapping up for today. thanks foolip for scribing. I'll share meeting minutes on the mailing list

AutomatedTester: thanks all

all: thanks David

Summary of action items

  1. cb to write a spec issue
  2. David to create agenda
Minutes manually created (not a transcript), formatted by scribe.perl version 192 (Tue Jun 28 16:55:30 2022 UTC).

Diagnostics

Succeeded: i/Apple does not comment/gsnedders sharing https://matrix.org/_matrix/media/r0/download/mozilla.org/774c24c6adccccd812b410301c19d3eedf64a92c/Untitled%202.001.png

Succeeded 6 times: s/Sam Sneddon [:gsnedders]/gsnedders/g

Failed: s/[missing question]/if anything needs to change to support what Simon asked for, if we're not talking about the hub/grid URLs that you can connect to, and then upgrade to BiDi./

Succeeded: s/in the spec/of the spec/

Succeeded: s/yet/yes/

Succeeded: s/in Munich/or Munich/

Maybe present: all, Natasha, rrsagent, Someone