– DRAFT –
WebDriver-BiDi

<jgraham> RRSAgent: stop

<jgraham> RRSAgent: Make minutes v2

Navigation status update

jgraham: this is an FYI
… the initial patches have landed in webdriver and html spec
… one change that occured during code review
… if you abort a navigation then there is a specific event for that
… before it was treated an error

<foolip> The HTML side of this can be seen mostly in https://html.spec.whatwg.org/#navigate, search for "WebDriver BiDi" and references to https://html.spec.whatwg.org/#concept-document-navigation-id

jgraham: From that I am currently rebasing the `reload` command
… for `force reloading` I am doing a lot of hand waving
… for `traversing the history` that requires a change to the html spec
… we need to make sure that the relevant events, like `pageShow` etc are sent
… if there are concerns, please let me know or raise issues otherwise we will see about merging

foolip: traversing history is back and forward?

jgraham: yes, that is correct

jgraham: in a pervious meeting we said that we were going to allow people to move via a delta
… so it's slightly different but aligns well with other automation frameworks
… it's not how CDP works but since the clients are doing this it feels like we should follow them

WebDriver BiDi documentation on MDN

https://bugzilla.mozilla.org/show_bug.cgi?id=1720275

https://github.com/mdn/content/issues/4721

whimboo: normally we speak about the spec in this meeting, however I would like to bring up how we are going to communicate with the browser
… historically we had some docs on webdriver http
… so I would like to propose that we keep this up to date
… historically, MDN used to be a wiki but now this is all done via github using MD

<foolip> This is https://github.com/mdn/browser-compat-data

whimboo: and it would also good to keep track of what features are in which browser
… this is in the MDN BCD repository

<foolip> There is actually some data for WebDriver there: https://github.com/mdn/browser-compat-data/tree/main/webdriver/commands

whimboo: I have linked the meta bug on Mozilla Bugzilla so that we can see what is happening

foolip: I am looking at the MDN content issue
… you talked about webdriver bidi or generally for all webdriver ?

whimboo: I think let's focus on webdriver bidi to start and then we can back fill

foolip: there are 2 things here:
… 1) details of what is doing.
… 2) for web devs...
… 1) we need to make sure that we hit the minimum bar of shipping
… and for 2) I don't think webdevs would care since it's not a public API. Am I correct?

<foolip> I guess this the same thing that I wonder about in https://github.com/mdn/content/issues/4721#issuecomment-844227457 I see now

whimboo: not sure how many peopl.e would benefit from this? For target audience, we wouldn't be aiming for webdev, it's for client devs
… the folks at Selelenium are able to do this quickly
… but outside of them I don't think there are many or who they might be

<foolip> 17 million

jgraham: Yes, this is only for client developers and that there are probably 100 of them compared to web developers
… but there is a spec and we need to think about
… and the spec has a lot more detail than the previous spec or cdp viewer
… one approach could be to have non-normative sections that generate

foolip: even if the audience is narrow it could still be useful
… we just need to be approach how we are going to review it

<foolip> https://github.com/openwebdocs/project/issues/42

<jgraham> RRSAgent: make minutes v2

foolip: and another way is prioritising it via a companies ttech writers or via open web docs
… I have linked to q3 planning
… q3 is the calendar

Side effects of navigator.webdriver

whimboo: THis is an issue that came up in a few places
… I follow in a few places
… in Firefox 88 we made a change that stopped people turning off navigator
… especially folks doing it
… `navigator.webdriver` is in the spec to allow websites to handle instances when this causes problems

see https://www.dropbox.com/sh/2fzqorqhviocerx/AAAApaT_H-pEkM0Rg7D8-utMa?dl=0
… i've seen that web pages stop loading
… there are people getting around with webextensions
… if there are logins, then websites never allow the login page to load using redirects
… do we know if puppeteer, etc have the same issue

<whimboo> https://github.com/microsoft/playwright-python/issues/527#issuecomment-785155678

automatedtester: I believe chrome does allow it to be turned off via a command line flag. It was added at the request of amazon and gmail to stop spamming and people breaching T&Cs.
… I think we should ask the folks from playwright and other to move to us rather than allow them to help breach T&Cs

jgraham: I had a look at Chromium source, by default when you set the remote debug port then it sets the `navigator.webdriver`... so puppeteeer is the same
… I think the agreement that we are not going to change this behaviour
… if people are motivated enough they can get around that
… do we want to make sure that feature is unforgeable?

<foolip> Chromium briefly used [LegacyUnforgeable] here: https://chromium-review.googlesource.com/c/chromium/src/+/2556519

jgraham: on the cookie issue, that is a real issue and that is fixed in bidi
… and I think is unrelated

jimevans: as another data point, there are many commercial sites that work very hard to block automation against the site

<foolip> Also, Puppeteer will be default have `navigator.webdriver` be true. I found https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth which does a lot of stuff to avoid being detected as automated.

jimevans: and there are companies that sell software
… the webdriver flag is a good thing and we don't want webdriver to support a use case that allows scaping sites that don't want to be scraped

foolip: it does turn it on by default with puppeteer
… there was a unforgeable added it in the past
… but then people wanted to copy the entire `navigator` object and then popped the webdriver key
… if firefox being too strict or too lax?

whimboo: people are complaining that it is too strict

jgraham: People are upset we made them have to work hard to do bad things

<foolip> `Object.defineProperty(navigator, 'webdriver', { value: false })`still works, right?

<jgraham> Yes

Action: automatedtester to speak to other frameworks about supporting `navigator.webdriver`

<whimboo> i think we have `true` or `undefined`

<jgraham> We previously had a pref that just prevented the webdriver property being defined at all, because there was concern about compat risk when first shipping

Refactoring work on sessions

jgraham: people might have noticed that I put a few large PR s to both specs
… the motivation is ... we have previous discussed about how people might conenct to the browser
… and we want to make sure that we support direct connections or upgrades
… the other motivation is to allow multiple connections (1 tool for perf, 1 tool for automation)
… there are going to be options about what people can do
… the patches allow people to pick and choose what they want to do here
… [discusses details in patch]
… this is largely non-normative changes

and a few spec bug fixes

whimboo: is this for http or for bidi?

jgraham: it's for both. bidi relies on changes into http

<jgraham> RRSAgent: make minutes v2