16:43:19 <RRSAgent> RRSAgent has joined #webdriver
16:43:19 <RRSAgent> logging to https://www.w3.org/2020/10/27-webdriver-irc
16:43:58 <AutomatedTester> Meeting: Browser Testing and Tools WG @ TPAC - Day 2
16:44:11 <AutomatedTester> Chair: AutomatedTester
16:44:41 <AutomatedTester> Agenda: https://www.w3.org/wiki/WebDriver/2020-TPAC
16:44:57 <AutomatedTester> Scribe: David Burns
16:45:17 <AutomatedTester> ScribeNick: AutomatedTester
16:45:23 <AutomatedTester> RRSAgent: nolisten
17:01:33 <AutomatedTester> present+
17:01:55 <jgraham> RRSAgent: make logs public
17:02:01 <jgraham> RRSAgent: make minutes v2
17:02:01 <RRSAgent> I have made the request to generate https://www.w3.org/2020/10/27-webdriver-minutes.html jgraham
17:02:14 <jimevans> jimevans has joined #webdriver
17:02:21 <shengfa> present+
17:02:28 <jimevans> present+
17:02:29 <simonstewart> simonstewart has joined #webdriver
17:02:30 <jgraham> present+
17:02:32 <brwalder> present+
17:02:37 <simonstewart> present+
17:02:39 <Honza> present+ (jodvarko in the invite)
17:02:40 <cb> present+
17:03:24 <jgraham> q?
17:03:46 <AutomatedTester> https://www.w3.org/wiki/WebDriver/2020-TPAC
17:04:15 <zghadyali> zghadyali has joined #webdriver
17:06:04 <AutomatedTester> Topic: BiDi Bootstrap scripts
17:06:36 <AutomatedTester> https://github.com/w3c/webdriver-bidi/issues/63
17:08:35 <AutomatedTester> jgraham: before we start on that it might be worth noting that I raised an issue from yesterdays discussion issue 63
17:08:59 <jgraham> GitHub Topic: https://github.com/w3c/webdriver-bidi/issues/65
17:10:20 <AutomatedTester> brwalder: Bootstrap scripts execute as the first thing that is executed in a realm as it is created. It allows them to introspect the realm or setup the page for the items needed for testing
17:11:04 <AutomatedTester> ... in a bidi world it would be a good have a mechanism for bootstrapping scripts and send it back to local end
17:11:39 <jgraham> q+
17:11:54 <simonstewart> q+
17:12:14 <AutomatedTester> ... there is a proposal that allows us to bootstrap a script assigned to a url pattern and when a pattern is matched it would inject the script on that realm
17:12:39 <AutomatedTester> ... and if we needed to inject it was on a realm ID it would create a race condition that we would like to avoid
17:13:14 <AutomatedTester> ... The script would allow you work on newly created realms
17:13:37 <AutomatedTester> ... [describing how a JS port and JS works in this context]
17:14:08 <AutomatedTester> ... the scenarios that this enables is listening for events on a page and send them from the bootstrap script to the local end
17:14:33 <AutomatedTester> ... and allows us to support items without having to specifically support items
17:14:56 <AutomatedTester> ... and we can have a scenario for handling JS errors and cause a test to end quicker
17:15:11 <AutomatedTester> q?
17:15:17 <AutomatedTester> ack jgraham
17:15:48 <AutomatedTester> jgraham: Overall this is a good idea, and it's a powerful idea. I have had cases where webdriver has not be able to do this.
17:16:07 <AutomatedTester> ... there are a number of open questions and these are partly script execution and this feature
17:16:27 <AutomatedTester> ... is this only for scripts associated with documents only
17:16:36 <AutomatedTester> ... e.g. worklets could be very difficult
17:17:32 <AutomatedTester> ... The other item to wonder about is this executes before the page has loaded. DO we need to have guarantees around the shape of the document/DOM
17:17:40 <AutomatedTester> ... and are scripts sandboxed?
17:18:01 <AutomatedTester> ... A precedence for here are extensions
17:19:04 <AutomatedTester> ... The final thing is around the value communication issue. The message ports is ok but that only works with certain values
17:19:21 <AutomatedTester> ... and we should maybe model it like a DOM API?
17:20:23 <AutomatedTester> brwalder: Should this work in a document or all realms? The proposal doesn't mention this just yet
17:20:54 <AutomatedTester> ... and this touches slightly on the discussion yesterday on how script execution works
17:21:17 <AutomatedTester> ... so yes we need to have more expressive pattern matching
17:21:45 <AutomatedTester> ... re: sandboxing there are cases where we want to be limited to the sandbox and sometimes we don't
17:21:57 <AutomatedTester> ... the proposal already handles this
17:22:27 <AutomatedTester> ... re: message port... if we are using message porting as specified won't work here. I think we need to have a way to do the serialisation better
17:22:34 <AutomatedTester> q?
17:22:37 <AutomatedTester> ack simonstewart
17:23:02 <AutomatedTester> simonstewart: I second the proposal on the serialization
17:23:20 <AutomatedTester> ... is the message port is just a 1-way comms?
17:23:51 <AutomatedTester> brwalder: I missed mentioning this earlier, there would be a specific command for this
17:24:24 <AutomatedTester> simonstewart: with worklets... they are supposed to be high performance... how will this impact this performance here?
17:25:08 <AutomatedTester> jgraham: my thought is that we start by making this on JS Realms that are part of a Window global and then go to realms as the use cases present themselves
17:25:30 <AutomatedTester> simonstewart: I can see people wanting workers and we can add as the need but worklets less likely
17:25:52 <AutomatedTester> ... though we can see about extensions in later versions of the specification
17:26:00 <AutomatedTester> jgraham: yes, we can easily do that.
17:26:24 <AutomatedTester> ... so the question to implementors, are there any concerns in this area?
17:26:35 <AutomatedTester> ... and hopefully it will be like extensions
17:27:04 <AutomatedTester> brwalder: this is certainly possibly... CDP has the bootstrapping but its for ALL realms
17:27:24 <AutomatedTester> ... and is possible
17:27:39 <AutomatedTester> brrian: and in webkit it is possible
17:28:36 <AutomatedTester> Resolution: brwalder to create a formal PR for this area.
17:28:59 <AutomatedTester> jgraham: we want to make sure that this works with general script execution
17:29:15 <AutomatedTester> ... the open issues on general script execution are :
17:29:31 <AutomatedTester> ... a) how will this work on sandboxing
17:30:07 <AutomatedTester> ... b) what the API for comms will look like? Message Ports? DOM API?
17:31:26 <AutomatedTester> github-bot:  end topic
17:32:29 <AutomatedTester> Topic: Navigation
17:33:10 <jgraham> GitHub Bot: https://github.com/w3c/webdriver-bidi/issues/43
17:33:30 <AutomatedTester> Github topic: https://github.com/w3c/webdriver-bidi/issues/43
17:33:52 <whimboo> present+
17:33:53 <AutomatedTester> jgraham: the point of this topic, is how should navigation in the bidi topic
17:34:22 <AutomatedTester> ... [reads from the comment in the issue]
17:35:05 <AutomatedTester> q?
17:36:08 <drousso> drousso has joined #webdriver
17:36:11 <drousso> present+
17:36:31 <simonstewart> q+
17:37:15 <AutomatedTester> ack simonstewart
17:37:44 <brwalder> q+
17:37:52 <AutomatedTester> simonstewart: From the comments in the issue, when the navigation starts it returns automatically and then people will need to listen for events
17:37:56 <jimevans> q+
17:38:25 <AutomatedTester> ... I think that it would be good to have webdriver http reformulated to work ontop of bidi here.
17:38:31 <AutomatedTester> ack brwalder
17:39:02 <AutomatedTester> brwalder: From reading the comments, I imagine this a command that doesnt return instantly and then listening for events
17:39:39 <AutomatedTester> ... so people want to use this then use the original page load strategies from webdriver http
17:40:18 <AutomatedTester> ... as doing it as instant return and listening for events is making things very complex for what should be thought of by users as simple
17:40:22 <jgraham> q+
17:40:25 <AutomatedTester> ... and I agree with simonstewart here
17:41:08 <AutomatedTester> simonstewart: I like the ability passing in the page load strategy
17:41:22 <mathiasbynens> +1 global state (setting a default page load strategy) is evil
17:41:22 <AutomatedTester> jgraham: global state is evil and we should avoid it
17:42:05 <AutomatedTester> brwalder: I wasn't thinkking of a global state but the client would be sending this per commande
17:42:09 <AutomatedTester> q?
17:42:14 <AutomatedTester> ack jimevans
17:42:35 <AutomatedTester> jimevans: is there mileage in leveraging the DOM events, like puppeteer?
17:43:08 <AutomatedTester> ... It would be relatively easy for a client library to descript it's page load strategy based on DOM Events
17:43:24 <AutomatedTester> ... WebDriver HTTP kinda does this for it's page load strategy
17:44:06 <simonstewart> q+
17:44:34 <AutomatedTester> ... would allow clients to become opinionated in this way
17:44:43 <AutomatedTester> ack simonstewart
17:45:14 <AutomatedTester> simonstewart: brwalder highlighted that if you wanted to avoid the dance of setting things up correctly
17:45:28 <AutomatedTester> ... and that it would be a lot of work for clients
17:45:38 <simonstewart> q-
17:46:06 <AutomatedTester> jgraham: my initial take is page load strategy does things simply
17:46:28 <AutomatedTester> ... and as whimboo will attest, navigation is full of edge cases
17:46:38 <shengfa> q+
17:46:41 <AutomatedTester> ... [describes a use case]
17:46:43 <brrian> q+
17:46:50 <AutomatedTester> ack jgraham
17:47:04 <brwalder> q+
17:47:11 <AutomatedTester> ... and we need to make sure that the load event matches where you're expecting
17:47:25 <AutomatedTester> ... and what does navigate return?
17:47:33 <AutomatedTester> ... in CDP you get a loader event
17:47:50 <AutomatedTester> ... I dont know if this is described in the platform
17:48:04 <AutomatedTester> q?
17:48:17 <AutomatedTester> ack shengfa
17:48:32 <AutomatedTester> shengfa: I would like to comment on how CHromedriver does it
17:48:40 <AutomatedTester> ... there are loads of edge cases here
17:48:55 <AutomatedTester> ... we try keep track of all the frames
17:49:07 <AutomatedTester> ... and we look at frame start/stop loading events
17:49:20 <AutomatedTester> ... and checking readyState
17:49:38 <AutomatedTester> ... I don't know if we need to specify things better in bidi
17:49:53 <AutomatedTester> ... so do we want to expose the page loading strategy or all the events
17:50:06 <AutomatedTester> ... for exposing the events, we can specify the webdriver events
17:50:20 <jgraham> q+
17:50:29 <AutomatedTester> ... or expose the events we think are relevant to the client
17:50:54 <AutomatedTester> ... and regarding the workflow, the navigation events would just ack and then listening for events
17:51:13 <AutomatedTester> ... and the navigation is just a "subscribe to these events" type call
17:51:17 <AutomatedTester> q?
17:51:26 <AutomatedTester> ack brwalder
17:51:35 <AutomatedTester> ack brwalder
17:51:41 <AutomatedTester> ack brrian
17:51:54 <jgraham> q+ brwalder
17:51:58 <jgraham> q- later
17:52:11 <AutomatedTester> brrian: from safari there are lot of junk in this area
17:52:45 <AutomatedTester> ... and people do want events
17:53:06 <AutomatedTester> ... and the happy path should look nicely to them
17:53:18 <AutomatedTester> ... and I don't want to be exposing the loaders to the clients
17:53:36 <AutomatedTester> ... and we should make sure that we solve the correct use cases
17:53:42 <AutomatedTester> q?
17:53:52 <AutomatedTester> ack brwalder
17:54:13 <AutomatedTester> brwalder: It sounds like there are 2 legitimate paths for end users
17:54:24 <AutomatedTester> ... a happy path -> THe page just is loaded
17:54:31 <simonstewart> q+
17:54:47 <AutomatedTester> ... and the more exotic usecases and allow people to do more "advanced"
17:55:09 <AutomatedTester> ... and we can accept the page load strategy and we do "best case"
17:55:32 <AutomatedTester> ... and then provide an escape hatch and allow immediate return and listen for events
17:55:42 <AutomatedTester> ... and I think we should support both
17:55:45 <AutomatedTester> q?
17:55:52 <drousso> q+
17:55:56 <AutomatedTester> ack jgraham
17:56:11 <AutomatedTester> jgraham: there is definitely a concensus on making sure th
17:56:29 <AutomatedTester> ... what WEbdriver http already does and then for what ee
17:56:32 <AutomatedTester> events.
17:57:00 <AutomatedTester> ... [describes CDP implementation]
17:57:46 <AutomatedTester> ... we should look into the feasibility that if you subscribe events to a navigation so that we know that things are tied back to the initial command
17:58:09 <AutomatedTester> ... and we don't want people to have to guess that an event came from the command... hopefully
17:58:11 <AutomatedTester> q?
17:58:16 <AutomatedTester> ack simonstewart
17:58:46 <AutomatedTester> simonstewart: one thing we forgot so far... if this coming from a Selenium cloud providers
17:59:15 <drousso> q-
17:59:21 <AutomatedTester> ... and it would be good to make sure that we dont send too much data back for those clients as we have 2 internets in the way
18:00:07 <AutomatedTester> q?
18:01:23 <AutomatedTester> Resolution: Specify navigation taking a page load strategy parameter to match WebDriver HTTP and investigate the page load life cycle events and exposing those
18:02:12 <AutomatedTester> github-bot: end topic
18:02:27 <AutomatedTester> Topic: Logging
18:02:59 <AutomatedTester> jgraham: One of the value add module proposals is a logging module
18:03:26 <AutomatedTester> ... we know that Selenium have asked this as a priority
18:03:48 <AutomatedTester> ... so it would be good get requirements
18:03:55 <AutomatedTester> github topic: https://github.com/w3c/webdriver-bidi/issues/45
18:04:29 <AutomatedTester> ... is this a good enough or do we need to extend it?
18:05:12 <AutomatedTester> ... do we need to get browser internal logs? Intermediary logs?
18:05:59 <AutomatedTester> ... and for clarification network logging is not part of this proposal and we can add it to a newer API
18:07:16 <AutomatedTester> q+
18:08:28 <john_chen> john_chen has joined #webdriver
18:09:57 <jgraham> q+
18:10:01 <jgraham> ack AutomatedTester
18:10:30 <AutomatedTester> AutomatedTester: I have a few questions around this around filtering of logging as cloud providers could DDoS clients and then how granular is the data?
18:10:39 <AutomatedTester> ack jgraham
18:11:19 <AutomatedTester> jgraham: the 2nd question : we allow people in the spec currently to handle this in browser context or realm
18:11:48 <jimevans> q+
18:12:01 <AutomatedTester> ...and for the first question for this we don't have a mechanism for this at the moment?
18:12:07 <AutomatedTester> ... and we dont have a precendent in this area to learn from
18:13:05 <AutomatedTester> AutomatedTester: I was thinking of the difference between console.log vs console.error and only getting the latter
18:13:09 <AutomatedTester> q?
18:13:32 <AutomatedTester> jimevans: At a high level here that I hear from selenium users is
18:13:51 <AutomatedTester> ... they want to get console.log between 2 time points
18:14:06 <AutomatedTester> ... and if there are unhandled JS errors, notify me
18:14:28 <jgraham> RRSAgent: make minutes v2
18:14:28 <RRSAgent> I have made the request to generate https://www.w3.org/2020/10/27-webdriver-minutes.html jgraham
18:14:38 <AutomatedTester> ... at a less frequency is I want to get the performance logging that I see in my devtools console
18:15:08 <jgraham> q+
18:15:14 <jgraham> ack jimevans
18:15:14 <AutomatedTester> ... at a bare minimum we need to do the console logs and unhandled exceptions
18:15:33 <AutomatedTester> ... and I agree that there are concerns around chattiness
18:15:52 <AutomatedTester> ... from my experience of using CDP, those 2 things are similar to me
18:16:17 <AutomatedTester> ... it would be good to find out from a driver to get what logging they support
18:16:36 <AutomatedTester> ... and we need to agree on the general shape of the log entries
18:16:38 <AutomatedTester> q?
18:17:09 <cb> q+
18:17:10 <AutomatedTester> jgraham: The performance is in scope for the spec  but not high priority
18:17:36 <AutomatedTester> ... since perf is browser specific... it's important but not low hanging fruit
18:17:58 <simonstewart> q+
18:18:21 <jgraham> ack jgraham
18:18:23 <AutomatedTester> ... the main issue that I am hearing is what type of filtering is there
18:18:29 <AutomatedTester> ack cb
18:19:15 <AutomatedTester> cb: from my experience implementing perf tooling it would be hard for us to spec
18:19:48 <AutomatedTester> ... it would be good to have the log entry to be more generic in where it's cominf from
18:19:52 <AutomatedTester> q?
18:19:56 <AutomatedTester> ack simonstewart
18:20:20 <drousso> q+
18:20:29 <AutomatedTester> simonstewart: this is bikeshedding. Having a logging module shouldnt be there. It should be on a JavaScript module
18:20:42 <simonstewart> Or a `Console` module
18:20:44 <AutomatedTester> ... and happy to take this bikeshedding to the proposal later
18:20:47 <jgraham> s/JavaScript/JavaScript or Console/
18:20:55 <AutomatedTester> q?
18:21:04 <AutomatedTester> ack drousso
18:21:53 <AutomatedTester> drousso: as a friendly warning there are always new console messages... and filtering should be done via an event
18:22:09 <AutomatedTester> ... this is how webinspector works
18:22:39 <AutomatedTester> jgraham: luckly this is what is in the proposal but thank you for the warning on new items being added
18:23:15 <AutomatedTester> Resolution: Take proposal in issue and turn into spec prose
18:23:20 <AutomatedTester> github-bot: end topic
18:23:57 <AutomatedTester> Topic: Putative modules
18:24:03 <simonstewart> https://github.com/SeleniumHQ/selenium/blob/trunk/java/client/src/org/openqa/selenium/devtools/idealized/Domains.java
18:24:50 <AutomatedTester> simonstewart: In the selenium project we need to multiple CDP implementations in each of the Chromium Drivers
18:25:09 <AutomatedTester> ... and I have put the link above that shows this "idealised" API
18:25:47 <AutomatedTester> ... jimevans has correctly described the naming isn't the best but we are homing in
18:26:21 <AutomatedTester> ... and we need to think of ways of naming the modules so that they don't clash with CDP or webinspector
18:26:23 <jgraham> q+
18:26:43 <AutomatedTester> q?
18:26:49 <AutomatedTester> ack jgraham
18:27:12 <AutomatedTester> jgraham: In terms of implementing this in gecko, we agree it's an issue.
18:27:24 <AutomatedTester> ... we partial implementation of CDP
18:27:40 <AutomatedTester> ... and we want to reuse as much as possible for webdriver Bidi
18:28:16 <simonstewart> q+
18:28:28 <AutomatedTester> ... and we want make sure that we dont have a case where we are dancing around names that are obvious to use but can't use for CDP using it
18:28:43 <AutomatedTester> ... and we dont want anything that will make migration is hard
18:28:56 <AutomatedTester> ... and we dont want to also be super careful about how we name things
18:28:59 <AutomatedTester> q?
18:29:04 <simonstewart> https://trac.webkit.org/browser/webkit/trunk/Source/JavaScriptCore/inspector/protocol
18:29:09 <AutomatedTester> ack simonstewart
18:29:51 <AutomatedTester> simonstewart: a lot of the conversation is around CDP and I have put a link to JSC protocol
18:30:21 <AutomatedTester> ... it doesn't have the sprawl that CDP has but doesn't have all the things
18:30:40 <AutomatedTester> ... jgraham have you found a way to not clash and reuse
18:30:55 <AutomatedTester> jgraham: it's a question in the air, we are currently discussing
18:31:28 <AutomatedTester> ... we see this a real problem. We know we can't always use reuse code but want to where possible
18:32:07 <AutomatedTester> ... we can see where we get. I want to make sure we don't get to a place where we can't use Console where it is the best name
18:32:30 <AutomatedTester> ... and I think from a Firefox point of view is that you can opt into a specific protocol
18:32:50 <simonstewart> q+
18:33:03 <brwalder> q+
18:33:16 <AutomatedTester> ... I can see where people would want to make sure that they can use both protocols because implementations are not complete
18:33:27 <AutomatedTester> ... we need to make sure it's not too hacky either
18:33:33 <AutomatedTester> q?
18:33:39 <AutomatedTester> ack simonstewart
18:34:39 <AutomatedTester> simonstewart: as a concrete proposal we don't constrain ourselves and we just namespace it. That way we can have the name we want and then they get mapped internally to the correct place
18:34:42 <drousso> +1, we should use good names for the sake of good naming, not because something else already uses it
18:34:49 <AutomatedTester> q?
18:34:50 <jgraham> Also +1
18:35:01 <AutomatedTester> ack brwalder
18:35:15 <AutomatedTester> brwalder: I was going to say what simonstewart said
18:35:59 <AutomatedTester> ... if we keep propietrary and bidi on separate sockets then things will be simple
18:36:09 <AutomatedTester> ... we shouldnt reuse the same channel
18:36:43 <AutomatedTester> ... [describes  how we could do extension commands
18:36:46 <AutomatedTester> q?
18:39:07 <AutomatedTester> Topic: Navigation during commands for WebDriver HTTP
18:39:56 <jgraham> fission == site isolation
18:40:25 <AutomatedTester> whimboo: The Mozilla Fission project (site isolation) when are doing there are some unknown issues
18:40:39 <AutomatedTester> ... so when there is a click or actions
18:40:52 <AutomatedTester> ... and in execute script
18:41:04 <simonstewart> q+
18:41:13 <AutomatedTester> ... and when these things like happen what should we doing
18:41:31 <AutomatedTester> jgraham: for clarity fission is site isolation
18:42:18 <AutomatedTester> ... so we want to know what to do if there a long running task and the page loads then the state could all be lost
18:43:03 <AutomatedTester> ... so should we store in the parent process or abort
18:43:44 <AutomatedTester> whimboo: executescript is the worst as we inject the whole script so we don't know where it was when the actor is "flipped"
18:43:49 <AutomatedTester> q?
18:43:58 <AutomatedTester> ack simonstewart
18:44:25 <AutomatedTester> simonstewart: it depends... some commands needs to survive a new page
18:45:00 <AutomatedTester> ... but with actions we never wanted to support it and it's legitimate to error out to the user
18:45:19 <AutomatedTester> ... we may want store state around mouse and keyboard
18:45:58 <AutomatedTester> ... in execute script we should error because people are probably not expecting it
18:46:25 <AutomatedTester> ... but I think we need to go through the commands and do it case by case which I am happy to at a later stage
18:46:29 <AutomatedTester> q?
18:47:13 <AutomatedTester> jgraham: I think this makes sense and we need to update the spec text
18:47:41 <AutomatedTester> ... and we may want to see how other implementations handle this
18:48:31 <AutomatedTester> ... if people are not aware of use cases then we can go have a look
18:50:36 <Hexcles> Hexcles has joined #webdriver
18:51:11 <jgraham> RRSAgent: stop