00:13:49 RRSAgent has joined #webdriver 00:13:49 logging to https://www.w3.org/2019/09/19-webdriver-irc 00:14:34 CalebRouleau has joined #webdriver 00:14:36 RRSAgent, this meeting spans midnight 00:14:43 RRSAgent make logs public 00:14:52 RRSAgent make minutes 00:16:00 RRSAgent create the minutes v2 00:17:30 RRSAgent, make logs public 00:17:35 RRSAgent, create the minutes v2 00:17:35 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html jgraham 00:19:49 Zakim has joined #webdriver 00:23:34 jwer_ has joined #webdriver 00:24:09 bwalderman has joined #webdriver 00:24:14 plh has joined #webdriver 00:24:18 https://mit.webex.com/mit/j.php?MTID=mf0c6a95eedfa61b9e4d6dbdc08e3798a 00:24:29 Meeting number: 00:24:30 644 219 299 00:24:30 Password: 00:24:30 w3c 00:27:14 https://docs.google.com/presentation/d/1jKiPIrbIH6RdJE15nYWA-xr1DDVuYAeurpfhu6Dug-c/edit#slide=id.g5e27cbf49c_0_0 00:28:01 simonstewart has joined #webdriver 00:28:28 G'day, one and all 00:29:46 Meeting: Browser Tools- and Testing WG, Day 1, TPAC 2019, Fukuoka 00:29:54 Present+ 00:30:06 zghadyali has joined #webdriver 00:30:20 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html ato 00:31:32 projector_webdriver has joined #webdriver 00:31:42 https://mit.webex.com/mit/j.php?MTID=mf0c6a95eedfa61b9e4d6dbdc08e3798a 00:31:50 Present+ 00:31:55 present+ 00:32:24 Boaz has joined #webdriver 00:32:25 present+ 00:32:25 present+ 00:32:27 twisniewski has joined #webdriver 00:32:32 miketaylr has joined #webdriver 00:32:44 JohnChen has joined #webdriver 00:32:53 agenda: https://www.w3.org/wiki/Webdriver/2019-TPAC 00:33:24 titusfortner has joined #webdriver 00:34:22 present+ David Burns, Mozilla 00:34:37 present+ 00:34:50 present? 00:34:52 present+ 00:34:59 present+ 00:34:59 rrsagent, make minutes 00:34:59 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html JohnJansen 00:35:00 Seat map: https://docs.google.com/spreadsheets/d/1mZGuKR4SR6HGhqf4wTJQg6eJQBkqnP4G3ibkPTIP_1Q/edit#gid=0 00:35:00 present+ 00:35:01 present+ 00:35:04 present+ 00:35:09 present+ 00:35:14 present+ 00:35:16 lukebjerring has joined #webdriver 00:35:16 present+ 00:35:21 zcorpan has joined #webdriver 00:35:24 present+ 00:35:28 present+ 00:35:29 present+ 00:35:42 present+ 00:35:43 drousso has joined #webdriver 00:35:46 present+ 00:35:49 present+ 00:35:51 present+ 00:36:07 https://docs.google.com/spreadsheets/d/1mZGuKR4SR6HGhqf4wTJQg6eJQBkqnP4G3ibkPTIP_1Q/edit#gid=0 00:36:44 rrsagent, make minutes 00:36:44 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html JohnJansen 00:37:23 heejin has joined #webdriver 00:37:34 https://www.w3.org/wiki/Webdriver/2019-TPAC 00:38:07 https://docs.google.com/presentation/d/1jKiPIrbIH6RdJE15nYWA-xr1DDVuYAeurpfhu6Dug-c/edit#slide=id.g5e27cbf49c_0_0 00:38:18 proposed items from me: https://www.w3.org/wiki/Webdriver/2019-TPAC 00:39:05 Topic: Continuous Standards Development 00:39:27 https://docs.google.com/presentation/d/1jKiPIrbIH6RdJE15nYWA-xr1DDVuYAeurpfhu6Dug-c/edit#slide=id.g5e27cbf49c_0_0 00:39:32 present+ 00:39:33 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html ato 00:39:36 present+ David Burns 00:58:19 zcorpan has joined #webdriver 01:00:51 Hexcles has joined #webdriver 01:02:47 Topic: Agenda 01:02:53 ScribeNick: ato 01:02:55 Topic: Agenda 01:03:02 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html ato 01:09:38 Chair: AutomatedTester 01:09:47 s/David// 01:09:50 s/Burns// 01:15:52 jugglinmike has joined #webdriver 01:23:14 Hexcles has joined #webdriver 01:53:44 drousso has joined #webdriver 01:54:51 Hexcles has joined #webdriver 01:55:34 bwalderman has joined #webdriver 01:56:33 titusfortner has joined #webdriver 01:57:43 Agenda prioritised: https://www.w3.org/wiki/Webdriver/2019-TPAC 01:57:48 diemol has joined #webdriver 01:57:50 RRS, make minutes 01:58:18 RRSAgent, make minutes 01:58:18 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html JohnJansen 01:59:37 s/David_Burns// 01:59:52 s/Mozilla// 02:01:15 Boaz has joined #webdriver 02:01:21 present+ 02:01:33 present+ 02:01:38 mmerrell has joined #webdriver 02:01:39 CalebRouleau has joined #webdriver 02:01:43 ScribeNick: mmerrell 02:02:04 Seat map: https://docs.google.com/spreadsheets/d/1mZGuKR4SR6HGhqf4wTJQg6eJQBkqnP4G3ibkPTIP_1Q/edit#gid=0 02:02:22 present+ Diego Molina, Sauce Labs 02:02:35 present+ 02:02:53 you cannot use a comma in your "present" 02:02:55 Topic: Bi-directional communication 02:03:04 it looks like Sauce Labs is here :-) 02:03:17 In a sense, they are 02:03:28 jgraham: we need timeslots for the discussions 02:03:46 AutomatedTester: discuss until 12:30, at which point we'll break for lunch 02:03:49 RRSAgent, make minutes 02:03:49 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html JohnJansen 02:04:30 s/Sauce_Labs// 02:04:42 s/Diego_Molina// 02:04:57 agenda: https://docs.google.com/document/d/1gUm7Be-akW2-4mjr15cnZlzwoAfOlfL7b3tWCDrb1Jg/edit 02:06:17 Boaz: there's another agenda topic: articulating a workflow or set of guidelines for how to use the interfaces to create test materials 02:06:25 Boaz: specifically for browser vendors 02:06:35 Boaz: propose discussing that for Friday afternoon 02:07:09 AutomatedTester: lunch 12:30-1:30, break 3:30 - 4, at which point we do the Aria Driver demo 02:07:16 nao_watanabe_access has joined #webdriver 02:07:55 npm has joined #webdriver 02:08:37 jgraham: "bi-di" is short for "bi-directional WebDriver" 02:08:50 jgraham: 16:30 should start Shadow DOM discussion 02:09:05 jgraham: first thing Friday will be long-running session 02:10:29 simonstewart: Custom Selectors will be a 30 minute discussion 02:11:06 zcorpan has joined #webdriver 02:12:12 https://docs.google.com/document/d/1gUm7Be-akW2-4mjr15cnZlzwoAfOlfL7b3tWCDrb1Jg/edit# 02:13:41 Boaz: there is interest in creating bi-directional communication functionality added to the spec 02:13:55 AutomatedTester: 3 elements: use cases, transports, APIs 02:14:06 AutomatedTester: this is not standardizing on CDP, this is faciilitating test automation 02:14:24 AutomatedTester: frame use cases around that principle, we wont' get bogged down in historical implementations 02:14:33 AutomatedTester: focus on what this group needs for a bi-di tool 02:15:05 AutomatedTester: starting with use cases, we have input from Sauce Labs and others 02:15:07 https://mit.webex.com/mit/j.php?MTID=mf0c6a95eedfa61b9e4d6dbdc08e3798a 02:15:14 q+ 02:15:19 q? 02:15:24 jgraham: [explains how to use the queue for speakers] 02:15:34 q+ 02:15:52 q+ 02:15:55 q? 02:16:01 ack simonstewart 02:16:01 q? 02:16:22 simonstewart: use cases: people use Cypress and Puppeteer in certain ways that should inform this feature 02:16:31 simonstewart: the ability to wait for an event in the DOM 02:16:43 simonstewart: being notified of those events allows stable tests 02:16:53 simonstewart: logging what's going on in the browser, including console and JS errors 02:17:23 simonstewart: people really like to fail tests on ANY JS error, which they do by loading a page and executing a script, but which can cause race conditions 02:17:36 simonstewart: also, stubbing out back-ends 02:18:02 simonstewart: people are trying to record traffic, then simulate the back-end operations. Supporting that woul dhelp 02:18:14 simonstewart: CDP gives you a full-page screenshot option 02:18:21 ack JohnChen 02:18:23 simonstewart: [4 use cases total] 02:18:39 JohnChen: people like the features of Puppeteer, and want those features in WebDriver 02:19:10 JohnChen: e.g. intercepting HTTP requests, and modifying them dynamically 02:19:28 CalebRouleau: is that the same as Simon's point about stubbing back-end requests 02:19:34 jgraham: it's the same thing 02:19:43 It's a better formulation of what I said 02:19:58 JohnChen: users need to be able to get notified without having to poll the page 02:20:03 q? 02:20:10 ack cb 02:20:51 cb: at Sauce Labs, we have the ability to bypass the crhomedriver, which allows us to allow custom commands (throttling CPU, simulating performance issues, etc) 02:21:41 q+ 02:21:47 cb: SL performance product also requires these internal accesses, and incorporating that ability into the protocol would help greatly 02:22:09 ack jgraham 02:22:12 cb: meaningful error messages, access to internal devtool protocols, all would be helpful for customers to be able to write better tests 02:22:53 jgraham: is it important to SL that you can get the same profiling data out of all browsers, or can these things be more browser-specific? 02:23:14 jgraham: FF has different data from Chrome, which greatly complicates the possibility to do this 02:23:30 cb: we just need to get the same kind of information, understanding that it will be different 02:23:38 q? 02:23:52 cb: we need to get as much information about the AUT as possible, no matter how we can get it (page load timings, logs, etc) 02:24:18 jgraham: for these introspection APIs, these use cases *are* satisfied even though there isn't uniformity among the browsers 02:24:18 q+ 02:24:22 ack ato 02:24:22 ack ato 02:24:41 ato: one more use case: from WPT, a question came up 02:25:16 ato: in order to be able to write good tests for the browsers, functionality could be exposed in this manner that would help browser developers as well as end users 02:25:20 rrsagent, this meeting spans midnight 02:25:41 jgraham: do you mean event-based user interactions? 02:26:52 ato: there's an interest from spec authors who want to expose bi-di APIs, which is currently made difficult by WebDriver 02:27:23 jgraham: there are other contraints here: some of these "mocking" features are very difficult to implement in terms of a command-response protocol 02:28:09 q? 02:28:10 Boaz: I've invited Reilly [from the Chrome team] to discuss this particular thing Friday afternoon 02:28:34 Boaz: this will allow us to state these use cases to one of the most important stakeholders 02:28:35 q+ 02:29:05 jgraham: to ato: is this for developer ergonomics for the browser developers, or are they being prevented from developing these things? 02:29:45 ato: one example is the Gutenberg project, which is a test suite--this should be used as a target for the kinds of features being asked for 02:30:00 https://github.com/WordPress/gutenberg/tree/master/packages/e2e-tests/specs 02:30:27 ato: the test suite isn't doing anything particularly challenging, but it would be a different programming model. There's a desire from modern web developers to have an API that allows async comms 02:30:30 q+ 02:30:59 ato: the one thing the test suite does, where WD would be helpful, is in the cases of complicated keyboard interactions 02:31:09 ato: these cases woul dbe better expressed in WebDriver 02:31:23 q? 02:31:23 ato: it exposes and surfaces browser-specific interactions 02:31:26 ack cb 02:32:00 cb: for SL customers, bi-di mechanism would allow for much better state management of tests 02:32:08 q? 02:32:09 simonstewart: that's risky, because so many things can go wrong 02:32:39 cb: agreed, but those things that can go wrong are commonly network-based 02:33:43 ato: another use case: people writing clients for automation could benefit from other features, e.g. dynamic changes to iFrame or documents 02:34:04 ato: these events are important when writing clients 02:34:38 ato: another use is performance logs: people need to know about the performance logs for internal timings 02:34:53 ato: these things are inherently browser-specific, and they already exist 02:35:01 AutomatedTester: moving on, to simonstewart 02:35:03 ack simonstewart 02:35:56 simonstewart: summary of use cases: log (performance, console, javascript errors), network interception, and stubbing out req/res, mutation/observation in waiting for events or new contexts 02:36:39 q+ 02:36:48 jgraham: we should not assume this is the complete set of use cases, and we should get more data from other tooling 02:37:07 Boaz: we should also consider web-USB and others 02:37:34 Boaz: as well as mutation/observation of non-browser features, e.g. Geolocation 02:37:49 ato: these things are not dependent on a bi-di comm interfact 02:37:58 q? 02:38:06 CalebRouleau: there is probably overlap of these use cases 02:38:11 ack ato 02:39:05 simonstewart: bi-di comms expose 2 common patterns: 1 request, I want 1 response, or "I'm registering a listener, so give me responses as they come up" 02:39:21 simonstewart: registering for a listener is "network interception" 02:39:44 simonstewart: e.g. "I want to click a button" or "I want to type something" 02:40:13 simonstewart: people usually set up a web socket as a long-running connection (usually on localhost only) 02:40:26 simonstewart: this normally sets up the web socket 02:40:47 simonstewart: when not on localhost, we introduce risks around corporate networks 02:40:47 q+ 02:40:48 q+ 02:41:15 ack jgraham 02:41:21 simonstewart: need a mechanism for reconnecting and being notified of what was missed (in the case of complcated, corporate networks) 02:41:28 rrsagent, make minutes 02:41:28 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html JohnJansen 02:41:34 jgraham: feels like a lot of scope creep in the context of what exists 02:41:53 q+ 02:41:53 jgraham: buffering up these things won't work well. Listening to network requests adds risk 02:42:20 jgraham: timeouts are a risk 02:42:53 jgraham: need to hear that this is super-important from the most prolific users before we make the browser responsible for caching all these requests 02:43:20 cb: the VM will shut down if the request times out 02:43:23 q+ 02:43:55 jgraham: is network flakiness such a big problem that we need to later the protocol? do we need to abstract over that with an event buffer to keep the integrity of the comms in place to that extent? 02:44:07 cb: we can always build an abstraction around it that doesn't need to be part of the protocol 02:44:12 q+ 02:44:37 q- 02:44:59 jgraham: not sure about the web socket issue, but would like to know whether this body considers that to be a risky or unstable thing to rely on 02:45:10 simonstewart: HTTP2 server is an alternative 02:45:18 Server push 02:45:29 q? 02:45:34 ack ato 02:45:41 MikeSmith: WebSockets are essentially deprecated at this point 02:45:55 s/HTTP2 server/HTTP2 server push/ 02:45:56 q? 02:46:24 ato: we are jumping ahead of the discussion - many assumptions going on about how the protocols work 02:46:46 ato: we've agreed that having a bi-di mechanism is good, but we are constrained by the existing protocols that are out there 02:46:59 ato: designing a new one from scratch seems silly 02:47:38 ato: I'd like to see a good x-browser automation protocol, designed from scratch, which covers all use cases, which fixes the problems in the existing protocols, but that's not possible at this point 02:48:29 ato: SL says they want to expose the existing internals (CDP/Puppeteer, etc), but not all these protocols would support the kind of embedding necessary for the "resumption of session" that simonstewart wanted 02:48:55 simonstewart: one of the problems we had developing the WD standard is the constant improvement/change we had during the process 02:49:09 q+ 02:49:13 q? 02:49:13 jgraham: I believe SL didn't say they wanted those features--they're already doing them 02:49:39 ato: this isn't about CDP specifically, even though that's what it sounds like when I listen to this body 02:50:15 ato: CDP does support session resuming, web socket connections, etc... by going in to that conversation, we're jumping ahead... we're adding constraints 02:50:29 q+ 02:50:32 q? 02:50:35 ato: I would like to know the opinions of the browser vendors to know what the outcome should be 02:50:39 q+ 02:50:46 q- 02:51:03 ack CalebRouleau 02:51:09 ato: need more clarity as to what exactly would be *in* that web socket, if the web socket is exposed 02:51:18 CalebRouleau: we want to think about what's currently there with CDP 02:51:54 q+ 02:52:08 CalebRouleau: we don't want to implement something from scratch in the Chrome Team. We've worked for a long time on the CDP, and need the new feature set to be at least somewhat compatible with what we've done already 02:52:49 CalebRouleau: we don't want WD to be overly complicated 02:52:57 q? 02:53:00 simonstewart: we should take the caching of things off the table for now 02:53:26 +1 to simonstewart taking caching off the table 02:53:49 drousso: Apple's position is that we want to see how this will work before we sign off. Whether it's WebSocket or HTTP2, it's a rewrite for us 02:53:52 s/We've worked for a long time on the CDP/we have a lot of stuff that already depends on CDP and we don't want to support both/ 02:54:02 drousso: so if we're going to do it, we need to know more before we can commit 02:54:18 jgraham: are you saying you're interested, but with no plans to implement? 02:54:47 brrian: we don't currently allow for either Web Sockets or HTTP2, and this would represent a significant spending of resources 02:55:00 q? 02:55:05 CalebRouleau: how does the CDP work for Safari, in detail? 02:55:20 simonstewart: the Safari implementation of the debug protocol is currently locked down 02:55:40 q? 02:55:44 CalebRouleau: I'm trying to ask what is publicly available, but it sounds like it's basically nothing 02:56:14 brrian: we need specific questions and specific feature requests in order to get this conversation going for Safari 02:56:49 RRSAgent, make minutes 02:56:49 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html Hexcles 02:56:59 jgraham: at some point we need a clear decision about the debug protocol, some entry criteria... but as long as we can agree on the messaging itself, that will be sufficient 02:57:20 q? 02:57:35 ack cb 02:57:43 scribenick: AutomatedTester 02:58:12 cb: the SL major use case is being able to access the browser internals to surface that to people 02:58:41 CalebRouleau: you dont care the transport, you just want access 02:58:44 cb yes 02:58:48 s/SL/SauceLabs 02:58:48 q? 02:58:56 ack jgraham 03:00:09 jgraham: for Mozilla we think there is a cross browser automation protocol that facilitates automation. We need to be aware that there is a move to something like puppeteer is a threat tot he web since it only covers 1 browser 03:00:23 s/ SL / SauceLabs /g 03:00:23 s/tot he/to the/ 03:00:50 q+ 03:01:06 q? 03:01:11 q+ 03:01:13 q+ 03:01:18 q+ 03:01:33 q+ 03:01:52 jgraham: There has been some discussion around CDP for reuse. One of the historic complaints is that CDP is unstble and only gives us certain things that are chrome specific. We would need to see what is stable and what we can solve those 03:02:05 simonstewart: and this is why we focused on use cases 03:02:23 q? 03:02:29 ack boaz 03:02:43 JohnChen: yes, my point is that there are implicit stability from the APIs that use CDP 03:02:58 s/JohnChen/jgraham 03:04:34 q? 03:04:48 ack brrian 03:05:40 brrian: it would be great to expose the API and just add to it. That is a bad idea. We need to make sure that we versioning and even with that we break things all the time. 03:06:34 brrian: it is very difficult to test at the moment 03:07:09 brrian: whatever we do needs to make sure is testable and that we dont rely on browser internals 03:07:54 brrian: in webkit we have processes that come and go and our tools should not have to follow that 03:08:21 ... we are going to need some compat layer to figure this out. 03:09:26 ... [discusses examples of where webdriver and a tool might look different] 03:09:29 q? 03:10:06 brrian: as for JSON RPC, we have used it for 11 years and it is solid so I dont think we need to change that 03:10:26 q? 03:11:05 karl has joined #webdriver 03:11:23 jgraham: is this very different to what the chrome engineers have said ? 03:11:39 brrian: no, this is not going to be a lot of work, its just going to be well tested 03:12:28 drousso: one of the things that I would caution against. A lot of things have browser leakage and we need to make sure that browser impl info is not bled out 03:13:02 q? 03:13:04 CalebRouleau: there are lot of issues, like site allocation, we need to make we are a level above that 03:13:18 ack bwalderman 03:13:23 s/site allocation/site isolate/ 03:13:52 bwalderman: we need to work top down from use cases like brrian suggested 03:14:17 ... and it might be better to keep them separate from the debug protocols out there 03:15:06 q? 03:16:15 q+ 03:17:43 AutomatedTester: are you suggesting 2 connections into the browser or that the shim would handle 2 and then do magic into the browser 03:17:59 bwalderman: it would be in the shim 03:18:18 AutomatedTester: good as there could be an attack vector into the browser if there are 2 and we need to be aware 03:18:52 jgraham: ok but we need to make sure that the connection can handle that 03:19:06 ... and that the shaping of packets can handle it 03:20:15 ... we have seen in gecko that there are cases where this doesnt work. We are constrained on what the low level remote protocols can handle. 03:20:33 bwalderman: I am not suggesting that we write from scratch 03:20:39 q? 03:20:39 q? 03:20:50 ack ato 03:25:36 ato: is it reasonable to have something that maps down to other protocol? 03:25:42 Context about what ato is saying http://operasoftware.github.io/scope-interface/ 03:25:42 https://dev.opera.com/blog/opera-scope-protocol-specification-released/ 03:26:30 ... we have seen multiple versions and there has been times to standardise 03:26:54 ... we need to make sure we agree on what protocol actually means and transport layer actual means 03:27:04 q? 03:27:53 ack CalebRouleau 03:27:56 ack JohnJansen 03:27:56 CalebRouleau: the new devtools team are interested in standards 03:28:03 ack JohnChen 03:28:12 Zakim: close the queue 03:28:35 JohnChen: we have been experimenting with webdriver and upgrade to a bidi connection via CDP 03:28:43 close the queue 03:29:19 q? 03:29:27 q- 03:30:53 brrian: as far as 2 protocols... for security... we have already 2 and they enter the stack at 2 different points. 03:31:13 ... we have mechanisms to protect the user now 03:31:58 ... [explains how a bad actor could attack] 03:32:12 things from the past https://github.com/WICG/devtools-protocol 03:32:59 ... and there are people writing a lot of adaptors for VSCode and they dont think that it is a lot of work 03:33:04 RRSAgent, make minutes 03:33:04 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html Hexcles 03:33:05 LUNCH 03:33:51 Hexcles has joined #webdriver 03:35:14 Hexcles has joined #webdriver 04:02:53 zcorpan has joined #webdriver 04:08:49 Hexcles has joined #webdriver 04:27:08 bwalderman has joined #webdriver 04:28:24 diemol has joined #webdriver 04:30:49 titusfortner has joined #webdriver 04:31:14 zghadyali has joined #webdriver 04:31:33 Hexcles has joined #webdriver 04:31:45 reopen the queue 04:34:55 CalebRouleau has joined #webdriver 04:36:16 mmerrell has joined #webdriver 04:40:43 drousso has joined #webdriver 04:44:29 https://gist.github.com/Hexcles/69f44b94aa616981a564efff11e5f4bb 04:48:43 jgraham: change in the agenda 04:48:49 scribenick: mmerrell 04:48:51 karl has joined #webdriver 04:49:11 jgraham: move the continuing discussion about bi-di to Friday morning 04:49:34 https://docs.google.com/document/d/1eJx437A9vKyngOQ49lYYD3GspDUwZ6KpKDgcE2eR00g/edit 04:49:38 jgraham: homework needs to be done, where Google team proposes beginnings of an outline for the bi-di protocol 04:50:01 CalebRouleau: it's not clear that we should start with load event--we should start with something else in order to satisfy the 3 other use cases 04:50:19 stevenb has joined #webdriver 04:50:41 simonstewart: one clear thing is that we're going to re-hash bi-di on top of WD, which won't satisfy all needs 04:51:16 ... need to put building blocks in place to move forward 04:51:29 simonstewart has joined #webdriver 04:51:33 present+ 04:51:40 present+ 04:51:42 CalebRouleau: to JohnChen: could you write up a proposal 04:51:45 present+ 04:52:10 JohnChen: today, you can't do these use cases with WD, but you can do some amount of "loading" 04:52:40 simonstewart: agree with jgraham: this will allow the network proposal to move forward 04:53:02 CalebRouleau: maybe simonstewart can come up with the network proposal while Google team works with jgraham to work on a proposal for Loading 04:53:13 jgraham: the loading proposal will look like CDP without the target stuff 04:53:23 jgraham: "how do I address a loading context?" 04:53:34 urata has joined #webdriver 04:53:37 jgraham: not everyone has to participate in the side meeting, but it should be open 04:54:04 jgraham: if we're not making progress on loading, we can address later 04:54:06 nao_watanabe_access has joined #webdriver 04:54:22 AutomatedTester: moving bi-di discussion to Friday morning. Agenda has been updated 04:54:43 CalebRouleau_ has joined #webdriver 04:55:03 AutomatedTester: need to start with custom selector strategies, then shadow DOM, then break, then ARIA driver, then the laundry list of other items on the list 04:55:15 AutomatedTester: (the non-contentious ones) 04:55:27 [general assent ensued] 04:55:33 topic: Custom selector strategies 04:55:50 AutomatedTester: from cb: custom selectors 04:56:10 cb: objective is to help user of WD protocol to automate tests for new frameworks like React/Vue 04:56:44 cb: friction is that new frameworks are built on browser features that aren't mapped well to WebDriver element locators 04:57:14 cb: automation engineers are having trouble getting to some of these new kinds of elements. Protocol needs to be extended to help locate these kinds of elements 04:57:15 q? 04:57:21 ack brrian 04:58:06 q+ 04:58:24 cb: 2 proposals: drivers can share libraries of atoms. WDio can fetch elements by property name or by component property, taking advantage of shared components in new frameworks. Risk is that this creates organizational overhead for application teams 04:58:41 q+ 04:59:06 q+ 04:59:38 cb: other option is "custom selector strategy", allowing vendors to be able to intercept selectors and be smarter about actually locating the element. Advantage here is that there's no new work for this body. Downside is that implementations could be so different that there's more overhead for browser vendors, and behavior won't be standardized 05:00:28 ... alternative is that users could register scripts that would collate and cache selectors and selector scripts to avoid unnecessary wire calls 05:01:01 q 05:01:02 q+ 05:01:07 titusfortner: this would allow the driver to operate much more quickly. TestCafe allows location by component, not just CSS. Don't know about implementation, but we need to know if all browsers work the same wrt WDio? 05:01:08 diervo has joined #webdriver 05:01:22 cb: yes it does--the abstraction goes through the virtual DOM 05:01:25 q? 05:01:31 ack jgraham 05:01:33 q- 05:01:35 present+ 05:01:48 rrsagent, make minutes 05:01:48 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html JohnJansen 05:01:53 q? 05:02:28 jgraham: bad idea to try to standardize on large chunks of JS atoms. it's not something that will be future-proof: web frameworks come and go, and standardized javascript ends up being brittle 05:03:18 jgraham: we should either double-down on vendor-specific extensions to the protocol, or register JS scripts (caching) to improve performance 05:04:01 q? 05:04:05 titusfortner: this wouldn't be limited to just locators--it could be any javascript 05:04:09 cb: yes, it could be 05:04:17 q? 05:04:27 ato: question: can you explain why this is needed? 05:04:44 ... do you mean to register particular methods? 05:04:48 cb: yes 05:05:03 ato: these frameworks manipulate the DOM in a way that makes it difficult to just use CSS selector? 05:05:48 cb: yes--React developers write these components using more dynamic logic, and the QA engineer has trouble defining the locator 05:05:55 ack simonstewart 05:06:11 titusfortner: but it's not the QA eng we're concerned with. This is for developers 05:06:34 q+ 05:07:48 simonstewart: there's something here, about registering javascript snippets, which would return a handle. Use cases: good points about React and Vue, compiled and inflated, produced by the client and uploaded. New "friendly" locators in Selenium as well as Watir's JS atoms, all upload these snippets, which becomes chatty and expensive really quickly 05:08:02 q? 05:08:03 q+ 05:08:20 simonstewart: you don't want to inject these scripts on every page load--we want to do this per-session. This would amount to an overloaded version of the JavascriptExecutor 05:08:47 simonstewart: the user-facing API would make it look like a selector, but underneath it's using the same handler, parameterized 05:09:13 simonstewart: this will end up being a client concern--not a WD spec concern 05:09:28 simonstewart: this will be hard to spec out 05:09:32 jgraham: it won't be that hard 05:10:51 ack ato 05:11:32 q? 05:11:43 ato: the latter proposal isn't too invasive--it makes sense to register scripts and allow devs to inject a JS library and re-use it per-session, but these solutions are so different that they address different problems 05:11:56 ato: the selector API needs to be locked down very tightly 05:12:02 q+ 05:12:52 ato: you'd want a capability to pre-register a selector and identify them by a string (the key in a hash), passed in the body of a function... it would be a wrapper accepting one argument, which would return the locator 05:12:52 q+ 05:13:04 ack titusfortner 05:13:32 q+ 05:13:35 titusfortner: is there a distinction between using a custom selector and pre-registering the scripts? how generic is the current mechanism? 05:14:20 titusfortner: should we have a generic "driver registration", and then have a component-based registry, all as part of the spec? 05:14:21 q+ 05:14:41 simonstewart: if you have this registry for any of the javascript, you could use it for any of these use cases 05:14:54 titusfortner: would it make sense to explicitly make it generic? 05:15:16 simonstewart: you want the remote end to be able to have non-Javascript locators 05:15:40 simonstewart: Jason Arbon demonstrated an ML-based element locator that wasn't Javascript 05:15:53 simonstewart: but in most cases it will be JS 05:16:02 jgraham: we should pin this down to stuff that can be implemented in JS 05:16:07 simonstewart: yes, agreed 05:16:42 q+ 05:18:00 jgraham: we shouldn't allow scope creep, with all that's going on, to include non-JavaScript features into the "pinning" (registering) piece we're talking about 05:18:14 ack drousso 05:18:25 q+ 05:18:27 q- 05:18:30 q+ 05:18:34 drousso: fundamentally, this isn't about pinning or caching--it's minimizing the chattiness of the comms 05:19:25 ... this isn't fundamentally a concern of the WebDriver spec. This is about implementation, and adding this to the spec would add more questions and opens the door for complications 05:19:31 q? 05:19:32 q? 05:20:01 RRSAgent, make minutes 05:20:01 I have made the request to generate https://www.w3.org/2019/09/19-webdriver-minutes.html gsnedders 05:20:22 ack diervo 05:20:26 diervo: we have a huge group of web components. From our POV, we shouldn't worry about the spec for locators, we should be able to register anything before the session 05:20:50 ... having this ability would foster custom elements, and would allow for advanced traversal, etc. 05:21:30 ... we would like syntactic sugar for being able to select things with custom commands 05:21:31 q? 05:21:39 ack cb 05:21:39 ack cb 05:21:40 simonstewart: this is, at its heart, JavaScript, though, right? 05:21:43 diervo: yes 05:22:26 present+ 05:22:26 cb: this proposal fosters the sharing of code, even when items are embedded in the code and in different places. This would allow shareable atoms 05:22:51 drousso: but you can already do that--you don't need to change the spec in order to make this happen. You can code it elsewhere 05:23:08 q? 05:23:11 q+ 05:23:19 simonstewart: there's a difference between what browser developers need and what execution vendors need 05:23:40 q+ 05:24:49 ack bwalderman 05:25:03 [fast discussion about caching, sandboxing, bootstrapping ensues, from which no point arose] 05:25:29 bwalderman: if we're going to add this ability, we should also allow messages to pass between the client and the script 05:26:00 ack ato 05:26:05 ... it would be interesting to register these events. It would help users, but is a little outside the scope of custom selectors 05:26:26 ... it's more in the scope of the bi-di discussion 05:26:34 q+ 05:27:03 ato: there's an alternative to consider--JS frameworks are sensitive to DOM modification [gives examples] 05:28:05 ... there's risk in registering these snippets to the global window. It could result in race conditions (?) and other issues with dynamic changing of page contents when multiple regions of the page are affected 05:28:08 q+ 05:28:50 q- 05:29:22 jgraham: qq: the proposal was to dump text into a map, which you can then later pull it out... the proposal was not to have live JS scripts executing at will within the browser. This would present a large interaction/security risk. The intent is to optimize performance, not to enhance feature function or to execute scripts different than is currently done 05:29:31 q+ 05:30:13 karl has joined #webdriver 05:30:19 ato: not quite--the concern was about maintainability of these functions. The client should be able to pre-register a function, but the concern is that there will be a maintenance burden 05:30:56 ato: maybe the browser could be pre-loaded with a webdriver running? 05:31:05 q? 05:31:47 cb: if we could pin these JS snippets to the session, we could provide these features easily. SL customers would benefit from these features, both for performance purposes as well as test stability 05:31:57 ato: wouldn't it be better if the React devs owned the selector? 05:32:13 cb: that's not important--we just need to be able to support the feature? 05:32:41 ato: it's not future-proof--old versions of React won't work the same way as current ones, and that will be a continuing risk to any project implementing this 05:32:45 simonstewart: agreed 05:32:56 jgraham: we shouldn't be discussing SL product strategy here 05:33:00 q? 05:33:04 simonstewart: but pinning a script still makes sense 05:33:07 ack simonstewart 05:33:11 q- 05:33:54 simonstewart: it's a useful feature, which could be implemented in many ways... as a bootstrap script, this would alter the AUT, but it could also be put into the driver process, and that would allow better performance 05:34:04 simonstewart: you need 2 endpoints: upload and call 05:34:09 jgraham: we already have call 05:34:28 q? 05:34:37 jgraham: call could be used for this 05:34:47 titusfortner: what's the downside of having another endpoint? 05:35:06 ack JohnChen 05:35:08 isn't this the proposal? 05:35:08 https://docs.google.com/document/d/13ycIhXJxoCq0K6ti10VpFp_l9hLtTFrx941uC_aSwfk/edit 05:35:33 simonstewart: we shouldn't need another endpoint--but anyway, we're arguing over how much it will cost, not whether or not we need it, so it sounds like we've made our decision 05:35:56 q+ 05:35:59 That proposal needs clearing up to talk about script pinning 05:36:24 JohnChen: this script will still need to be send to the app, no matter what. There might not be a real way, at least in chrome, to pin these scripts in a way that will be markedly improved wrt performance 05:37:15 JohnChen: the selector pinning strategy as proposed will be exceedingly difficult to implement due to how the already-huge chunks of JS are stored in Chrome 05:37:43 JohnChen: we can store these snippets, but it will be difficult, and it won't help that much 05:38:00 jgraham: this would supplant the existing JS around "findElement" 05:38:21 q? 05:39:13 JohnChen: not in every case. It varies depending on how many elements you're looking for, and where they're located in relation to each other. There's a way to do this, but we shouldn't try to merge this with existing findElement--we should offer a mechanism for uploading JS that would replace it 05:39:38 titusfortner: this might not need to be part of the spec 05:40:08 s/send to the app/sent to Chrome/ 05:41:03 ack brrian 05:41:14 brrian: if these atoms are used in testing, and used in the app, can't the app include a snippet that would "assist" with locating the element? 05:41:32 q? 05:42:02 simonstewart: quite often people minify their JS, which mangles the ability for the app to display the same element that is stored in the app 05:42:27 q? 05:42:34 simonstewart: testers are usually separate from the developers and have no influence there 05:43:24 simonstewart: testers can't have the locators changed to something more friendly 05:43:45 mmerrell: the wall between devs and testers is the norm, not the exception 05:44:05 scribenick: AutomatedTester 05:44:21 brrian: this is not going to make things faster 05:44:25 q? 05:44:38 titusfortner: yes but this is for larger connections we are sending lots of data all the time 05:44:41 ... from safaridriver onward (to devices, other apps, etc) 05:46:13 brrian: why are we caring about people perf, they should 05:46:15 q? 05:46:18 q+ 05:46:45 titusfortner: well a lot of people are doing things that are silly like they are sending 20 commands for finding 1 element 05:47:24 q- 05:47:26 q- 05:47:53 ack CalebRouleau_ 05:47:57 ACTION: Saucelabs to write a propsal and send to the group 05:48:19 q? 05:48:22 q- 05:48:30 CalebRouleau: ++ 05:48:48 Topic: ShadowDOM Support 05:49:53 q+ 05:49:57 https://github.com/w3c/webdriver/pull/1320 05:50:02 ScribeNick: ato 05:50:09 AutomatedTester: We have discussed this before, briefly. 05:50:09 👆 05:50:34 ... I had a proposal in a PR from before. 05:50:36 ... We need to think of a process, and happy to throw away this PR and start from scratch. 05:50:41 ... Shadow DOM has interesting problems. 05:51:00 ... We need to be able to interact with it, and there are two types of nodes. 05:51:14 ... Sometime you can see into these nodes, and sometimes they operate as black boxes. 05:51:35 ... Frameworks like React are looking into using Shadow DOM, and we need to support this use case otherwise it will be hard to automate. 05:51:39 q? 05:51:41 cb: use "/msg" to send a private message :) 05:52:19 jgraham: You have a shadow host element, which conceals a shadow DOM. 05:53:16 ... If you want to go into the shadow DOM. 05:53:44 ... Two shadow DOM host elements: open and closed. 05:53:56 ... Theoretically with open elements the inner elements are exposed to JS. 05:54:04 ... You can polyfill this today with JS. 05:54:27 ... You can Execute Script on the DOM property that gives you the shadow root, and from there you have access to the tree below that to manipulate them further. 05:54:35 ... My proposal is that we hoist that into a WebDriver endpoint. 05:54:46 ... Should should also work for closed shadow DOM trees. 05:55:01 ... If you want to pierce the encapslation field for testing, that is possible to do from content. 05:55:13 ... DevTools allows you to pierce it. 05:55:38 ... "element/shadow" could return a shadow host for that element. 05:56:11 s/encapslation/encapsulation/ 05:56:48 ... The alternative is for Find Element to take an extra parameter: but instead of returning the element, it would return the shadow DOM root inside that element. 05:57:08 lukebjerring: Can I see the original proposal? 05:57:20 jgraham: The original proposal worked like frames, that you’d have to switch into the shadow DOM. 05:57:51 jgraham: There are however a few things that work for frames that may not work for shadow DOM, so I don’t think it’s entirely workable. 05:58:00 q+ 05:58:17 AutomatedTester: When you want to click on something, you need to make sure you’re in the right space. 05:58:33 ... For example, inside the