07:38:48 RRSAgent has joined #webdriver 07:38:52 logging to https://www.w3.org/2023/09/15-webdriver-irc 07:38:53 Zakim has joined #webdriver 07:39:16 present+ 07:39:18 present+ 07:39:19 present+ 07:39:21 Meeting: Browser Tools & Testing @ TPAC 2023 - Day 2 07:39:31 present+ 07:39:45 Chair: David Burns 07:39:58 present+ 07:40:05 present+ 07:40:06 present+ 07:40:06 present+ 07:40:17 present+ 07:40:20 Topic: Consider whether to allow multiple sessions in browsers 07:40:29 github: https://github.com/w3c/webdriver-bidi/issues/103 07:42:06 ScribeNick: shs96c 07:42:17 q? 07:42:27 present+ 07:42:37 RRSAgent: make minutes 07:42:38 I have made the request to generate https://www.w3.org/2023/09/15-webdriver-minutes.html jgraham 07:42:51 q+ 07:43:45 jgraham: the original issue was that Classic only allowed on session. With bidi, we could have one tool running tests, and another constructing HAR files from network traffic. 07:43:54 q+ 07:44:07 jgraham: obvious way to support this is to have more than one session connecting to a browser. 07:44:59 jgraham: the spec theoretically currently assumes this is the case. There's a PR that never landed in Classic to make this actually work, since the spec currently uses Classic's definition of a session 07:45:06 ack orkon 07:45:47 q+ 07:46:02 orkon: in puppeteer, we can connect to a running browser. Want to clarify status. Maybe not use `session.new`, but `session.connect` to connect to a running browser. Why? Because you can't change capabilities of a running session, so connecting is a different workflow 07:46:09 ack AutomatedTester 07:47:19 q+ 07:47:20 AutomatedTester: this is something we are going to need to do. Potential overlap between sandboxing and multiple sessions. The use case I'd like to support is to create a sandbox tab and have a session per tab, so a client can run tests in parallel more quickly. This is something that Playwright does. 07:47:21 q+ 07:47:34 ack jgraham 07:48:06 q+ 07:48:32 jgraham: about session.new I don't think it's necessary to have a new command. You can always fail to create a session if you're trying to change the capabilities of a running browser in a way that's not supported. 07:49:44 jgraham: I agree that conceptually there are two different operations (starting and connecting to a browser process) Historically we've done this to make the protocol more RAII. Adding a new command feels unnecessary. 07:50:51 jgraham: we should get to this point when we come back to sandboxing. There seem to be two different ideas about how this might work. Need to allow complete access to a running session (eg. to capture network traffic, or monitor windows) 07:50:53 ack whimboo 07:50:54 ack next 07:51:10 qq+ orkon 07:52:39 whimboo: we also have situations where you are running a browser and later connect. In our case, we've seen people are running the browser with the protocol active, but without a current session. We would still be able to pass in capabilities. This new session would definitely want to set capabilities. 07:53:11 whimboo: so we could probably move this to the later sandbox discussion 07:54:22 ack next 07:54:23 orkon, you wanted to react to orkon 07:54:33 q+ 07:54:45 orkon: I think that after hearing the arguments, an extra command is unnecessary. Some capabilities are sent to the browser, some to the session, and some to both. Wanted to mention about the sandbox discussion we had yesterday. That doesn't require you to have a separate bidi session. We can discuss that later 07:54:51 ack next 07:55:48 q+ Sam Sneddon [:gsnedders] 07:55:56 ack whimboo 07:56:50 whimboo: when we want front end sessions to connect with capabilities. Will they be overridden by the new session? If a new session that wants to connect can't do this, how should it fail? 07:57:26 q+ 07:57:38 ack Sam Sneddon [:gsnedders] 07:58:26 I think we do need a way to distinguish whether a new session is being connected, because the local end probably cares about whether they're getting a browser in a clean state or not. 07:58:38 q+ 07:58:48 I don't care whether that's via a new command or a capability or whatever, but I think it's important that it is distinguishable 07:58:53 ack next 07:59:13 ack next 07:59:21 ack next 07:59:35 ack 07:59:39 * ack next 07:59:45 ack next 08:01:15 jgraham: regarding capabilities: you can fail the session on creation if the session can't be created. The spec is reasonably clear about those cases. In terms of needing to know whether you're getting a new browser instance or not, the local end would be in control of that, as it would know whether it was trying to connect to an existing browser instance or whether it was trying to connect to an existing one. 08:01:25 if you're trying to run tests in parallel, you might not know that. imagine Selenium with Python and pytest and pytest-xdist. 08:01:41 jgraham: I assume clients would make it clear to users which it was trying to do (connect to new or existing browser) 08:02:29 jgraham: The service would make it clear whether you're getting an existing browser or not. Not sure what would happen at the protocol layer to support that. 08:03:04 q+ 08:03:04 spectranaut_ has joined #webdriver 08:03:04 jgraham: if you're connecting to an existing browser, presumably you could query to find out open windows, or such 08:03:04 jamesn has joined #webdriver 08:03:04 ack next 08:03:08 scribe: automatedtester 08:03:33 Sam Sneddon [:gsnedders]: I'm imagining that, but I don't quite see the case where it would be a problem. 08:03:41 shs96c: the problem that can arise with selenium providers in the cloud. There is no guarantee that the routing will get to a running machine 08:03:53 ... I think we should solve this at a protocol level so we can handle this 08:04:23 ... we can have a session.connect that can put it to the right machine 08:04:52 ... I can see that I would want to connect a test and handle things and then the provider to have a internal tool for building HARs 08:05:11 jgraham: if you have the session I can't see this being an issue 08:06:13 shs96c: In the network intercept module to change things and then the provider to also have a session in to collect this info 08:06:21 jgraham: I can see that then 08:06:55 shs96c: the routing in a cloud provider is why we shouldnt have a new command 08:07:08 s/shouldn't/should/ 08:07:15 s/shouldnt/should/ 08:07:21 Specifically the thing that makes sense to me is that there might be a situation where you want different clients to get different events, even if you can share session ids. 08:07:44 q? 08:07:51 ack next 08:08:18 q+ 08:08:28 orkon: regarding Sam Sneddon [:gsnedders] question. I think it would be really useful to be able to show how many connections are available 08:08:37 ... then you can see if you are getting a "fresh" browser 08:08:37 orkon: it would be really useful if the response to new session tells you which sessions are already available in the instance. Then you can tell whether you're "alone" 08:08:49 scribe: shs96c 08:09:08 orkon: unsure about using session id for routing, you would get a different socket url 08:09:34 shs96c: traditionally, all cloud providers have used session id as a routing key 08:09:41 q? 08:10:48 AutomatedTester: generally what happens is that a client connects to a hub. The hub then attempts to find the closest new machine, and then will route commands to that. Will do that over websockets using forwarding of commands. Any messages coming back are sent to the local end. 08:11:31 q? 08:11:41 ack next 08:13:10 jgraham: I'm a little nervous about returning "there are N open sessions" It feels like it should be opaque to you that there are other sessions. You can't leak the session IDs. It's not obviously a security concern to know whether there's another session, but it's not the information you care about. 08:13:11 q+ 08:13:56 jgraham: you probably want to know the state that the browser is in, and we have commands for that. If you expect the browser in a specific state, then your tests should query the session to ensure that, rather than assuming some default base case. 08:15:02 jgraham: we could make this work with a single command, even in the cloud case (eg. by providing an existing session id as a capability, and returning a new session only if there's an existing session with that id) 08:15:23 jgraham: for the local use case, forcing the design to support the remote use case seems like a lot of work 08:15:45 ack next 08:17:54 q+ 08:17:55 q+ 08:19:26 shs96c: two points. 1/ sessions should be opaque to each other so that cloud providers can (eg) monitor sessions without users being aware that the monitoring is happening. 2/ disambiguating connect from new session is useful for a number of reasons, such as memory footprint (capabilities can be large if there's a profile in there), and so that it's crystal clear in logs what was happening 08:19:33 ack next 08:20:47 jgraham: your use case of cloud provider running a session and you don't need to know about that suggests there are cases that users will want to run "new session" 08:22:24 jgraham: cloud providers might have a pool of "warm" sessions ready to go, so I think new session should work to connect to an existing browser. There may be a use case for connect for cloud service providers, specifically for cloud service providers, where you might only know the session id. 08:22:24 ack orkon 08:22:24 q+ 08:23:30 orkon: I feel that the problem is a bit related to the fact that `new session` does multiple things. It not only creates a new session, but is also responsible for launching a browser. If we separated these cases, we could launch the browser without any sessions, and then we could create a new session. Third use case would be about creating a session 08:23:52 jgraham: there's nothing in the spec that says that new session will launch the browser. 08:24:25 jgraham: with WPT the typical thing we do is that we pre-launch the browsers, and then create a webdriver session. That allows us to have a pool of browsers waiting. 08:24:53 jgraham: there's nothing in the spec that requires new session to start a browser, but that's typically what we do. Cloud providers might also maintain a pool of instances. 08:25:16 orkon: from the perspective of cloud providers, don't they want some way to get a browser to be launched. 08:26:08 jgraham: new session just somehow makes the session available. Describes typical local workflow, where the command finds a driver binary, starts it, and then that starts a browser. But there's nothing in the spec that says that has to happen 08:26:17 q+ 08:26:24 ack AutomatedTester 08:27:41 AutomatedTester: when it comes to cloud providers, we can't necessarily have a warm pool to connect to, because one of the key things is that people want to amend the `new session` requests on startup. If the browser is already started, you need to restart the browser, so it doesn't make sense to have a pool. You also can't ever trust the previous session, so you have to clean up the entire OS (eg. by rolling back to a previous OS snapshot, or 08:27:41 starting a clean container) 08:28:06 I didn't mean to suggest that there was necessarily going to be a warm pool, just that it seems like a valid optimization (e.g. wpt has exactly this optimisation for Firefox instances). Maybe cloud providers aren't the best example here. 08:29:12 AutomatedTester: I don't think we can connect to something that's already spun up in that sense. But there are going to be times when we want to (eg) collect all the logs or performance data, so that we can build in features that customers might want. But the end user clients are likely going to just want a clean session. If they find out other sessions are connected, they'll want to know why. If customers have a sense that their sessions 08:29:12 aren't entirely there, they'll lose trust in the service. 08:29:57 qq+ 08:31:47 jgragam: connect feels like a subset of what new session does. A lot of the scenarios I'm thinking of are ones where you may not know the session id. For example, if I'm running some tests and I want to connect my VS.Code instance to running tests, I may know the process ID of the browser, but not the session id. 08:31:58 s/jgragam/jgraham/ 08:32:16 jgraham: I agree for a cloud provider, the opposite is true. 08:32:59 shs96c: in selenium /status gives you a list of the session ids running in the process, so if you have the port to connect to, you can find out the sessions ids 08:33:06 ack AutomatedTester 08:33:06 AutomatedTester, you wanted to react to AutomatedTester 08:33:57 AutomatedTester: Christian Bromann was advocating for a more async `new session` command, which fits in with the "warming up" idea. That's because it may take a long time to start a new browser. That might be something to look into. 08:34:30 ack sadym 08:34:51 sadym (IRC): to clarify, we have a webdriver server, and we want to support a few websocket connections to that server, and some of those connections should connect to the same browser instance, and some of those connections to different browsers 08:35:12 sadym (IRC): if I create a new websocket connection, i need some kind of ID, and that's the session id 08:37:14 jgraham: the answer is "maybe". In the case of a cloud provider, the session id unambiguously identifies the browser. If I'm running firefox locally, there isn't necessarily a server instance involved, but I have shared information that there is a firefox instance that was started with webdriver bidi, and if I connect to the local port. I can do that without knowing whether another program is running 08:37:15 q+ 08:37:44 jgraham: so I can start a test, start a network recording program, and then the port is the bit of shared information we need. 08:38:13 jgraham: in some cases the session id is enough. In other the PID or the port number is enough. If we connect via pipes, then the PID would be the thing you want to share around 08:38:45 sadym (IRC): for geckodriver, it only supports one instance of firefox. But in the more general case, you have a N:M connection. 08:39:36 shs96c: test runners typically run multiple tests locally 08:44:24 q+ 08:44:40 (discussion between shs96c and jgraham) 08:45:54 shs96c: what about a connect key to allow the use case of session connect 08:46:11 jgraham: for the non-cloud use case, that feels like overkill 08:46:36 shs96c: but there's no guarantee of a stable identifier that can be used locally 08:46:48 jgraham: that would be the URL, wouldn't it? 08:47:02 q+ 08:48:27 ack shs96c 08:48:28 q+ 08:48:31 q++ 08:48:56 qq+ gsnedders 08:49:11 ack orkon 08:49:19 q- + 08:49:19 orkon: it sounds like cloud providers should add a session id or similar to a websocket url 08:50:26 orkon: the websocket url is the identifier that can be used. Therefore it's fine to have just a `new session` command. Cloud providers know which sessions have started and can maintain a routing table based on url. 08:52:12 AutomatedTester: that's what the session id is trying to do. Users connect to a single URL, not to multiple ones. Session ID can be part of the URL (classic) or payload (bidi). One of the downsides with the CDP model is that we need to build out forwarding websockets that go through a hub for accounting purposes. Selenium makes that second connection. The suggestion of using the url puts a lot of complexity into the intermdiary nodes 08:52:38 ack Sam Sneddon [:gsnedders] 08:54:00 Sam Sneddon [:gsnedders]: when you're running locally, it's often the case that you can specify the port the session will listen on. That gives you an ID you can look up on at the point where you create it. As soon as you add in any other ID being required, you either need to continue being able to pass in the additional information, or you need to pay more attention when the session is established. 08:54:09 Sam Sneddon [:gsnedders]: let's not break the simple case locally 08:54:15 +1 on not breaking the simple case locally 08:55:06 q+ to mention that we're already returning the websocket URL in session.new 08:55:23 ack next 08:55:25 gsnedders, you wanted to react to shs96c 08:57:18 q+ 08:57:44 zakim close the queue 08:57:54 zakim, close the queue 08:57:54 ok, jgraham, the speaker queue is closed 08:59:42 shs96c: I believe that consistency is key, and something that we should be striving for. I also believe that the "local" case is not always local: many people run a selenium server (or similar) to connect to a webdriver session. As such, optimisations, such as keeping track of a URL can't be relied upon locally. Having new session return some unique identifier (either a session id or a connection key) is consistent, and is guaranteed to 08:59:42 work 08:59:45 ack whimboo 09:00:25 shs96c: adding in support for things like finding out PIDs locally adds complexity to local ends in a way that isn't beneficial in all cases, and serves to muddy the waters about what's going on. 09:02:22 whimboo: I would like to get started with a security concern about using a session id to connect to an existing session. With CDP, we have access to all the data from the session, no matter which client is connecting. I don't see a problem when this is running locally, but when remotely it may allow an attacker to hijack a session and exfiltrate data. So new connections should be sandboxed. In classic there's just one session. 09:02:38 q+ to respond to whimboo's security concern 09:02:59 whimboo: maybe sessions can prevent any other session from connecting, perhaps through a new capability 09:04:19 shs96c: I think we are relying on https requests. I think we can rely on the security of the connection. this leaves the sec issue could be that on the local data being taken out. If it is in the URL then it is can easily be done 09:04:38 ... if its in the payload then it will be handled through the https/wss 09:05:14 ... if we block other "connections" then it will limit the cloud providers from adding the value adds 09:05:50 shs96c: I think we're relying on connections being made through HTTPS without MitM attacks. We can rely on connection security. Then the problem is whether they key can be exfiltrated. If it's encoded int he URL it can. If it's in the payload it's just like security would be if running locally. The use case we're thinking of for being able to to connect is cloud providers being able to add extra facilities. They might also want to record a 09:05:50 video stream. Preventing any other session from connecting to a session would prevent that functionality. I don't think it's a problem unless we're worried about transport security being breached. Being able to lock a session would prevent cloud provides being able to add these things. 09:06:00 whimboo: we may need to check in firefox we don't have a secure one. HTTPS will be important 09:06:01 in chrome it is recommended to use pipes instead of ws 09:06:18 q? 09:06:38 ack jgraham 09:06:38 jgraham, you wanted to mention that we're already returning the websocket URL in session.new 09:06:39 s/HTTPS will be important/WSS will be important 09:07:44 jgraham: firefox at the moment only supports local connections, so I think unless we're worried about a process on the same machine as an attacker, I'm not really worried. Cloud providers would wrap the connection. Local attackers could also read pipe data 09:09:15 jgraham: in new session we return a url that allows one to connect to the session. We could also return a new url to create a new session. For firefox, that would be the url you connected to originally. For a cloud provider that would be something different to connect to an existing session. URL would be tied to an existing instance. Lacking the URL would indicate that only one session at a time is supported. 09:09:24 ack maksim 09:09:31 ack sadym 09:10:37 Zakim (IRC): currently, we have a websocket connection, which can be upgraded to a bidi session. That's the only way to connect to a server. From that perspective, the URL we need to connect to is the browser instance. It seems like, the websocket is a bit overloaded, and it doesn't seem the right place to connect a new session to. 09:11:07 Zakim (IRC): There is a thing that creates the browser that is out of bidi, and it seems natural to put this logic in the thing that starts the browser. 09:11:35 Zakim (IRC): the problem is that it would move capabilities out of `new session` in bidi, and it has to be set before the instance is running. 09:12:12 s/Zakim (IRC): currently, /sadym: currently, 09:12:31 s/Zakim (IRC): There /sadym: There / 09:12:44 * /sadym: currently, / 09:13:00 s/Zakim (IRC): the problem/sadym: the problem/ 09:32:31 zakim, open queue 09:32:31 ok, AutomatedTester, the speaker queue is open 09:37:43 shs96c: in summary: We are agreed that we want multiple sessions, we are agreed that are opaque to each other. In puppeteer it would be good to separate . I think that connect command is useful for cloud providers. We are in agreement that needs a unique identify it. THis can be in a URL. We return a ws url for clients to connect back. I propose we add `session.connect` that connects to the ws url that is in the 09:37:43 capabilities that have been returned when starting the browser. 09:38:48 jgraham: I think this is good but there are some things I disagree with that it would be good to continue the implementation detail in the issue 09:39:40 s//initialising the browser from connecting a session/ 09:40:22 topic: Sandbox mode 09:40:35 github: https://github.com/w3c/webdriver-bidi/issues/289 09:41:55 spec draft:... (full message at ) 09:42:18 q+ 09:42:30 shs96c: to summarize from a previous discussion is that we want a way to create a session isolation for automation and working across browsers. This would be something like containers 09:42:44 qq+ gsnedders 09:44:01 Sam Sneddon [:gsnedders]: in safari concept of profiles is not really exposed and can only be changed when the browser is started. Safari is a singleton like all MacOS applications 09:44:01 ... what safari can do is have many different private sessions 09:44:04 ... and we only provide ephemeral session for webdriver 09:44:30 shs96c: Is the proposed API going to be implementable? 09:44:40 ack Sam Sneddon [:gsnedders] 09:44:41 ack next 09:44:42 gsnedders, you wanted to react to sadym 09:44:49 proposed API: https://matrix.org/_matrix/media/v3/download/matrix.org/OskbYencEkxrfnjrRBJGMHRI 09:45:05 ack next 09:45:28 whimboo: I think that it should probably sit on the browser module rather than the browsing context module 09:45:41 and then would like fit the model already in CDP 09:46:01 q> 09:46:03 q? 09:46:03 +1 to this being the wrong place in the API to define this kind of thing. 09:47:08 RRSAgent: make logs public 09:47:10 RRSAgent: make minutes 09:47:11 I have made the request to generate https://www.w3.org/2023/09/15-webdriver-minutes.html jgraham 09:47:17 I don't mind defining it elsewhere, my thought process was that it is a container for browsing contexts, therefore, could be in the browsing context. In CDP, it is in the Target domain 09:47:41 q+ 09:47:47 ack next 09:48:40 orkon: the idea again is that we can achieve storage isolations 09:48:40 ... and then you wouldn't be aware of the other sessions that are ahppening 09:48:46 ... the first API would create a container that would do some implementation specific 09:48:56 ... and container is implementation specifici 09:49:22 ... and then you create a browsing context in the container 09:49:47 ... and then the next item is that you can close all containers when shutting down the container 09:51:11 s/ahppening/happening 09:51:38 q+ 09:51:43 RRSAgent: make minutes 09:51:44 I have made the request to generate https://www.w3.org/2023/09/15-webdriver-minutes.html AutomatedTester 09:51:55 ack next 09:52:02 q+ 09:52:53 sadym (IRC): this API aligns with API already being proposed in bidi 09:53:07 ack next 09:53:30 jgraham: I think this seems implementable in Firefox 09:53:36 s/with API/with realm sandox API 09:53:43 ... aligns well with how the containers API works in Firefox 09:53:57 q+ 09:54:11 ... it comes with some limitations around temporary storage but they are being worked on 09:54:23 re: the specific API, does browsingContext.GetTreeParameters need to contain the Container? does the browsingContext.BrowsingContext not already uniquely identify the browsing context across all containers? 09:55:15 gsnedders: it might be redundant. It looked like it was needed but perhaps we can remove that part. 09:55:34 ... my question re: webkit. If you have multiple automated ephemeral sessions does it map to this API where webdriver can see everything or does ever epheremeral session isolated 09:55:48 Sam Sneddon [:gsnedders]: I don't see any reason why it can't be implemented 09:56:01 orkon: For a list of top-level bc's we probably want to have it to exclude tabs not using this container 09:56:23 ... ultimated the UI process knows about everything and everything passes through it so it probvably doesnt matter 09:56:28 q? 09:56:50 Also, I'd suggest we use some terminology aligned with https://privacycg.github.io/storage-partitioning/ rather than "Container". But that's a bikeshed discussion. 09:57:05 jgraham: I think the thing we will need to do is go through the APIs and see if need to share information about the container so people can route back to that container 09:57:23 ... or commands take an argumnet that has the container ID 09:57:50 ... there is some implementation complexity but something worth doing 09:57:55 q? 09:58:06 ack next 09:58:24 shs96c: I think the requirement about how prodfile data is stored is a impl detail 09:58:42 In particular, it seems important that we can refer to all contexts in a container now or in the future for things like event subscriptions. 09:58:52 ... and I 2nd what Sam says about using https://privacycg.github.io/storage-partitioning/ 09:59:18 jgraham: is that what we want? 10:00:07 q+ 10:00:07 Sam Sneddon [:gsnedders]: yes I think so. There are some differences we need to be aware of and this CG document is still very early stages 10:00:07 ... but the terms in there are probably what we want to be using 10:00:54 ... we should check what the term ephmeral sessions means and make sure they match with everyone is doing 10:00:54 q? 10:00:54 ack next 10:01:10 orkon: I think using terminology from partition is good 10:01:20 q+ 10:01:43 ... I think that the grouping of this going to move things in the future so it could be more than storage partitioning 10:01:57 ... I am open to any suggestions on this moving forward 10:02:01 ack next 10:02:20 sadym (IRC): API wise this one of the options and it is quite meaningful 10:02:43 ... we could also put this on to session new as an argument 10:02:49 q+ 10:02:56 ack next 10:03:28 jgraham: I dont want us to overload new session here as well as we have already discussed about this earlier 10:03:34 ... this is a top level concept 10:03:54 I am with James on this 10:04:24 q+ 10:04:28 ... I perfer the API shown above as it is simpler to follow and use and is explicit 10:04:31 ack next 10:05:41 sadym (IRC): another question is around subscription to events. What happens to global events? Do we subscribe to the container from a global session 10:05:41 q+ 10:05:41 jgraham: this is why I suggested we review all APIs when doing this work 10:05:41 ack next 10:06:25 q+ 10:06:27 orkon: re: subscriptions I dont think we would to subscribe to the container and get those as you subscribe to the context and then events bubble from that 10:06:35 ... so we don't need to change anything 10:06:39 ack next 10:07:50 jgraham: It is not part of the MVP for this to be a feature. I am sure we can land this and review. We would need to update the wdspec tests and 10:08:15 q? 10:09:44 topic: Add support for examining/manipulating intercepted network response bodies 10:09:59 github: https://github.com/w3c/webdriver-bidi/issues/541 10:10:01 ScribeNick: orkon 10:11:26 jgraham: with the network request interception, we can change req body and headers but we don't allow intercept the actual network response body and edit it in place 10:11:37 jgraham: the reason is that the body can be very large 10:12:18 jgraham: sending large bodies in a json base 64 message is problematic 10:12:50 jgraham: this is a feature request that you could rewrite the body and we should support it 10:13:12 q+ 10:13:13 jgraham: so supporting it depends on the general facility on doing streamed IO of large payloads 10:13:48 jgraham: basically, it works in CDP: instead of returning text in one go, you return a handle that allows you to pull more data in chunks 10:14:29 jgraham: I think it is the most obvious design. For network req interception you also need a way to send the data from the client to the browser (the reverse) 10:14:47 jgraham: you might need a write steam in addition to the read stream 10:15:06 jgraham: and a way to fall back to the existing stream 10:15:26 jgraham: for network request interception it will be an opt in per request 10:15:42 jgraham: so you intercept the requests and tell that you will need a body later 10:16:29 jgraham: do we want to be declared upfront or a decision during the lifecycle? 10:16:41 q+ 10:17:43 ack next 10:18:48 https://github.com/googleapis/googleapis/blob/6598bda8b438cd39440f71bbe88915587ec79c05/google/bytestream/bytestream.proto 10:19:11 shs96c: just two things: we are going to be sending binary data back and forth. JSON is not ideal format for it. At some point we might consider an alternative format for the spec, perhaps, protobuf or some other encoding supporting binary data. Second: we want to read and write memory remotely, ByteStreams might be a good fit. 10:19:24 shs96c: perhaps we don't need to create new IO mechanism 10:19:50 ack next 10:20:54 Two things. Firstly, I would prefer to decide on a request by request basis. It's more consistent with how puppeteer works, and it allows us to be more flexible (eg. reacting to other data, or modifying data for every N requests) 10:21:48 orkon: Secondly, uploading the binary data after the request, we'll need to see what support we have in CDP. It only supports a base64 encoded body. Not sure if there's a precedent for file upload 10:21:58 s/Two things/orkon: Two things/ 10:22:12 ScribeNick: orkon 10:22:21 q? 10:22:28 q+ 10:22:46 q? 10:23:07 ack next 10:23:44 q? 10:24:45 @sadym: is the use case modifying the response body? 10:25:10 Jim Evans: the point of the issue is that I am as a user that I want to modify the entire body or its parts that gets returned in the response 10:25:30 Jim Evans: not only replacing the complete body or manipulating the parts of the body 10:26:09 sadym (IRC): we can currently fulfil the static body 10:26:13 jgraham: we already have it 10:26:25 shs96c: only if the response fits into memory, currently no streaming 10:26:36 q+ 10:26:42 ack next 10:27:55 shs96c: the first thing the list of URLs is optional, the second thing is if providing the complete response will be handled in the consistent way, e.g., to indicate if it is a final part of the body. Or we always send the IOHandle 10:28:12 q+ 10:28:28 shs96c: so we need a way to get the body from the response 10:28:52 shs96c: but when we do network provideResponse we replace the body with the IO handle 10:29:21 shs96c: I guess the IO handle for read will return ??? 10:29:45 shs96c: it will return network.Bytes 10:29:55 and if it is the end it will return eof 10:30:10 and write would take a network.Bytes value 10:30:41 shs96c: so we perhaps have a read handle and a write handle 10:31:02 jgraham: it makes sense to have separate handles 10:31:12 jgraham: next steps is to define the details 10:31:25 q- 10:32:06 Topic: Web Extensions testing 10:32:11 ScribeNick: shs96c 10:32:58 q+ 10:33:58 q+ 10:34:05 ack orkon 10:34:09 orkon: we support limited extension testing. For install and remove, for Chrome it is only done in capabilities. The Web Extensions group tell us this is difficult to change in chromium. 10:34:30 orkon: is there some cross-browser agreement we can have on how to install an extension 10:34:34 i don't see an issue. probably we can file one and then attach the log in there afterward 10:34:40 ack next 10:36:11 jgraham: the thing that happened earlier in the week, the web extensions CG's goal is to incorporate web extensions into WPT, where each test will be its own extension. The API they're looking for is `install extension` and `remove extension` without needing to restart the browser every time. They may want more (eg. activate and de-activate extensions), but the main thing is being able to install extensions at run time, 10:36:21 jgraham: this would be useful for Mozilla. 10:37:08 jgraham: I think there are use cases for things other than the extension just lasting the lifetime of the browser. 10:37:15 request for geckodriver and WebDriver classic: https://github.com/mozilla/geckodriver/issues/1476 10:38:00 orkon: was told that installing extensions at run time was "practically impossible" 10:38:19 q+ 10:38:28 ack next 10:38:45 shs96c: do we want to make this a problem for the web extension CG to solve using a webdriver bidi extension? 10:38:55 q+ 10:38:58 orkon: perhaps, yes 10:39:05 ack next 10:39:45 whimboo: in geckodriver we can install an extension at runtime. Can even do this in private browsing mode. 10:39:52 q+ 10:40:03 whimboo: the goal is to avoid changing the payload of the test per browser 10:41:15 whimboo: the most important short-term goal is to be able to install extensions at `new session` time by having a standardised capability containing the extensions we want to install. Both Firefox and Chromium have support for this. 10:41:44 jgraham: do we think ignoring the web extensions cg stuff, do we have use cases in firefox that require us to install and remove extensions at run time 10:41:47 q+ 10:41:53 whimboo: probably not 10:42:25 whimboo: I can imagine that extension authors might want to test the `install` and `unload` scripts are being correctly called 10:42:32 ack next 10:42:41 ack next 10:42:48 ScribeNick: orkon 10:43:28 shs96c: I remember from the Selenium project what SauceLabs has said: being able to install extensions at runtime is really useful because it allows having the browser ready to go and then users amend it with extensions at runtime 10:43:55 q+ 10:44:01 shs96c: it is debatable if the time is any different but looks like it's use case 10:44:06 ScribeNick: shs96c 10:44:09 I created an issue: https://github.com/w3c/webdriver-bidi/issues/548 10:44:13 ack next 10:45:00 whimboo: whenever people want to test extensions, enabling and disabling is useful. It would still be good to have extension installation at runtime 10:45:41 q? 10:47:14 jgraham: summarising. There seems to be a general appetite for a capability for installing extensions at start up time, and that's something we can standardise. Runtime seems more of a requirement of the Web Extensions CG. The CG cannot write web standards, so for practical reasons it might be easier for us to write the standard, but they could also specify a non-standard track document which specifies the whole thing 10:47:27 sgtm 10:47:41 shs96c: Is anyone opposed to us adding that capability ourselves? 10:48:07 Zakim (IRC): I've filed issue 548 for this 10:48:10 q+ 10:48:50 github: https://github.com/w3c/webdriver-bidi/pull/421 10:48:52 shs96c: shs96c: fyi it's sadym (IRC) 10:49:35 q? 10:49:39 ack next 10:49:40