Browser Tools- and Testing WG meeting, Thursday 9 November

09 Nov 2017



jgraham, ato, jimevans, clmartin, soareschen, AutomatedTester, alrra, simonstewart, JohnJansen, juangj, MikeSmith, rbyers, auchenberg
ato, JohnJansen_


<jgraham> RRSAgent: make minutes

<clmartin> scribenick: clmartin

<ato> brrian: We are starting the meeting momentarily.

<JohnJansen> Bbrian is in the car. I’m going to call him when we start.

<ato> OK

<ato> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F

<MikeSmith> agenda: https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F

ato: need to discuss what we have to complete first/prioritization of the agenda

AutomatedTester: doing introductions
... Moving on to the next item in the list, we have a state of the union so we can describe where we are

simonstewart: We are currently at candidate rec. in order to transition from cr to pr (proposed rec) we need the w3c to give us the thumbs up.
... to do that we need proof of interoperable implementations of the standard. we have w3c versions of the protocol via iedriver and geckodriver

jgraham: do we have a url?

simonstewart: we do and david will get you those. we need to come up with a convincing argument for why this is good enough or we need more demonstrations of interoperable implementations
... that's where we are
... do you have anything else to add david?

AutomatedTester: I don't think so. we have other things we want to move forward on but we want a better understanding of the process before we do that which is why there are a large number of pr's on the GitHub repo
... they're on a separate branch

jgraham: for example we're waiting on new features that can be merged however they all need tests

<simonstewart> Test results can be found here: https://w3c.github.io/webdriver/results/2017-10/html/all.html

ato: to expand on what jgraham just said, we now have a policy that any change to the spec needs to be coupled with tests going into the web platform tests which seems reasonable as we now have our test suite up and running
... I wrote an email to the mailing list because as an implementor i'm not concerned about it moving to rec so much as having a living document that reflects the current reality. there are pr's in the queue that actively prevent marionette from implementing and it's only going to get worse. automatedtester branched off so we can merge things from the queue into that branch

rbyers: i assume the test match the master branch?

ato: that's correct

rbyers: what are firefox's results?

simonstewart: I've posted a link to that

AutomatedTester: that's where we are currently. the key part is moving forward we need to see where everyone's implementations are

<rbyers> There's also live results here: https://wpt.fyi/webdriver

AutomatedTester: we can see between geckodriver and iedriver what's passing/failing. feel like it's doing a bit of disservice as we don't have something for chrome
... or for edge or for safari, we should expand on that

<AutomatedTester> https://w3c.github.io/webdriver/results/2017-10/html/all.html

JohnJansen: mind pasting the link for the results?

simonstewart: that's really easy to generate btw

jgraham: if you want we can project the results

AutomatedTester: i'll start with geckodriver implementation. for the most part we are doing quite well. we've been from the wd spec tests passing a large majority of it. there are a few parts where some things have changed and we need to fix those up but in general terms it looks like if we kind of implement a major feature of the specification our implementation results goes up quite significantly which is around alert handling
... which we don't have support for yet. it is a priority and we are going to be working on it sometime soon. i'm not sure how much detail people would want to go into from geckodriver's point of view but feel free to ask any questions on what you feel might be missing. i do think in general terms that the test suite needs to be expanded where possible because we need more coverage/better coverage

jgraham: there's two things, one is do we think that the testing could be better? yes. do we think that that should be a prerequisite?

AutomatedTester: no i don't think we should gate necessarily on this being a full test suite

ato: can i expand on that. as AutomatedTester said we are passing a majority as we wrote them, but i do suspect as more vendors right more tests will run into problems. over the last year we have made some strides on the spec, navigation, page load strategies, pointer interactability with help from google for edge cases
... in addition to that in the past year we've done lots of refactoring on how marionette does window tracking
... I've seen the light and think we'll make progress into the next year

<ato> Ah, I forgot to talk about wpt.fyi.

rbyers: let me do a quick framing from google. big picture - we aren't investing enough. this group knows sam, he's no longer at google. john started recently on our interop team but that focuses on our interop tests. i think we should be investing more but i want to understand the arguments/where everything else is to sell back to the larger org. with that john has been working on webdriver

kereliuk: one of the things that took a while is actions
... i have to go through all the commands in our current implementation and verify that they conform to the spec. a lot of the code in chromedriver is written by people i have to track down. aside from that the next steps from me would be (once that is merged/good) is making sure i can get a good sense on the interop suite. ran into some issues, there's a lot of overhead to that like set window rec
... very soon we should be able to have a nice interoperable implementation in the next 2-3 months

simonstewart: one of the things we've tried to do with the spec is make the delta between the json wire protocol/w3c as small as possible. new session/actions are where we diverted. a successful result for instance look the same.
... hopefully when kereliuk says you need to go through and review that it's mostly painless.

kereliuk: I've done a solid 50% of that and most of what I've encountered is around error codes/error messages

jimevans: we don't properly check for the tlbc being closed

kereliuk: setting timeouts is different

simonstewart: yeah

kereliuk: should be some fixes in the next release for the ruby bindings
... if anyone has any specific questions please ask
... i'm the one who works on chromedriver right now

simonstewart: would you like if i flipped the selenium project to run the w3c tests against chromedriver?

AutomatedTester: how would one do that for chromedriver?

ato: i don't think a selenium discussion is relvenat

jgraham: a lot of the fixes won't be there as we're using the latest release chromedriver and not the latest master build

kereliuk: i want to have nightly release for a project like wpt. they are available now but you have to login with a google account. we did this so we could fix wpt quicker.

ato: sounds like there is a little plumbing we need to do

jgraham: for wpt.fyi in particular as some people are interested in the conformance report more than that, there's a url we can get at that has a link to the nightly and that is kept up to date and kereliuk could consider providing the same

rbyers: we should also be clear that at the moment our priority is not to pass all the wpt (web platform tests) but ensuring we can automate wpt with webdriver, which is why we did pointer events first.
... tell me if you think that's the wrong priority

alex: not on irc, i represent CoSmo, we develop a tool called kite to test for webrtc. we want to test with webdriver
... don't know you guys yet so come meet me at the break so we can work together on bugs when we find them for webrtc testing

ato: are we done with the chromedriver update?

<ato> https://bugzilla.mozilla.org/showdependencytree.cgi?id=721859&hide_resolved=1

JohnJansen: for edge we have an interesting case for wpt.fyi in that our webdriver has to match the branch/build we're testing against. we have fairly current availability on browserstack but there are still some hiccups but i can submit them locally and get the results. i have a guy debugging it right now back in redmond. i think the tests do a check for the session to see if it's a w3c session and if it's not it'll stop. we'll pass 80% right

now but we're not w3c compliant.

JohnJansen: we're at about 80% but i don't know if that accounts for the current failures

jgraham: for what it's worth previously for chrome i was happy to add a hack to wptrun which is what wpt.fyi is using in the tests to say you can start an old style session so you can have the tests run. in the end chromedriver fixed it but i'm personally not opposed a failure in new session fails the new session tests not stopping the rest from running

ato: i think for slightly different reasons that making everything run on wpt.fyi is the right priority for the working group

jgraham: i think having it run there is relevant for moving the specification forward but it's just good in general

rbyers: when you change code you want to see how that impacts results.
... can't see how you would move forward without those results.

ato: not disagreeing but we have internal tests we pass that fail in wpt.fyi because of issues in wpt.fyi

JohnJansen: wpt.fyi is new so we can't rely on it yet for results

jgraham: i don't think the people who are working on making the drivers better are not the same people working on bug fixing wpt.fyi. it makes sense that we both have people work on the driver to make the conformance better and also working on wpt.fyi
... i mean the shared testing stack for wptrunning, the fixtures, the build port, etc.

AutomatedTester: we can finish discussing wpt.fyi in a minute, want to get through impl status

JohnJansen: we need to prioritize. we've been looking at internally running the tests that exist on the w3c GitHub wpt repo as well as our internal so we make sure we don't break things as we move forward. i have no worry about the high level interop, it's in production/being used by lots of teams across msft.
... not concerned with rec moving forward/being able to be interoperable it's just a matter of prioritization/time

AutomatedTester: any more question for john? no.

brrian: i'm here now

JohnJansen: we were doing status, what is apple's impl status?

brrian: not there yet but it's started. we'll be using the same engine hooks that the gtk driver uses.

ato: we should get an update from jim

<simonstewart> GTK WebDriver: https://blogs.igalia.com/carlosgc/2017/09/09/webdriver-support-in-webkitgtk-2-18/

jimevans: one of the biggest challenges was that for a while the wpt tests used a js construct that ie didn't support
... got some stuff he needs to fix and get merged back in
... other than that it's a matter of allocating the time to fix up the bugs that are the failures in the test
... i'm committed to getting it done. if there is a failure in the tests right now then it's a bug.

JohnJansen: ... do you know any that can be fixed? are you worried about risky bugs/

jimevans: you mean bugs from the impl standpoint?

JohnJansen: yeah

jimevans: no, only a couple that are real challenges such as fullscreen

ato: i think that should be fine as the fullscreen endpoint is one of the few commands we allow an error if it's not supported

jimevans: right, i don't think i have anything blocking my impl. it's just a matter of fixing bugs.
... nothing i know of that would cause regression behavior that i know about

rbyers: can we also hear from the people using webdriver
... things like how well it's meeting needs in terms of functionality
... i'm trying to understand separate from moving the spec to rec how well we're doing

ato: it's a good question but the only impls we have right now are geckodriver/iedriver. most of the bugs we've had in the last 12 months have been due to selenium problems such as something hasn't been wired up in geckodriver or selenium

simonstewart: i think one of the things that's really important here is from a users point of view they're using selenium, they don't care about the protocol. they don't care if it's json wire of w3c, it all just works. the actions api are a significant step forward.

ato: the actions api is what we inherited from before in that selenium hasn't exposed everything

simonstewart: the java ones do now

ato: excellent

simonstewart: you can do new actions.tick and you can specify anything you want and it will run at the same time.

rbyers: that's not supported in any browser yet right?

simonstewart: geckodriver/iedriver support it and soon chrome

AutomatedTester: the new design is awesome because we can do new things.

jgraham: should we hear from the people using it now, we've gotten derailed

juangj: it's not perfect yet. it may be that complex gestures work with the new api but until it works in every browser people aren't interested in using it. from surveying existing actions users no one is making an action change longer than about 3 things as it's complex/risky. moving forward that should improve and it should meet peoples needs. rbyers mentioned people wanted pinch gestures and the like. optimistic about it meeting peoples

demands but that's still in the future.

juangj: there isn't a good way in expressing that in the old apis so until the new things are ready they're hesitant to move off of the old api

JohnJansen: from sharepoint they aren't ready to move off of it/emails i have from sharepoint guys they're happy with how it works right now and aren't keen on putting risk into the system at the moment until later.

rbyers: i feel like we're de-railed on actions
... how is adoption going for webdriver across google juangj ?

juangj: mobile is the biggest complaint we get, with mobile it's touch actions. as far as other things it's permission apis/geolocation
... similar kinds of things where there is a user prompts outside of the DOM/in the browser chrome. although those aren't really things you need to cover in the test.

jgraham: you can disable that in chrome yeah?

juangj: yeah

simonstewart: fixing that comes under the door hangers

juangj: that particular feature you wouldn't expect webdriver to addres

ato: it's on the agenda

<boazsender> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F

<ato> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F

JohnJansen: can someone paste the agenda

<AutomatedTester> RRSAgent: make minutes

juangj: when you say are people successfully using it? yes but they're coping with the features that are available. if there are grand features that people wish existed/have dreams they can do something they have given up as this is what we have/what we can do now

rbyers: what about interop, do they have issues with cross browser

juangj: from a webdriver perspective no, for interops sake most people expect that they will work across browsers and they mostly do except that they use the legacy firefox driver. among people who have migrated interop has been good.

simonstewart: how many selenium tests are still being run. when i left google every team had a project being run.

juangj: there are no tests using selenium, they use the api
... there are several thousand webdriver tests

JohnJansen: internally the thing that i hear about the most is not a bug in webdriver. we've joked about having a conversation in visibility. the edge browser itself reports things as being obfuscated/not seen on the screen. displayedness is tough.

ato: like i said earlier to do with pointer interaction, we had google run geckodriver with a special capability that has the new pointer work and we have found more problems but those aren't problems in geckodriver but problems in the spec/impl of other things

rbyers: how often does juangj take new chromedriver release

juangj: they do cause problems, we don't take new chromedriver drops as quickly as kereliuk updates as they cause problems. if there's a bug fixed then someone inevitably was accidently relying on that bug.

jgraham: do you still have people using firefox 47/old firefox driver?

juangj: yes about a quarter of them are still using the firefox driver
... some people were waiting of geckodriver to implement. mostly it's people procrastinating. there are a lot of tests relying on pointer interact ability not being implemented.

AutomatedTester: moving on to sauce labs

titusfortner: i think most of the interop things i see are timing issues between how the DOM is rendered. i don't anything specific to the protocol that causes issues. most users swapped from firefox to chrome.
... maybe that will move back with w3c, i'm not sure

juangj: i see people moving from firefox to chrome as they wanted to avoid the change.

titusfortner: a lot of people defaulted to firefox when running locally because it didn't require an extra download.

simonstewart: i think people find the upgrade process from legacy firefox/selenium to be tough. they change everything at once and they get burnt and blame geckodriver unfairly.

juangj: it's a bit daunting, there are some cases where you can't change only one thing at a time.

AutomatedTester: are there any other questions for sauce labs
... mike since you're back, we have an implementation updates/how people are working. people have offered to create impl reports. what are the things required of this group to move forward? what will be expected? how do we make everyones life simpler

<ato> RRSAgent: make minutes

MikeSmith: we need to evaluate our exit criteria and see how we are doing
... can we look at the exact language in that

rbyers: it's similar to other groups. every test must be passing in at least 2 places.

AutomatedTester: so there are a number of things that are failing at the moment or we need to fix our failures.

jgraham: if you look at the current impl report there are two classes of things you see. one is things that are unimplemented, you're not going to convince anyone that something without two implementations can go to rec. if the goal is to get to rec in the shortest time possible we should drop the things that aren't implemented.
... for the other things we have implementations but we have a strange situation such as some of the tests failing in iedriver not because of the tests but because iedriver is buggy.

<MikeSmith> https://w3c.github.io/webdriver/results/2017-10/html/all.html

jrossi: i agree, one thing we've done in other working groups in the second phase is the opportunity to provide comments. for pointer events we didn't have two tests passing because it was a browser bug but if that's the case then plh was okay with that.

<MikeSmith> https://w3c.github.io/webdriver/results/2017-10/html/less-than-2.html

AutomatedTester: i don't feel like we need to remove things to ship.

jgraham: it depends on what you want to happen. if you want to reach rec before the time it would take to implement those features. it's quite meaningless to do. you move them from the rec to the living doc.

<MikeSmith> https://w3c.github.io/webdriver/results/2017-10/html/complete-fails.html

jgraham: i agree aesthetically it's not pleasing but the reality is that we'll be gated on getting to rec on having those things impl'd.

rbyers: we did that for chrome, moved things that didn't work out of spec into incubation

simonstewart: seems like it's mainly around alert handling

jgraham: there are issues with the report

simonstewart: we need to update our drop from wpt

ato: about user prompts it wouldn't be extremely hard to add to firefox but when it comes to returning web windows/frames the serialization of that when returning from execute script, we won't be able to do that in a reasonable time

simonstewart: sounds like we should yank execute script window handles from level 1
... and leave alerts there

jgraham: if we do that we need to get commitments

simonstewart: agreed we need to chat with people setting priorities

ato: don't care either way as we have something that works but the safe call seems to be yanking from level 1 and put it in master

rbyers: you should always yank it and if someone implements bring it back

simonstewart: the user expectation around returning window id's from script isn't there.
... prompts however are

titusfortner: different drivers implements that differently

simonstewart: they all do it

titusfortner: chrome doesn't

ato: if that is true than that puts us in a slightly different position

titusfortner: we have bugs around this

jrossi: one of the things we did in pointer events was amending the test results table to add in link to everybody's issue tracker to see what's failing

ato: you mean for making the report more valuable to plh?

jrossi: more in a way to track where the pain points are but it would also be helpful for reporting to plh

MikeSmith: ideally as far as test results go we don't want to see any red. right now superficially for anyone who doesn't know anything about webdriver won't be confident about those results as there is a ton of red there. you can explain individually why it's there or. there are two things we can do is either yank the things that aren't there and re-generate the results and go back to cr
... if we make substantive changes to the spec we have to go back to cr
... we could make the argument that removing things shouldn't require going back to cr
... we need to wait 4 weeks after publishing a draft for comments to come in
... when we remove features from the spec it invalidates previous reviews. when someone approves the cr based on old version they need a heads up on us removing features.
... in practice people may not care but it's more time
... we're out of time already, so we'll need to take more time anyways, it's just a matter of what we need to do
... we can remove those features and restart the cr with the new test results which may mean we get through cr fairly quickly and move on

simonstewart: we should have a convo around priority setting and find out

AutomatedTester: we remove it from the spec and move it into master, it seems silly to remove the tests and then regen the report.

MikeSmith: we can keep the tests but to generate the impl report we'll need to generate from a subset of the tests

jgraham: right that's trivial, keep a set of tests that we know apply to level 1 and filter the rest out

MikeSmith: right we just need to filter them out. if we have a few cases of red that's okay but we need the report overall to look good/green.

ato: for example ie11 has a bug that's okay?

MikeSmith: right we can have that ready to go but normally this is explained over the phone and as long as we're confident in that explanation we should be okay

jgraham: sounds like we should get the latest chromedriver into the impl report as it might add passing results for tests

kereliuk: at least publicly you can go to the chromedriver website and click a link

jgraham: someone should generate those results

rbyers: is it different from what we have on wpt.fyi

kereliuk: trying to think of what was released previously

jgraham: wpt.fyi downloads it for each run

<kereliuk> Latest stable release: https://sites.google.com/a/chromium.org/chromedriver/downloads

MikeSmith: okay so as far as discussion with plh goes he said he can be available after lunch. he can be available at 1

<kereliuk> Latest master:

MikeSmith: he has something at 3 so if not 1 then some time before 3

<kereliuk> https://sites.google.com/a/chromium.org/chromedriver/chromedriver-canary

MikeSmith: it's 10:30 so we should break?

jgraham: why are we still here?

AutomatedTester: we'll be back at 11

<AutomatedTester> RRSAgent: make minutes

<juangj> amending earlier statement that “there are no tests using selenium” — this is not accurate as I misinterpreted the question. The vast majority of browser tests that we run use WebDriver, and virtually all of those tests use Selenium. There may be a handful of people attempting to use the WebDriver API directly but they’re negligible.

<juangj> for whatever that's worth.

we're resuming

AutomatedTester: moving on, the next steps are to fix up the agenda and items that we want to discuss

<boazsender> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F

AutomatedTester: there's a whole bunch of things at the bottom that people would like to discuss at some point today
... We're going through the topic bank and move them up to what we think are important and flesh those out

clmartin: can we move new tab/delete history up, they're not controversial

simonstewart: in an equally uncontroversial world, file upload

ato: just to make it clear to everyone, we're now talking about features that will make it into webdriver in level 2
... this will be stuff that if potentially agreed to will go into the master branch

JohnJansen: for clarification, the full path to the branch that has v1 is that a branch or a url?

jgraham: what's the question?

JohnJansen: i want to make sure i'm looking at the v1 version of the spec or the v2 version of the spec

<simonstewart> https://github.com/w3c/webdriver/tree/level-1

clmartin: master is latest?

simonstewart: master is the living spec

jrossi: do you want to add to the v1 branch, change the title to say level 1?

ato: i have an action item to fix the url

<jgraham> ACTION: ato fix the urls for the spec

AutomatedTester: computer problems :(

simonstewart: are you building Mozilla?

AutomatedTester: i might be :)
... stopping from the top, copy/paste/selection

rbyers: are we still talking about the agenda? do we want to summarize what we're going to do with devtools

simonstewart: can we break these into things that are amendments, brand new features and amendments to existing features
... as right now we're randomly cherry picking

jgraham: if rbyers is here for the moment to talk about the debug protocol we should do that

rbyers: we're talking WebVR at some point, would we want this group to discuss the bigger picture of test adoption requirements?

jgraham: let's prioritize that first then

AutomatedTester: we'll start with re-chartering and proposed debug protocol stuff
... as part of when we started the work to move to pr, i created the draft for re-chartering this group for this work
... one of the things i have suggested which used to be part of this group was how we do debug protocols

<ato> I’ve updated the agenda: https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F#Thursday

AutomatedTester: wilhelm originally added it for the opera debug work and now I've added it for the cdp

Rechartering and proposed debug protocol standardisation

AutomatedTester: last night there was an informal chat on if we were to do this how it would look. the consensus was to remove it from the working group charter and move it to an incubator group and build it out that way
... because while people agree that working towards a standard for the debug protocol is what we want, the process around what we want to do in a meaningful way isn't there yet
... which is what's expected when trying to do things through consensus
... from that point of view we'll be setting up an incubator group, working with rbyers and others to set this up

MikeSmith: by that you mean a community group?

AutomatedTester: yeah

rbyers: key thing i wanted to see is, we create community groups when we want to collaborate, and the goal is to have a place where we can discuss/share docs/etc.

auchenberg: should we go ahead and make a suggestion here that we create an incubator group and each vendor can say whether they support that?

ato: yes and mike will talk about this later but that the debug protocol won't be a part of the re-charter

<Zakim> MikeSmith, you wanted to speak about rechartering

rbyers: we see good engineering as being more important as it needs to come first before the process/legal stuff
... the DevTools team in particular wants us to focus on code

jgraham: there's a few things there, one is the debug protocol stuff which is unrelated to the existing webdriver work which there is agreement on that happening separately. and then the other is that mike is saying we could also move all of webdriver into a community group and not have a working group until later when ready to publish. i'm not clear on what the tradeoffs are there on whether we can still have meetings

MikeSmith: we can still have space for those at TPAC

jgraham: my understanding is that the space is limited for that

MikeSmith: no it should be viable. jrossi has a question

jrossi: we'll need to re-charter

MikeSmith: no we just need to extend

jrossi: i view the webdriver work as very different from cdp. cdp is very nascent, webdriver is fairly mature from an adopted perspective.
... our perspective would be continue webdriver in a working group but the DevTools stuff on the other hand is a perfect example of what should be incubated

AutomatedTester: webdriver has goals to move forward with

jgraham: now that there exists a spec and it's clear what the requirement is for the editing process, the group has now learned how to write a spec/tests, we can potentially focus on a few things that everyone has agreed to implement in the short term and publish a new rec more often
... if we believe we can move towards that model then it doesn't make sense to shut down the working group and then spin it up again. it just makes sense to publish stuff more often. personally i'd like to see it go in that direction

AutomatedTester: i'd like to see it go in that direction for webdriver
... ato

ato: what is the cost from Microsoft to move from a working group to a community group?

jrossi: any group we join requires a legal process, it's totally doable, but webdriver's scope/deliverables are fairly straightforward/rubber stamped
... if there was a good consensus we could make that happen

rbyers: if we do incubation in the wicg there's no additional cost to the user?

jrossi: there is

ato: i think speaking for myself that over these 6 years there has been overhead in having a wg (working group) for webdriver and the value isn't clear to me. however i see jgraham's point that we now have the right information. so if we adopt a model where we can ship faster that seems reasonable. that said the wg model for webdriver has historically been important because we started with this not having all the browser vendors onboard. 6

years ago Microsoft/apple were very different so historically having the patent protection was important. if browser vendors are happy i'm okay to continue the current model, i just want this to move forward and have investment from everyone

AutomatedTester: auchenberg

auchenberg: i'd like to separate the convos here. there's one about webdriver and it's future and one about cdp and it's future. there are a lot of people here for cdp incubation. my suggestion is we move the incubation work for cdp forward and then allow the webdriver work to continue after that.

AutomatedTester: so rbyers wants the vendors to agree

rbyers: i just want to see who want to work in the wicg

ato: i'd note this group doesn't have everyone who's interested in working in the wicg. we have a DevTools team and we don't want to speak for them.

rbyers: i just want to see who has interest

jgraham: the point is to get the official consent

brendyna: Microsoft agrees

RESOLUTION: CDP is going into an incubator group/CG. There is sufficient support from vendors and independent parties.

rbyers: Google agrees

titusfortner: Sauce agrees

boazsender: Bocoup agrees

rbyers: next step is get a repo created and continue the convo on an email thread

JohnJansen: do community groups have chairs?

rbyers: no, wicg does but not per subset

brendyna: one last question, do we have Mozilla interested/

AutomatedTester: i think so but since Harald isn't here I don't want to speak for him
... the general consensus is yes but need to verify

auchenberg: any comment from apple?

brrian: i'd need to ask the DevTools team

AutomatedTester: fair, we got the two we needed and we can move forward

jgraham: we can move forward now

rbyers: second one was about the process for other groups to ...

ato: one second, can we have a resolution on what we're doing with this wg

JohnJansen: yeah

jgraham: it sounds like Microsoft want to keep publishing recs and that therefore means there has to be a wg

JohnJansen: we care about recs

<AutomatedTester> RRSAgent: make minutes

jgraham: so does it make sense to continue the wg or drop out and spin something else up

JohnJansen: keep the working group, don't drop other things. we should extend the charter in a way that allows apple/Microsoft to continue their participation in a way that it's royalty free

AutomatedTester: are we happy to do the extension? Microsoft has agreed, Mozilla will, my sense is apple needs it

brrian: we need it

AutomatedTester: that's fine

jrossi: do we want to re-charter for what we do after webdriver

jgraham: MikeSmith what's the difference between extending and re-chartering?

MikeSmith: we need to extend first, then get to rec, then declare victory. that's separate from re-charter
... after we extend, if we're going to have another charter. we can't extend the same charter we've had for 6 years.
... i'd also say it's not a given that if we decide we want to re-charter, that the w3c will say we can re-charter
... we'll need the w3c to decide if it's something we want to re-charter for
... w3c will take into account if vendors want to continue, and if we do then that'll be take into account

AutomatedTester: how do we do webdriver extensions for other wg's to add to webdriver for testing
... will it be better to have a wg for that scenario or will this fizzle out and hope you speak to the right people? how does that play out?

MikeSmith: it doesn't make any difference. e.g. in reality the technical group for html has been outside of the w3c so it's quite possible to have solid technical communication about work that isn't in the w3c. whether it helps across wg's i'm not sure it makes a difference.

AutomatedTester: what i mean is, potentially there could be no one to do those technical reviews for other wg's

jgraham: the question is does the work depend on there being a wg?
... i think the answer is no

AutomatedTester: WebVR as an example. will it be easy for a technical review to have a wg to reference?

MikeSmith: i don't think that will matter

AutomatedTester: if it doesn't matter then that is fine

jrossi: I've been in groups before where there were questions about the state of other specs

MikeSmith: the current policy does favor on normative reference/specs published in w3c space. the gist is from the w3c side the group doesn't have to be in complete agreement that a wg is necessary but we need a least a couple of people feeling strongly about a wg being necessary to tell us that

AutomatedTester: okay, we at least got 2 groups that could say that. apple want an extension. we want to move to a level 2 and have a list of goals. to me i don't think re-chartering sounds like a bad idea. we have goals we can work towards/ship. i agree with jgraham that we'd want to do things faster but to our defense we've been learning a lot along the way and i think we can now work faster moving forward.
... so i guess if people are happy we can put it to a vote

JohnJansen: we should vote on the extension and then the re-charter

jgraham: does anyone reject a renew?

RESOLUTION: Agreement to extend the charter until the specification goes to rec

<ato> +c

AutomatedTester: we'll have a mailing list later about re-charter

jgraham: sounds like though in the room that's what people want

JohnJansen: from a policy perspective does MikeSmith take an action?

MikeSmith: ... yeah

<scribe> ACTION: MikeSmith to follow up on renewing the charter

jgraham: the sense in the room is we're on board for re-chartering even if that's later

JohnJansen: next thing we need to do is decide if we can get to rec in 3 months or 6 months
... last time was 6 months and that was long

jgraham: think we're running out of good will

jrossi: I think that's why having a re-charter plan will be meaningful

ato: as long as the w3c is okay with us then we can continue otherwise we need to go somewhere else

MikeSmith: we're happy to continue having the wg but it's been delayed for quite a while/longer than it should have but i blame myself for not driving this.
... I need to come back with an answer and say we have strong support for keeping the wg going and that we won't make the same mistakes
... but we got that so we're okay

jgraham: specifically for charter extension do we need to pick an amount of time?

MikeSmith: we should say 6 months, plh may have problems but we have 4 weeks after we return to cr and then more time after. realistically we need more time

JohnJansen: plus holidays, 6 makes sense

jgraham: practically, to set the right exptectations around decisions. that gives us a maximum window of time to make adjustments like getting implementations done and/or removing stuff from the spec
... seems like 2 months of just process, which means that, with buffer, probably anything we're talking about should be implemented in the next 2 months
... so in regards to removing things or can we do it, it will be can we get it done by end of the year. mid january
... if this is a thing implementers are going to do quickly then we don't need to remove it, otherwise we do

JohnJansen: we also need to fix bugs in the test which cause us to think we're failing things we aren't

jgraham: for the details i agree, for other things like user prompt handlers, when people have this conversations had elsewhere, the framing people need to keep is that we have 2 months

<boazsender> scribenick: boazsender

Hooks into WebDriver from other specs

rbyers: the next item is, as we're pushing for web-platform-tests be the only way we do integration tests, and the biggest blocker has been automation. and we finally have a way to call testautomation.click() and it works. now we want to open the flood gates, to give a path for new features in chromium to use webdriver extensions.
... so the advice we've given for now is, look what we did for click, coordinate with BTT group, and write a web driver extension spec.

<scribe> scribenick: clmartin

rbyers: wanted to make sure this group was okay with that

AutomatedTester: this group agreed in TPAC/japan that this group wants to do that

boazsender: wanted to do?

AutomatedTester: the idea was the group would do extensions to webdriver. obviously some things are webdriver specific that possibly people will need help with but ideally just sending it to the mailing list and we can help will get things into somewhat of a process without it being to heavy handed. ideally it won't be heavy handed at all.

rbyers: i'm hoping to make this work effectively that we're close to telling every engineer that you can't make changes without wpt and we'd need webdriver to be ready

jgraham: i think this is a question about the policy here. if it doesn't make sense for web developers to use they should use dom apis, but we're also bad at what web developers needs. Bluetooth has a dom api but there are use cases to test Bluetooth that maybe should end up in webdriver

clmartin: if it's vendor specific it can also go behind a vendor specific webdriver command/capabilitiy

jgraham: right but it's more about it if it makes sense for webdriver

ato: for existing features finally webdriver is coming into play for automating wpt

<boazsender> https://github.com/w3c/permissions/pull/151

ato: i think the next question is what to do for other wg's who need to test their spec. e.g. webdriver doesn't have a great story for permissions. bocoup had an example of an improvement for the permissions spec. it's very exciting.

<boazsender> https://bocoup.github.io/permissions/index.html#automation

ato: been trying to follow up with bocoup as there is some confusion about how they hook into webdriver. it's in the spec that webdriver wants to allow other wg to add it in their own spec
... we have a job to do on following up with wg who want to use webdriver that we have the primitives necessary to adopt webdriver.
... so far we've had issues around things like errors

<kereliuk> ^the bocoup link is out of date from the current automation branch on bocoup github

ato: had a meeting with WebVR and they seem to have a fairly solid testing strategy as fair as i could tell and nothing that conflicts with the way webdriver works so they're going to drop by tomorrow

AutomatedTester: boazsender ?

boazsender: the thing that we experienced making this proposal was a lack of clarity on where it should happen. so an agreement from this group on how to do this it would unblock a lot of other people doing this. yesterday in the testing breakout Vincent was asking similar questions on Bluetooth/usb.

jgraham: i think the agreement exists in principal, the idea that people should be able to extend it in their own specs isn't something people have been opposed to.
... are you asking for documentation?

ato: the way that the terminology around extension commands is currently phrased, it's about vendor extensions, but in another chapter we talk about extension commands in a different context to be used
... we need to tighten that up as well

simonstewart: the original idea was to use a vendor prefix for experimental commands, and then move it to the spec when you're ready
... however it's also totally legit to do that in your own spec

jgraham: this should be clear in the spec, you must not use the extension point because it's for vendor specific thing

boazsender: for documentation/direction. even as early as the wicg people should be thinking about how they create a testing interface for that
... it'd be great to hear an example

rbyers: everything that comes will be standardized but part of incubation is we want it to be safe to fail. web Bluetooth is chrome only right now but we want to be able to fail but still be plausible to be a spec

simonstewart: so if web Bluetooth is a spec then define it there, if it's chrome specific then use the vendor specific

<scribe> ACTION: simonstewart to clean up documentation on how to hook into the spec.

boazsender: and documentation for how to think about testing your spec

clmartin: we care about this looking holistic/uniform across the api

simonstewart: i'll clear that up in the spec

AutomatedTester: ideally if a chromium engineer wants to do web Bluetooth and they write a command and JohnJansen says it needs work we can iterate on that

clmartin: what about if there's a disagreement

AutomatedTester: w3c should have process to settle disputes reasonably

<boazsender> RRSAgent: make minutes

AutomatedTester: should we break for lunch?

lunch time!

<ato> Scribe: ato

plh explains process

plh: Besides two features, my understanding is that you are almost ready to move to recommendation.

AutomatedTester: Yes, but not everything is green.

plh: There are edge cases, sure.

AutomatedTester: For all of those we have bugs on file, and we expect people to implement it.

plh: So besides these two features you would be able to move to rec?

JohnJansen: I believe so.

plh: If it makes sense to you, I can try to make sure it makes sense to the director as well.

AutomatedTester: We got some tests that are failing, but we have bugs on file that show that people are going to do that implementation.

ato: Was there agreement over lunch what to do about these features?
... Do we rip the features out? Would that make it easier to move to rec?

plh: I think you should publish without these two features.
... I am not asking you to remove it from the master branch.
... In the mean time you continue work on v2.

AutomatedTester: We do want to have those features.

plh: I would like the master branch to be the v2 right away.
... Can you publish an editor’s draft of v2 next week?

AutomatedTester: Yes.

plh: It is independent from the CR goingto rec.
... You can publish another rec right away, if it is supported by vendors. It would not take you two years.

simonstewart: If we drop these two features, we could move to rec right away?

plh: Yes, this means the specification is not at risk.
... In two months you would get another approval from the director.
... Then you would get your recommendation.
... You will keep focussing on master and the features you want to have in the browsers.

AutomatedTester: If we are dropping these features and it kicks off process, it is going to be two months/four months?

plh: I am going to give you precise dates.

clmartin: If we remove something from the CR, does that retrigger the CR process?

plh: I will ask legal.

MikeSmith: Is it really a legal thing?

plh: Yes.

MikeSmith: We would prefer not to go back to CR.

plh is showing a presentation with an astounding amount of detail.

<plh> https://w3c.github.io/spec-releases/milestones/?cr=2017-11-16&noFPWD=true

AutomatedTester: Thank you, but I’m not after specifics.

plh: 60 days exclusion opportunity last date with a PR 12/14/2017 will be January 15 2018.

<scribe> ACTION: plh to check with legal if we can move to PR.

simonstewart: Answer today would be nice.

plh: I will check.

AutomatedTester: The next thing, if we go through this process, we also need our charter extended.

plh: If you can guarantee how long it will take, I am willing to argue you should get the extension.
... I wanted to know what guarantees you can offer me.
... I have invested interest in making this successful.

AutomatedTester: We want an extension so we can get to recommendation.
... When, say we remove these two features, what are the potential next sticking points that might come up?
... Not knowing enough of the process, history has shown us we have gotten stuck.

plh: You will have to make a choice, do you address any incoming comments in level 1 or you do so in level 2, the master branch?
... My suggestion is you address them in v2.

AutomatedTester: We want to get patent protection.

plh: But if v2 is upcoming, that is what they will see.
... The bottom line is, in the worst case scenario, you have to go through the CR period again, and we should extend the group until the end of February.
... The best case, mid January.

AutomatedTester: Saying we do all these things, will the dates be moved back because of Christmas or Thanksgiving or things like that?

plh: The tool is smart.
... Publication moratorium is built in to the tool, so it pushes the different dates back.
... If I extend the group, the next question is going to be what is next after that?

AutomatedTester: We have a set of goals we want to hit.
... We have been slow in the past, but this is because we had no idea what we were doing.

plh: I would _love_ if you could go to rec every six months.
... Can we produce a CR, or can we go forward with the PR?
... That is what we need to figure out, and I will do that for you.

AutomatedTester: We need to finish this off, but we have general concensus that the WG will move on.

ato: Let us discuss the technical options.

<AutomatedTester> scribenick: automatedtester

scribenick automatedtester

jimevans: what does Geckodriver do when there is a prompt?

ato: at the moment it doesnt act. We have return an error when a prompt is hit

jimevans: I can move the IEDriver to match how GeckoDriver does things

jgraham: We need to update the spec to say return an error when a prompt has hit

RESOLUTION: We rip out web window, web frame et al. from the level-1 branch.

ato: for the frame/window returned from executescript we can remove since no one has implemented it

<ato> ACTION: jimevans to adjust IEDriver to match geckodriver’s behaviour.

<ato> ACTION: Adjust wording of user prompt handler to match behaviour of geckodriver so we can move to rec.

jgraham: it feels like we can have a patch by the end of today on this

RRSAgent: make minutes

<JohnJansen_> scribe: JohnJansen_

<AutomatedTester> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F#Thursday

<simonstewart> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F

simonstewart: break up the agenda easily
... changes to actions
... things not controversial

<scribe> ... new features

UNKNOWN_SPEAKER: let's do new things first then uncontroversial
... or easy first


clmartin: easy first!

simonstewart: ok
... lists agenda items

ato: new tab command

New Tab Command

ato: users of selenium are working around this currently
... geckodriver does not allow keyboard input to affect the os
... alternative "window.open()" does not actually open a new window
... so we need a primitive for enabling users to open a new TLBC

simonstewart: people want to do both

<scribe> ... (new window, new tab)

jgraham: agree there are some cases where you might prefer one or the other
... difficult to know how to talk about that
... say, a browser that does not have "tabs" or "windows"

simonstewart: suggest if you don't have tabs/windows ignore the command

ato: good idea for this
... what do you return for a browser that has neither?

simonstewart: if you are stuck with TLBC, return unsupported command

clmartin: and give them back a window handle id
... we have hacky workarounds

ato: volunteering to do the PR?

<scribe> ACTION: clmartin submit a pr for this

<ato> ACTION: clmartin to draft PR on New Window command

<ato> Scribe: ato

Delete History

AutomatedTester: This is from clmartin.

clmartin: we have a vendor specific command for this today.
... How can we clear out data for a fresh session?
... Because, with Edge, we can’t do that.
... We have ability to nuke the entire session.
... But that is too crude.

<JohnJansen_> ato: more things than history that you want to delete

<JohnJansen_> ... cookies is covered already

<JohnJansen_> ... but not all browsers support all these features that people might want to delete

<JohnJansen_> ... and allow browsers to add new things to the list of things

<AutomatedTester> scribe: JohnJansen_

clmartin: or maybe just have history

simonstewart: you have cookies, maybe then cookies should be in this general delete as well

ato: I don't think including cookies in this is the right approach
... even delete all cookies is just for domain

jgraham: localstorage as well

clmartin: starting point would be doing what happens when users click "Delete History"
... goal is to be in a clean state
... not mimic user clicking the button

ato: new browsers may not even have this command

jgraham: is the problem that Edge does not support clean sessions

clmartin: I've seen that users want to have a clean session at the start, plus also clear the session in the middle
... edge does not use a clean session, it uses current user

jgraham; new private window as well

<Zakim> plh, you wanted to provide a non-answer to the question of CR/PR

AutomatedTester: will you know more tomorrow?

plh: we may not need to go back to CR

simonstewart: how should we tell you

now back to history discussion

clmartin: even if we had multi session, this probably still has value

<ato> https://github.com/operasoftware/operaprestodriver/blob/master/protos/core.proto

ato: lines 13-28 are examples of different things we'd clear
... i think we need ot have an api that is vendor specific
... if we have specific endpoints for each type of data
... I'm a bit skeptical about that because we could add 15-20 commands

clmartin: yeah, it's better to have a command that can be browser specific

jgraham: usecase: just start a test with a clean state
... do people have specific use cases that are different?

jrossi: our UI does not allow granular deletion

clmartin: we would match what the UI does

ato: this is not really about acting like a user

jgraham: deleting history is something that a user can do
... maybe there are wpt cases for this
... delete history, cache in the middle of a test

clmartin: dev wanting to delete a storage mechanism

<jrossi> correction: Edge UI does allow granular deletion. Ignore me :-)

ato: if we are adding this so edge can delete data, then not a strong use case
... but if there is general interest from devs, then we should consider it

jgraham: vendor specific tests that are doing this via vendor api
... maybe google has layout tests that do this
... maybe this would help them
... could be part of the storage spec

clmartin: i've seen a few people ask, but that COULD be Edge specific
... definitely a use case for vendors, though

AutomatedTester: concern at the moment is: no one wants to restart the browser, it's expensive
... users might call this and expect it to nuke all, but some state comes through and breaks them without them knowing why

<scribe> ... new features, for example, that are in a browser but not added to webdriver

UNKNOWN_SPEAKER: restarting is a good clean start
... could create a bad state

ato: i feel like I'm lacking data on this
... I'd like to help solve Edge's problem here
... but need more input
... are there tests in chrome?
... are there tests at Moz where'd you want this
... testing a feature, it would be helpful to delete storage

brrian: these things would be great for WPT, but not necessarily for website testing

juangj: from a selenium perspective: people are using Edge and attempt to get a clean state
... very difficult to manage
... tell people to "give up, start a new VM"
... like brrian said, this is subtle
... maybe a product like selenium should not expose them

<ato> Scribe: ato

clmartin: I wouldn’t exactly create a defined list.

ato: There’s a simple solution: You add a getter returning a list of topics you could delete.

jgraham: Potentially this is a Selenium question.
... If we think it is required in WPT, we could expose it only through the protocol but not in Selenium.
... Option 2 is a case where we don’t add a WebDriver command for testing in WPT, but we invent some web API privileged API.
... That is a fine consensus if we don’t think we should expose this to developers due to the reason it could be a footgun.
... We shouldn’t reject a feature that is useful for testing browsers because we think it could be confusing to a subset of users.
... We should assume that it is a good idea to add new commands, but only if those are useful to developers.

clmartin: I’ve seen devs come ask for things I was surprised about.

jgraham: I don’t want to haveanother discussion about CDP, but perhaps the lesson from to be learned that people want low level access.

JohnJansen_: We have implemented this feature in a vendor specific endpoint.
... We have web devs who want it and people who will use it.
... I don’t see the harm is that someone uses it incorrectly.
... We not allow people who do understand it use it, and let those who don’t be surprised.

AutomatedTester: It’s a foot gun and this is what I want to prevent.

jrossi: At a granular level?

JohnJansen_: The only way to clear state in Edge is to nuke all cache.

jgraham: Are doing this at the start of a test?

JohnJansen_: No, people are also doing it in the middle of a test.

ato: Yes, and I guess that would be the exact use case one would have writing WPT tests.

brrian: Why not just open another window?

JohnJansen_: There are certain types of cache that exist globally.
... There is persistent user data as well, which is not web platform things.

AutomatedTester: I have never seen a request like this before.

jgraham: This is a request, to be fair.

AutomatedTester: In 10-15 years I have not gotten this request.

ato: There are multiple data points here.

jgraham: For all we know it is exposed in CDP.
... It is true that web apps are becoming more complicated.

brrian: I don’t understand why you could not rewrite the test to open a new session that is clean, and you log back into Sharepoint, and you do the whole thing again.

[jgraham explains a testing scenario]

AutomatedTester: OK, let’s file an issue/PR for this and continue the discussion there.

JohnJansen_: You should include concrete examples.

brrian: For validating utility I want to see concrete data points.
... Alternatively we can offer a clean session capability.

<scribe> ACTION: clmartin to Write issue/PR about deleting history &c.

File upload API (not based on send keys)

simonstewart: The way Selenium works, not WebDriver, is that we have a thing that looks at what you pass in to Element Send Keys, and if it is a file then it uploads that file to the Selenium server.
... Send Keys does not do a file upload in the spec.
... You want an upload file command moves the file to the server.
... Then one to upload it to the driver.
... For this existing WebDriver session, upload file.
... We want a generic command to upload an apk to the remote machine.
... I believe safaridriver has it.

brrian: Of?

AutomatedTester: No one has done it except the Java/Grid.
... It sends it to the specific machine.
... But none of the remote ends have historically had to know.

simonstewart: It is common behaviour for the intermediary nodes, so we can put in specification text that is specific to that node.

clmartin: My concern is that you don’t type into <input type=file>.

brrian: But you also don’t type a file path.

AutomatedTester: It’s because it’s an <input>.
... Each OS will open its own widget for <input type=file>.

simonstewart: I think I would like to have something in there that for intermediary nodes.

AutomatedTester: I’m not saying no.

brrian: safaridriver is the only thing that can put a file on that device.
... I don’t think we could run the Selenium server there.

ato: Good use case.

AutomatedTester: Yes.
... And we need to do more mobile friendly in the future.

simonstewart: You would like to have equivalent of <input type=file> Send Keys endpoint to upload a blob.

ato: I like the idea of the Element Send Keys blob.

simonstewart: The payload to Send Keys is {"text": "…"}.
... We could have {"file": […, …]}.
... I think this covers brrian use case.

brrian: Current semantics is that you make multiple calls to Element Send Keys to select multiple files.

<brrian> "titusfortner"

AutomatedTester: The question to Sauce was: a lot of this is going to be intermediary node heavy.
... What would you want?
... This will influence you quite heavily.

titusfortner: It’s not really any difference.

AutomatedTester: Then the other use case is to upload resource files, such as apks.

ato: Is that something people actually use?

titusfortner: Yes it is.

ato: Putting in a file transfer protocol in WebDriver sounds very scary.
... For Android I would use an adb debug bridge to move the file.
... I don’t know about iOS.

boazsender: Uploading a file to a file element sounds like something the driver could do.

juangj: If you just want the file to exist on the filesystem, it does not sound like WebDriver’s job.

<kereliuk> juangj

JohnJansen_: I don’t think we should be uploading arbitrary files to an endpoint.

AutomatedTester: People are not sure about intermediary node endpoint for uploading arbitrary files.
... If there was a /upload.

simonstewart: How do you get the apks to the machine?

titusfortner: I think we would prefer that to not be a concern of WeBDriver.

juangj: [agrees]

simonstewart: Maybe what we want is to have interop between browser stack and sauce.

jgraham: Cloud provide API spec is not really related to WebDriver.
... The browser driving protocol is our domain and I think this requires different expertise.

tboyles: If you gave us a standard, we would definitely use that standard.
... We have worked around it so it is not something that blocks us.
... We don’t need this today. We have solved this problem.

simonstewart: Array of file blob and file name.

ato: Just to be clear, we can’t change the current behaviour of Element Send Keys passing a file name to <input type=text>.

brrian: If you write your test in Python, it can encode it or it cannot.

JohnJansen_: Break?


<scribe> ACTION: simonstewart to write proposal for Element Send Keys to take new "file" field.

Copy/paste & text node selection

<clmartin> scribenick: clmartin

ato: it came up in the wpt testing session that webdriver should have a separate command for cutting/copying/pasting in the browser
... if we want to be more mobile friendly then we need to handle text selection more nicely

jgraham: implementing selecting text will be a disaster because it's complicated
... is there an existing platform api for selecting text so that the minimal change would be to copy the current selected text
... and a command for pasting it from the clipboard

ato: what you're suggesting is have some script that uses the platform apis

jgraham: yes

AutomatedTester: so like dom range?

jgraham: yes
... ideally we do this without creating an alternative to the DOM range API.

ato: Add a whole new range of serialization methods for different types of nodes.

jgraham: The point is you pass in I want the selection to cover this point in this node to this other point in this other node

titusfortner: Is it relevant that we have an atom for selecting text?

jgraham: Yes

titusfortner: We have an atom implementation in the ruby atoms

simonstewart: What does it look like?

titusfortner: Let me check

<titusfortner> https://github.com/SeleniumHQ/selenium/blob/master/javascript/watir-atoms/watir.js

ato: So the thing we want to make WebDriver do is take that selection (whatever selection is made in the browser) and put that on the clipboard.

simonstewart: I guess you want to post the keyboard and get the keyboard.

ato: It's not necessarily related to clipboard.

simonstewart: Clipboard*
... So posting would be "I want to modify the clipboard" and getting would be...

jgraham: That's the wrong model, you don't want to know the value of what's on the clipboard

simonstewart: Well you want to do copy/paste right?

jgraham: Yes exactly. So you should do two commands. One is to put something onto the clipboard, and the other is paste where we put it into something.

brrian: I've asked ryusuke to come over here to describe the ask.
... When you're copying you're copying everything whatever the OS does.

jimevans: Clipboard formats are not fun.
... We should not expose the clipboard formats

<inserted> gsnedders: there's also "paste & match style"

jgraham: Is what you're saying is that there are multiple kinds of paste.

ato: So how do you trigger those?

simonstewart: If you do apple-v paste you get one kind of paste, if you hold different keys you get other pasting formats

ato: But the use case is that they have an action sequence you can push the keys you want to validate

JohnJansen: When you paste into a spreadsheet you get options on what you want to paste.

jgraham: So it sounds like there is browser specific options when you go to paste

JohnJansen: Yes it is browser specific.
... We have a clipboard API for testing. Our browser inspects what is being pasted and tries to determine how you'd want to paste that.
... So you may want to paste a number vs. a string, or raw value vs html.

jgraham: Right, the difficult part is there might be multiple possible paste options.

JohnJansen: In Edge you get a table and the table changes size based on how complicated what you're pasting is.

jgraham: So the options are actually context specific.

JohnJansen: Yeah

gsnedders: It turns out browsers are complex

titusfortner: *laughs*

jgraham: I'm not sure if anybody implements the paste part of that

ato: It is entirely possible that we want to hook in at a deeper level.

jgraham: Yeah but that API is, clipboard access turns out to not be very secure

ato: brrian For that to be shippable, you'd have to null it out before you start the session.

brrian: If you want access to the real clipboard that's very dicey.
... and how do you abstract that in a way the browser works, idk

titusfortner: Is there a reason we can't just do cmd+c and cmd+v

jgraham: The problem John is describing is that the clipboard isn't as simple as you like.

gsnedders: Also which clipboard

titusfortner: I get what you're saying, if you want to emulate what a user would do then that wouldn't work.

jgraham: A vendor could do that if they want to.

tboyles: Would it make sense to do that?

ato: If something does ascii now and we release a new version that's differnet that's a problem

brrian: We're talking about clipboards

rniwa has joined the room

ato: We were talking about how different clipboards on different browser on different OS's are different

rniwa: correct

ato: John was talking about how copy/pasting depends on the context you're in.

JohnJansen: I was saying that we shouldn't try to implement this as it's more complex than simple copy/paste
... We also have security alerts when copying/pasting data
... Makes it hard to standardize

jgraham: but is it impossible?

JohnJansen: No it's just code but

jgraham: But we've agreed that browsers can be different.
... Edge could pick a default option and go with that

JohnJansen: Yeah we can do that

ato: I was curious about preventing business content from getting copied/pasting. Is that something that gets prevented at the os level?

rniwa: Content editors have the requirements that they can copy/paste stuff from Microsoft Word. For those you can't mimic the content that Word generates.
... With clipboards there are things that are very different between platforms. In Windows you have one impl. I don't know if Windows has a way to archive, store images.
... On a Mac this is different. Not only do we support plain HTML but we also have an archive format that can contain all the data for the file which means the mechanism by which things are serialized can be very differen.t
... So if you're using WebDriver to test copy/paste behavior it's important this works.
... If you're just copying within the page you almost don't need an API for this.

simonstewart: So you do need to modify the system clipboard?

rniwa: Right

ato: We brought this up because you brought this up on Tuesday (rniwa)

jgraham: I think things outside of the browser window are out of scope
... You have the use case you want to copy from Word, I think that's valid but standardizing that via WebDriver isn't going to work.

simonstewart: Setting the caret position might be helpful.

AutomatedTester: You can do that

ato: We can do the select via the DOM API's

simonstewart: What we see are users using the browser automation, then swap to a lower level automation, then swap back

ato: Right but we still need a way

simonstewart: So what you'd do is use it to copy then use a different tool to paste

ato: If I were to implement in geckodriver then I'd look at what copy does and I'd abstract that away, I'd call that endpoint

simonstewart: I agree with jgraham and ato but I'm saying this is one way people would do it
... Everything we do today allows you to test in parallel. The minute you start operating on the system clipboard you break that.

ato: I wouldn't expect geckodriver to work like that

jgraham: It's not about that

gsnedders: It's about global state

JohnJansen: I confirmed any web content, copied html content and tried to paste it to Microsoft.com and I got a warning
... If i paste the same text into a word doc in the browser it maintained the formatting
... It's very complex

rniwa: In general the two use cases are:
... 1 is copy paste in the wpt
... at the end of the day if we don't support any ability to test the system clipboard, then all authors that work on web properties that utilize the clipboard then they wouldn't be able to test it.

simonstewart: They will be via other tools

rniwa: I would prefer the approach that by default you use a synthetic clipboard, but provide some way to test the system clipboard. One tricky thing with the system clipboard is that there is an order of types. What the browser exposes and what is the default behavior would be different. Adding the ability to order these things in the system clipboard would be complicated.

jgraham: Do we think the use case of a synthetic clipboard makes any sense
... It sounds like the system clipboard is complicated but some allow you to create a per-app clipboard

rniwa: The system does

simonstewart: Does Windows?

JohnJansen: idk

gsnedders: Idk if Linux test environments do that

rniwa: For global no, but regular copy paste would

jgraham: If that exists then it would make things easier.
... If you want to interact with the system then you'd need a vendor extension. It's not worth spec'ing that.

ato: So I just checked on MDN and gecko has this concept of logical clipboards. A subset of those are mapped to the system clipboard.

JohnJansen: In the browser?

ato: Yes

jgraham: Not expose to content.

rniwa: It would be nice to trigger copy/paste without having to hard code the command (shortcut keys)

jgraham: Well that doesn't work.

rniwa: Right, so it would be good to...

ato: I agree it would be useful to add to WebDriver. I sense there's an action coming on that we need to do some more research.

JohnJansen: Yeah we need more research. I'm hesitant to access the clipboard right now.

rniwa: The system clipboard?

JohnJansen: I'm not sure.

gsnedders: If we do nothing then people will keep using ctrl+c

brrian: It doesn't work anymore

gsnedders: So it only works in some browsers

jgraham: From a security point of view if that does to work in MicrosoftWebDriver then you add a capability

AutomatedTester: Are we all agreeing that if we're going to do this we need to do more research?

ato: I will file an issue

<ato> clmartin: Thanks!

AutomatedTester: It's 3 minutes past 4, how's everyone doing?
... What can we handle?

simonstewart: The two additions to the actions API which don't feel terribly controversial

ato: Who added that?

simonstewart: I did

ato: If the element you want to reach is outside the viewport youc an't do that

simonstewart: Right what youw ant to do have is have it scroll into view so you can interact with it
... or even if not that the ability to scroll the view port

jgraham: For elements it can work like the existing click command so for the start you can find where it is and scroll that amount

simonstewart: The problem is imagine you're doing drag and drop

jgraham: That will work, before you do scroll into view, you ask where the element is on the page which has a set of coordinates and you say in order to have that in view you need to scroll by x and y and that's how much you scroll
... Because as you scroll you can also get scroll events

simonstewart: But we do want to be able to scroll?

jgraham: Yes

ato: What kind of primitive?

jgraham: A scroll primitive that works like point to move in the sense that you either provide it an offset relative to the top of the viewport or an element

simonstewart: So this is exactly like the origin astuff we have

jgraham: Like input type scroll

rniwa: One question, why does scroll to not work in this case?

simonstewart: The actions api is a sequence of events you fire off simultaneously so you don't pause in the middle of that

rniwa: I see

juangj: What I hear jgraham saying is that the primitive is we'll compute and do that and if it doesn't result in it coming into view that's okay

ato: Yeah

jgraham: That's how the existing stuff works so I would expect it to work like that

simonstewart: When we do a pointer move action we have an origin

<AutomatedTester> https://w3c.github.io/webdriver/webdriver-spec.html#dfn-dispatch-a-pointermove-action

jgraham: Right I'd expect it to be the same

ato: What kind of value or metric would we be passing that

simonstewart: We'd probably use the null input device

ato: Right but how much do I scroll

jgraham: It works like something, you give it an x/y or an element
... and it computes the amount for you

ato: So the x/y calling is fine, but in the case of an element you'd have to invoke the is pointer interactable

simonstewart: no

jgraham: no
... You just query the layout for where is this element and scroll to that position and if it moves during that then bad luck

simonstewart: I'm on board with that

ato: that works

jgraham: You have to choose where you want it to end up
... In the element case you copy the semantics of scroll into view as closely as possible

gsnedders: Which is what click already does

simonstewart: Yeah, we'd want element click to be implementable on top of the actions api

titusfortner: Does scroll into view move it into the center

ato: Because!!!!

jgraham: Okay, but...
... Lots of other words
... Do what the semantics are and stick to what is already being used for other thing

titusfortner: I just want to avoid static headers/footers if they exist because sometimes scroll into view will be hidden

AutomatedTester: If that happens we can't solve that

titusfortner: Well offset, if we have an offset then that's good

ato: How do you do that as an atomic action
... You can do it to an element but by an x/y there's nothing in the api that allows that

simonstewart: You treat it like a pause

ato: How do you move those few extra pixels

jgraham: Scroll into view does that

juangj: There's a way to do that

jgraham: regardless it's possible to scroll in the web browser by an amount

ato: But that's different. First you call the dom api for scroll into view

simonstewart: there's nothing conceptually difficult here right

jgraham: the DOM API may or may not allow thos eoptions, then you'd call the underlying API your scroll into view api is using

simonstewart: Yeah scroll into view with the element as the origin as 0,0
... If you called scroll into view with an origin that was relative you need an x,y

ato: But it doesn't support that

simonstewart: I need a new name, scroll action, with an x/y

jgraham: Just call it scroll

ato: We can scroll by line but not by pixel in gecko

AutomatedTester: You can scroll to a point, scroll to, and pass an x/y

ato: Yeah scroll by offset

jgraham: Any other issues?

ato: Just want to make sure we can do this

jgraham: Is it instant or smooth scroll?

simonstewart: It doesn't matter

jgraham: You might get different behavior
... You might want an option

rniwa: Hit testing will happen in browsers if you aren't doing an instant jump
... For drag/drop you'd get events

simonstewart: You'd allow scroll options to be passed in

jgraham: Especially if it's already int he scroll into view apis

juangj: scroll into view will give you that

ato: Scroll into view does, scroll by doesn't
... I think we can do that

An action to add this to the spec

simonstewart: The second suggestion is wait until interactable
... This is for suckerfish style menus
... When you move the mouse onto the element, hover and then wait

jgraham: How do you tell when that happened?

<AutomatedTester> https://w3c.github.io/webdriver/webdriver-spec.html#dfn-interactable

juangj: Give a CSS selector to an element you'd expect to appear and wait for it to appear

ato: It's technically possible I guess
... With a pointer source because we have that definitioin

simonstewart: You do the test with the pointer interactability
... It would be a pause of an undefined length

AutomatedTester: How long is that wait
... If that thing doesn't appear

gsnedders: A frame or two

jgraham: It might never appear

AutomatedTester: I don't care how we do it we need a way

simonstewart: 2 ways to do it, 1 specify the timeout and wait for it, 2 allow an upperbound on wait for interactability
... If you were to put it on the global timeout it would work that way

jgraham: It uses the implicit wait timeout

ato: No

simonstewart: No that's very different

jgraham: We have a note in the spec that says implicitly wait until it's interactable

ato: That's wrong

simonstewart: That's what it was originally

ato: It would wait for the element to be locatable

simonstewart: We can use the implicit wait timeout
... But ideally set it to 0

AutomatedTester: Always be 0

simonstewart: But with this it might take a while

jgraham: I'd strongly prefer you can specify a timeout for this action with the global state is the fallback

titusfortner: Interactability is not something you can query right

simonstewart: The spec has some clear definitions on what pointer interactable is

<ato> titusfortner: https://w3c.github.io/webdriver/webdriver-spec.html#dfn-pointer-interactable

rniwa: Does it make sense to wait for an event to be fired

AutomatedTester: What event?

<simonstewart> Interactable definition is here: https://w3c.github.io/webdriver/webdriver-spec.html#dfn-interactable

rniwa: You name an event and maybe while doing the test you need to do a query that takes an arbitrary amount of time and when you're done you fire

<simonstewart> Pointer interactable: https://w3c.github.io/webdriver/webdriver-spec.html#dfn-pointer-interactable

jgraham: WebDriver isn't built to handle that easily but you could do that via execute script
... In the middle of an action sequence I don't know what the use case is on waiting for an event

rniwa: Something about drag and drop

simonstewart: Wrong level, WebDriver is at the browser level

jgraham: I don't understand it but I don't think it should be dismissed

simonstewart: I'm not saying it's not in scope, I'm saying the mental model for how webdriver works doesn.t...

jgraham: What would async drag and drop look like

rniwa: Drag and drop we don't do this, but let's supposed we did. In that case maybe in a drag start you'd add something, then the browser would make a callback that I've added it, but

jgraham: What something?

rniwa: Like actual data to drag. In that case you'd want to wait for that thing to be done before it drops

jgraham: How would that work if a user drags/drops?

rniwa: There's an idea of a promise
... We might not have it in a browser but it's conceivable

simonstewart: So why wouldn't WebDriver use that, it's supposed to be a user
... Drag and drop should work and if it doesn't there's a bug

jgraham: If you drop something that needs to wait on data it should get a promise back

rniwa: This isn't a good example

simonstewart: I can see wanting to wait for a mutation event

jgraham: interesting!

ato: Yes

simonstewart: This is another primitive
... Which we may have consensus around

titusfortner: We can't wait for interactable outside of specific commands

jgraham: The theory is you can do this via the actions API

titusfortner: I'm thinking of the concept of a string of stuff but in actions

ato: Do we think that it's a useful primitive

simonstewart: Yes

jimevans: I think it's amazingly useful but we don't have it in place

ato: I wrote this thing, there are dragons

simonstewart: Add a definition for pointer interactable

AutomatedTester: Clay got lost

ato: The spec says you can set the timeout to 0 and wait indefinitely

simonstewart: That's fine

juangj: You'd expect most callers to set an explicit timeout

ato: With an unlimited wait they'll hit the HTTP timeout

simonstewart: Right

ato: I was thinking it was unrecoverable
... But it isn't

juangj: What we're saying is if you don't set a timeout we hit the implicit wait timeout why not jus tmake the timeout necessary

simonstewart: There's a different between should and must
... The spec should just default to the implicit

juangj: Why can't the spec require it

simonstewart: Nowhere else do we require it

ato: What do you mean by the upperbound?

simonstewart: A timeframe that youc an wait

ato: If it can be indefinite why would you have an upper bound

AutomatedTester: Where is it indefinite?

simonstewart: If you don't set an implicit wait timeout...

AutomatedTester: If implicit wait isn't there, set it to 0

ato: I was talking about script timeout

juangj: If you do this implicit wait timeout falls to 0 and the default timeout on wait to interactable is 0 but a 0 timeout isn't userful

simonstewart: No but there's a mechanism to override that

AutomatedTester: And client bindings can set that for them

simonstewart: Not everyone sets it to 0, if you see it set to 30 seconds hteen

juangj: The purpose of this primitive is that running it once isn't useful

simonstewart: Click in actions just does it
... We have a new action, wait until pointer interactable, and that waits for a set amount of time for an element to become pointer interactable, by default that time is the implicit wait timeout, but you can set an upperbound independently.

ato: Can we get the resolution on whether the is pointer interactable primitive is...
... I've tried implementing keyboard interactable and haven't been able to

simonstewart: How come?

ato: It's hard
... Had to dig through a lot of specs
... So the question is, wether you want the command for whether it's interactable in a keyboard and click, or independent

titusfortner: I don't need what I was asking for

ato: The intention of "is interactable" is can I, using the keyboard in any way interact with an element

titusfortner: Well yeah the idea was clicking on it

ato: I could see a use case for someone

simonstewart: I can see a wait command

ato: If you're working on google ads I could see this being useful

titusfortner: Sometimes I have users that would like to automatically retry when it fails, others don't
... When I was coming up with a solution it was tough
... If we had a wait for interactable that would be good

simonstewart: Would that be pointer && keyboard or pointer || keyboard

AutomatedTester: An object would be much better

ato: It's not bad but it's more complicated than it ought to be
... Could see two separate commands for pointer/keyboard

simonstewart: But then webvr

AutomatedTester: No but people throw it into a lambda

jgraham: I don't understand the use case for knowing is this thing... presumable the next thing you're going to do is interact with it with the keyboard or something else. but not at runtime

simonstewart: Yes you're right, I think AutomatedTester is quite elegant
... Because it allows for other variations

jgraham: I think that's a good reason not to do it like that
... I don't want this thing to have to fire up the VR subsystem to check if it was VR click interactable
... When you haven't been doing anything with VR up to that point

AutomatedTester: So that one is easily solved by checking "is subsystem x enabled" no
... But if you're in a VR mode and ask if it's interactable it would be possible to do that
... And you can gate it on the subsystem being running

jgraham: That's a lot of work for the browser

simonstewart: It is a separate endpoint, it's doing a full HTTP roundrip just to say yes/no

jgraham: I feel like if this was a thing you think it would be extended, i feel like telling people "oh yeah just put it in there" seems like it doesn't scale
... I don't want to say "doesn't scale" but "doesn't scale"

simonstewart: This is where David's point comes in, if you aren't using a subsystem then don't run those checks

jgraham: Then it pushes complexity onto extension spec authors to properly define that
... A specific endpoint solves the use case

simonstewart: You're suggestion is a separate endpoint for each type?

jgraham: Yeah

juangj: You know what you want to do

ato: It's also more in style with the other web element specific methods

simonstewart: Is displayed, enabled
... attribute as well

jgraham: There aren't other cases like that

simonstewart: Which way would people prefer?
... Should we take a vote?
... Is there a problem with multiple endpoints?

No one disagrees

simonstewart: Do we add the endpoints at all?

ato: I think if we're exposing them through the Actions API, I guess it would make sense to also expose them as endpoints. No one has asked us though.

titusfortner: I have a use case. A transition is happening, try to click it, it's not interactable, it throws an exception. In Watir we abstract that at a high enough level that if we try to do that it's complicated. If we let the user specify it works better

<AutomatedTester> RRSAgent: make minutes

ato: Do we think it could be confusing to test this in a way that's interactable?
... I could see someone going "I want to click this button but it's disabled"
... But the is interactable will return true

simonstewart: Oh that's fine

ato: But do people understand that difference?

simonstewart: I see what you're saying

titusfortner: But it's in the spec what it means


simonstewart: Imagine you're not a good developer

titusfortner: Such a stretch

simonstewart: and you what developers always do and they don't read the docs, and you have the option between interactable and not
... If you named the method ...

titusfortner: You can click a disable elemenet


ato: I don't care what you expose it as
... I like juangj idea doing it in a restful way
... You could do AutomatedTester's thing as well

jgraham: I vote against clever

juangj: Coming back to the question whether it satisfies your use case, waiting for it to be interactable, that's not what this would do. It would just return if it's interactable.

titusfortner: That's fine

ato: I'm in favor of adding this

RESOLUTION: add interactable check command

<AutomatedTester> RRSAgent: make minutes

Summary of Action Items

[NEW] ACTION: Adjust wording of user prompt handler to match behaviour of geckodriver so we can move to rec.
[NEW] ACTION: ato fix the urls for the spec
[NEW] ACTION: clmartin submit a pr for this
[NEW] ACTION: clmartin to draft PR on New Window command
[NEW] ACTION: clmartin to Write issue/PR about deleting history &c.
[NEW] ACTION: jimevans to adjust IEDriver to match geckodriver’s behaviour.
[NEW] ACTION: MikeSmith to follow up on renewing the charter
[NEW] ACTION: plh to check with legal if we can move to PR.
[NEW] ACTION: simonstewart to clean up documentation on how to hook into the spec.
[NEW] ACTION: simonstewart to write proposal for Element Send Keys to take new "file" field.

Summary of Resolutions

  1. CDP is going into an incubator group/CG. There is sufficient support from vendors and independent parties.
  2. Agreement to extend the charter until the specification goes to rec
  3. We rip out web window, web frame et al. from the level-1 branch.
  4. add interactable check command
[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.152 (CVS log)
$Date: 2017/11/10 00:46:34 $

Scribe.perl diagnostic output

[Delete this section before finalizing the minutes.]
This is scribe.perl Revision: 1.152  of Date: 2017/02/06 11:04:15  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/re./rec/
Succeeded: s/a/the/
Succeeded: s/thigns/things/
Succeeded: s/stuff/agenda items/
Succeeded: s/illegal argument/unsupported command/
Succeeded: s/shoudl/should/
Succeeded: s/users/users/
Succeeded: s/users/user/
Succeeded: s/start/part/
Succeeded: s/boazsender/juangj/
Succeeded: s/keyboard/clipboard/
Succeeded: i/Is what you're saying is that/gsnedders: there's also "paste & match style"
Succeeded: s/doesn't scale but doesn't scale/"doesn't scale" but "doesn't scale"/
Present: jgraham ato jimevans clmartin soareschen AutomatedTester alrra simonstewart JohnJansen juangj MikeSmith rbyers auchenberg
Found ScribeNick: clmartin
Found ScribeNick: boazsender
Found ScribeNick: clmartin
Found Scribe: ato
Inferring ScribeNick: ato
Found ScribeNick: automatedtester
Found Scribe: JohnJansen_
Inferring ScribeNick: JohnJansen_
Found Scribe: ato
Inferring ScribeNick: ato
Found Scribe: JohnJansen_
Inferring ScribeNick: JohnJansen_
Found Scribe: ato
Inferring ScribeNick: ato
Found ScribeNick: clmartin
Scribes: ato, JohnJansen_
ScribeNicks: clmartin, boazsender, ato, automatedtester, JohnJansen_
Agenda: https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: adjust ato clmartin handler jimevans mikesmith of plh prompt simonstewart user wording

WARNING: Possible internal error: join/leave lines remaining: 
        <scribe> rniwa has joined the room

WARNING: Possible internal error: join/leave lines remaining: 
        <scribe> rniwa has joined the room

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)

[End of scribe.perl diagnostic output]