<jgraham> RRSAgent: make minutes
<clmartin> scribenick: clmartin
<ato> brrian: We are starting the meeting momentarily.
<JohnJansen> Bbrian is in the car. I’m going to call him when we start.
<ato> OK
<ato> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
<MikeSmith> agenda: https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
ato: need to discuss what we have to complete first/prioritization of the agenda
AutomatedTester: doing
introductions
... Moving on to the next item in the list, we have a state of
the union so we can describe where we are
simonstewart: We are currently at
candidate rec. in order to transition from cr to pr (proposed
rec) we need the w3c to give us the thumbs up.
... to do that we need proof of interoperable implementations
of the standard. we have w3c versions of the protocol via
iedriver and geckodriver
jgraham: do we have a url?
simonstewart: we do and david
will get you those. we need to come up with a convincing
argument for why this is good enough or we need more
demonstrations of interoperable implementations
... that's where we are
... do you have anything else to add david?
AutomatedTester: I don't think
so. we have other things we want to move forward on but we want
a better understanding of the process before we do that which
is why there are a large number of pr's on the GitHub
repo
... they're on a separate branch
jgraham: for example we're waiting on new features that can be merged however they all need tests
<simonstewart> Test results can be found here: https://w3c.github.io/webdriver/results/2017-10/html/all.html
ato: to expand on what jgraham
just said, we now have a policy that any change to the spec
needs to be coupled with tests going into the web platform
tests which seems reasonable as we now have our test suite up
and running
... I wrote an email to the mailing list because as an
implementor i'm not concerned about it moving to rec so much as
having a living document that reflects the current reality.
there are pr's in the queue that actively prevent marionette
from implementing and it's only going to get worse.
automatedtester branched off so we can merge things from the
queue into that branch
rbyers: i assume the test match the master branch?
ato: that's correct
rbyers: what are firefox's results?
simonstewart: I've posted a link to that
AutomatedTester: that's where we are currently. the key part is moving forward we need to see where everyone's implementations are
<rbyers> There's also live results here: https://wpt.fyi/webdriver
AutomatedTester: we can see
between geckodriver and iedriver what's passing/failing. feel
like it's doing a bit of disservice as we don't have something
for chrome
... or for edge or for safari, we should expand on that
<AutomatedTester> https://w3c.github.io/webdriver/results/2017-10/html/all.html
JohnJansen: mind pasting the link for the results?
simonstewart: that's really easy to generate btw
jgraham: if you want we can project the results
AutomatedTester: i'll start with
geckodriver implementation. for the most part we are doing
quite well. we've been from the wd spec tests passing a large
majority of it. there are a few parts where some things have
changed and we need to fix those up but in general terms it
looks like if we kind of implement a major feature of the
specification our implementation results goes up quite
significantly which is around alert handling
... which we don't have support for yet. it is a priority and
we are going to be working on it sometime soon. i'm not sure
how much detail people would want to go into from geckodriver's
point of view but feel free to ask any questions on what you
feel might be missing. i do think in general terms that the
test suite needs to be expanded where possible because we need
more coverage/better coverage
jgraham: there's two things, one is do we think that the testing could be better? yes. do we think that that should be a prerequisite?
AutomatedTester: no i don't think we should gate necessarily on this being a full test suite
ato: can i expand on that. as
AutomatedTester said we are passing a majority as we wrote
them, but i do suspect as more vendors right more tests will
run into problems. over the last year we have made some strides
on the spec, navigation, page load strategies, pointer
interactability with help from google for edge cases
... in addition to that in the past year we've done lots of
refactoring on how marionette does window tracking
... I've seen the light and think we'll make progress into the
next year
<ato> Ah, I forgot to talk about wpt.fyi.
rbyers: let me do a quick framing from google. big picture - we aren't investing enough. this group knows sam, he's no longer at google. john started recently on our interop team but that focuses on our interop tests. i think we should be investing more but i want to understand the arguments/where everything else is to sell back to the larger org. with that john has been working on webdriver
kereliuk: one of the things that
took a while is actions
... i have to go through all the commands in our current
implementation and verify that they conform to the spec. a lot
of the code in chromedriver is written by people i have to
track down. aside from that the next steps from me would be
(once that is merged/good) is making sure i can get a good
sense on the interop suite. ran into some issues, there's a lot
of overhead to that like set window rec
... very soon we should be able to have a nice interoperable
implementation in the next 2-3 months
simonstewart: one of the things
we've tried to do with the spec is make the delta between the
json wire protocol/w3c as small as possible. new
session/actions are where we diverted. a successful result for
instance look the same.
... hopefully when kereliuk says you need to go through and
review that it's mostly painless.
kereliuk: I've done a solid 50% of that and most of what I've encountered is around error codes/error messages
jimevans: we don't properly check for the tlbc being closed
kereliuk: setting timeouts is different
simonstewart: yeah
kereliuk: should be some fixes in
the next release for the ruby bindings
... if anyone has any specific questions please ask
... i'm the one who works on chromedriver right now
simonstewart: would you like if i flipped the selenium project to run the w3c tests against chromedriver?
AutomatedTester: how would one do that for chromedriver?
ato: i don't think a selenium discussion is relvenat
jgraham: a lot of the fixes won't be there as we're using the latest release chromedriver and not the latest master build
kereliuk: i want to have nightly release for a project like wpt. they are available now but you have to login with a google account. we did this so we could fix wpt quicker.
ato: sounds like there is a little plumbing we need to do
jgraham: for wpt.fyi in particular as some people are interested in the conformance report more than that, there's a url we can get at that has a link to the nightly and that is kept up to date and kereliuk could consider providing the same
rbyers: we should also be clear
that at the moment our priority is not to pass all the wpt (web
platform tests) but ensuring we can automate wpt with
webdriver, which is why we did pointer events first.
... tell me if you think that's the wrong priority
alex: not on irc, i represent
CoSmo, we develop a tool called kite to test for webrtc. we
want to test with webdriver
... don't know you guys yet so come meet me at the break so we
can work together on bugs when we find them for webrtc
testing
ato: are we done with the chromedriver update?
<ato> https://bugzilla.mozilla.org/showdependencytree.cgi?id=721859&hide_resolved=1
JohnJansen: for edge we have an interesting case for wpt.fyi in that our webdriver has to match the branch/build we're testing against. we have fairly current availability on browserstack but there are still some hiccups but i can submit them locally and get the results. i have a guy debugging it right now back in redmond. i think the tests do a check for the session to see if it's a w3c session and if it's not it'll stop. we'll pass 80% right
now but we're not w3c compliant.
JohnJansen: we're at about 80% but i don't know if that accounts for the current failures
jgraham: for what it's worth previously for chrome i was happy to add a hack to wptrun which is what wpt.fyi is using in the tests to say you can start an old style session so you can have the tests run. in the end chromedriver fixed it but i'm personally not opposed a failure in new session fails the new session tests not stopping the rest from running
ato: i think for slightly different reasons that making everything run on wpt.fyi is the right priority for the working group
jgraham: i think having it run there is relevant for moving the specification forward but it's just good in general
rbyers: when you change code you
want to see how that impacts results.
... can't see how you would move forward without those
results.
ato: not disagreeing but we have internal tests we pass that fail in wpt.fyi because of issues in wpt.fyi
JohnJansen: wpt.fyi is new so we can't rely on it yet for results
jgraham: i don't think the people
who are working on making the drivers better are not the same
people working on bug fixing wpt.fyi. it makes sense that we
both have people work on the driver to make the conformance
better and also working on wpt.fyi
... i mean the shared testing stack for wptrunning, the
fixtures, the build port, etc.
AutomatedTester: we can finish discussing wpt.fyi in a minute, want to get through impl status
JohnJansen: we need to
prioritize. we've been looking at internally running the tests
that exist on the w3c GitHub wpt repo as well as our internal
so we make sure we don't break things as we move forward. i
have no worry about the high level interop, it's in
production/being used by lots of teams across msft.
... not concerned with rec moving forward/being able to be
interoperable it's just a matter of prioritization/time
AutomatedTester: any more question for john? no.
brrian: i'm here now
JohnJansen: we were doing status, what is apple's impl status?
brrian: not there yet but it's started. we'll be using the same engine hooks that the gtk driver uses.
ato: we should get an update from jim
<simonstewart> GTK WebDriver: https://blogs.igalia.com/carlosgc/2017/09/09/webdriver-support-in-webkitgtk-2-18/
jimevans: one of the biggest
challenges was that for a while the wpt tests used a js
construct that ie didn't support
... got some stuff he needs to fix and get merged back in
... other than that it's a matter of allocating the time to fix
up the bugs that are the failures in the test
... i'm committed to getting it done. if there is a failure in
the tests right now then it's a bug.
JohnJansen: ... do you know any that can be fixed? are you worried about risky bugs/
jimevans: you mean bugs from the impl standpoint?
JohnJansen: yeah
jimevans: no, only a couple that are real challenges such as fullscreen
ato: i think that should be fine as the fullscreen endpoint is one of the few commands we allow an error if it's not supported
jimevans: right, i don't think i
have anything blocking my impl. it's just a matter of fixing
bugs.
... nothing i know of that would cause regression behavior that
i know about
rbyers: can we also hear from the
people using webdriver
... things like how well it's meeting needs in terms of
functionality
... i'm trying to understand separate from moving the spec to
rec how well we're doing
ato: it's a good question but the only impls we have right now are geckodriver/iedriver. most of the bugs we've had in the last 12 months have been due to selenium problems such as something hasn't been wired up in geckodriver or selenium
simonstewart: i think one of the things that's really important here is from a users point of view they're using selenium, they don't care about the protocol. they don't care if it's json wire of w3c, it all just works. the actions api are a significant step forward.
ato: the actions api is what we inherited from before in that selenium hasn't exposed everything
simonstewart: the java ones do now
ato: excellent
simonstewart: you can do new actions.tick and you can specify anything you want and it will run at the same time.
rbyers: that's not supported in any browser yet right?
simonstewart: geckodriver/iedriver support it and soon chrome
AutomatedTester: the new design is awesome because we can do new things.
jgraham: should we hear from the people using it now, we've gotten derailed
juangj: it's not perfect yet. it may be that complex gestures work with the new api but until it works in every browser people aren't interested in using it. from surveying existing actions users no one is making an action change longer than about 3 things as it's complex/risky. moving forward that should improve and it should meet peoples needs. rbyers mentioned people wanted pinch gestures and the like. optimistic about it meeting peoples
demands but that's still in the future.
juangj: there isn't a good way in expressing that in the old apis so until the new things are ready they're hesitant to move off of the old api
JohnJansen: from sharepoint they aren't ready to move off of it/emails i have from sharepoint guys they're happy with how it works right now and aren't keen on putting risk into the system at the moment until later.
rbyers: i feel like we're
de-railed on actions
... how is adoption going for webdriver across google juangj
?
juangj: mobile is the biggest
complaint we get, with mobile it's touch actions. as far as
other things it's permission apis/geolocation
... similar kinds of things where there is a user prompts
outside of the DOM/in the browser chrome. although those aren't
really things you need to cover in the test.
jgraham: you can disable that in chrome yeah?
juangj: yeah
simonstewart: fixing that comes under the door hangers
juangj: that particular feature you wouldn't expect webdriver to addres
ato: it's on the agenda
<boazsender> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
<ato> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
JohnJansen: can someone paste the agenda
<AutomatedTester> RRSAgent: make minutes
juangj: when you say are people successfully using it? yes but they're coping with the features that are available. if there are grand features that people wish existed/have dreams they can do something they have given up as this is what we have/what we can do now
rbyers: what about interop, do they have issues with cross browser
juangj: from a webdriver perspective no, for interops sake most people expect that they will work across browsers and they mostly do except that they use the legacy firefox driver. among people who have migrated interop has been good.
simonstewart: how many selenium tests are still being run. when i left google every team had a project being run.
juangj: there are no tests using
selenium, they use the api
... there are several thousand webdriver tests
JohnJansen: internally the thing that i hear about the most is not a bug in webdriver. we've joked about having a conversation in visibility. the edge browser itself reports things as being obfuscated/not seen on the screen. displayedness is tough.
ato: like i said earlier to do with pointer interaction, we had google run geckodriver with a special capability that has the new pointer work and we have found more problems but those aren't problems in geckodriver but problems in the spec/impl of other things
rbyers: how often does juangj take new chromedriver release
juangj: they do cause problems, we don't take new chromedriver drops as quickly as kereliuk updates as they cause problems. if there's a bug fixed then someone inevitably was accidently relying on that bug.
jgraham: do you still have people using firefox 47/old firefox driver?
juangj: yes about a quarter of
them are still using the firefox driver
... some people were waiting of geckodriver to implement.
mostly it's people procrastinating. there are a lot of tests
relying on pointer interact ability not being implemented.
AutomatedTester: moving on to sauce labs
titusfortner: i think most of the
interop things i see are timing issues between how the DOM is
rendered. i don't anything specific to the protocol that causes
issues. most users swapped from firefox to chrome.
... maybe that will move back with w3c, i'm not sure
juangj: i see people moving from firefox to chrome as they wanted to avoid the change.
titusfortner: a lot of people defaulted to firefox when running locally because it didn't require an extra download.
simonstewart: i think people find the upgrade process from legacy firefox/selenium to be tough. they change everything at once and they get burnt and blame geckodriver unfairly.
juangj: it's a bit daunting, there are some cases where you can't change only one thing at a time.
AutomatedTester: are there any
other questions for sauce labs
... mike since you're back, we have an implementation
updates/how people are working. people have offered to create
impl reports. what are the things required of this group to
move forward? what will be expected? how do we make everyones
life simpler
<ato> RRSAgent: make minutes
MikeSmith: we need to evaluate
our exit criteria and see how we are doing
... can we look at the exact language in that
rbyers: it's similar to other groups. every test must be passing in at least 2 places.
AutomatedTester: so there are a number of things that are failing at the moment or we need to fix our failures.
jgraham: if you look at the
current impl report there are two classes of things you see.
one is things that are unimplemented, you're not going to
convince anyone that something without two implementations can
go to rec. if the goal is to get to rec in the shortest time
possible we should drop the things that aren't
implemented.
... for the other things we have implementations but we have a
strange situation such as some of the tests failing in iedriver
not because of the tests but because iedriver is buggy.
<MikeSmith> https://w3c.github.io/webdriver/results/2017-10/html/all.html
jrossi: i agree, one thing we've done in other working groups in the second phase is the opportunity to provide comments. for pointer events we didn't have two tests passing because it was a browser bug but if that's the case then plh was okay with that.
<MikeSmith> https://w3c.github.io/webdriver/results/2017-10/html/less-than-2.html
AutomatedTester: i don't feel like we need to remove things to ship.
jgraham: it depends on what you want to happen. if you want to reach rec before the time it would take to implement those features. it's quite meaningless to do. you move them from the rec to the living doc.
<MikeSmith> https://w3c.github.io/webdriver/results/2017-10/html/complete-fails.html
jgraham: i agree aesthetically it's not pleasing but the reality is that we'll be gated on getting to rec on having those things impl'd.
rbyers: we did that for chrome, moved things that didn't work out of spec into incubation
simonstewart: seems like it's mainly around alert handling
jgraham: there are issues with the report
simonstewart: we need to update our drop from wpt
ato: about user prompts it wouldn't be extremely hard to add to firefox but when it comes to returning web windows/frames the serialization of that when returning from execute script, we won't be able to do that in a reasonable time
simonstewart: sounds like we
should yank execute script window handles from level 1
... and leave alerts there
jgraham: if we do that we need to get commitments
simonstewart: agreed we need to chat with people setting priorities
ato: don't care either way as we have something that works but the safe call seems to be yanking from level 1 and put it in master
rbyers: you should always yank it and if someone implements bring it back
simonstewart: the user
expectation around returning window id's from script isn't
there.
... prompts however are
titusfortner: different drivers implements that differently
simonstewart: they all do it
titusfortner: chrome doesn't
ato: if that is true than that puts us in a slightly different position
titusfortner: we have bugs around this
jrossi: one of the things we did in pointer events was amending the test results table to add in link to everybody's issue tracker to see what's failing
ato: you mean for making the report more valuable to plh?
jrossi: more in a way to track where the pain points are but it would also be helpful for reporting to plh
MikeSmith: ideally as far as test
results go we don't want to see any red. right now
superficially for anyone who doesn't know anything about
webdriver won't be confident about those results as there is a
ton of red there. you can explain individually why it's there
or. there are two things we can do is either yank the things
that aren't there and re-generate the results and go back to
cr
... if we make substantive changes to the spec we have to go
back to cr
... we could make the argument that removing things shouldn't
require going back to cr
... we need to wait 4 weeks after publishing a draft for
comments to come in
... when we remove features from the spec it invalidates
previous reviews. when someone approves the cr based on old
version they need a heads up on us removing features.
... in practice people may not care but it's more time
... we're out of time already, so we'll need to take more time
anyways, it's just a matter of what we need to do
... we can remove those features and restart the cr with the
new test results which may mean we get through cr fairly
quickly and move on
simonstewart: we should have a convo around priority setting and find out
AutomatedTester: we remove it from the spec and move it into master, it seems silly to remove the tests and then regen the report.
MikeSmith: we can keep the tests but to generate the impl report we'll need to generate from a subset of the tests
jgraham: right that's trivial, keep a set of tests that we know apply to level 1 and filter the rest out
MikeSmith: right we just need to filter them out. if we have a few cases of red that's okay but we need the report overall to look good/green.
ato: for example ie11 has a bug that's okay?
MikeSmith: right we can have that ready to go but normally this is explained over the phone and as long as we're confident in that explanation we should be okay
jgraham: sounds like we should get the latest chromedriver into the impl report as it might add passing results for tests
kereliuk: at least publicly you can go to the chromedriver website and click a link
jgraham: someone should generate those results
rbyers: is it different from what we have on wpt.fyi
kereliuk: trying to think of what was released previously
jgraham: wpt.fyi downloads it for each run
<kereliuk> Latest stable release: https://sites.google.com/a/chromium.org/chromedriver/downloads
MikeSmith: okay so as far as discussion with plh goes he said he can be available after lunch. he can be available at 1
<kereliuk> Latest master:
MikeSmith: he has something at 3 so if not 1 then some time before 3
<kereliuk> https://sites.google.com/a/chromium.org/chromedriver/chromedriver-canary
MikeSmith: it's 10:30 so we should break?
jgraham: why are we still here?
AutomatedTester: we'll be back at 11
<AutomatedTester> RRSAgent: make minutes
<juangj> amending earlier statement that “there are no tests using selenium” — this is not accurate as I misinterpreted the question. The vast majority of browser tests that we run use WebDriver, and virtually all of those tests use Selenium. There may be a handful of people attempting to use the WebDriver API directly but they’re negligible.
<juangj> for whatever that's worth.
we're resuming
AutomatedTester: moving on, the next steps are to fix up the agenda and items that we want to discuss
<boazsender> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
AutomatedTester: there's a whole
bunch of things at the bottom that people would like to discuss
at some point today
... We're going through the topic bank and move them up to what
we think are important and flesh those out
clmartin: can we move new tab/delete history up, they're not controversial
simonstewart: in an equally uncontroversial world, file upload
ato: just to make it clear to
everyone, we're now talking about features that will make it
into webdriver in level 2
... this will be stuff that if potentially agreed to will go
into the master branch
JohnJansen: for clarification, the full path to the branch that has v1 is that a branch or a url?
jgraham: what's the question?
JohnJansen: i want to make sure i'm looking at the v1 version of the spec or the v2 version of the spec
<simonstewart> https://github.com/w3c/webdriver/tree/level-1
clmartin: master is latest?
simonstewart: master is the living spec
jrossi: do you want to add to the v1 branch, change the title to say level 1?
ato: i have an action item to fix the url
<jgraham> ACTION: ato fix the urls for the spec
AutomatedTester: computer problems :(
simonstewart: are you building Mozilla?
AutomatedTester: i might be
:)
... stopping from the top, copy/paste/selection
rbyers: are we still talking about the agenda? do we want to summarize what we're going to do with devtools
simonstewart: can we break these
into things that are amendments, brand new features and
amendments to existing features
... as right now we're randomly cherry picking
jgraham: if rbyers is here for the moment to talk about the debug protocol we should do that
rbyers: we're talking WebVR at some point, would we want this group to discuss the bigger picture of test adoption requirements?
jgraham: let's prioritize that first then
AutomatedTester: we'll start with
re-chartering and proposed debug protocol stuff
... as part of when we started the work to move to pr, i
created the draft for re-chartering this group for this
work
... one of the things i have suggested which used to be part of
this group was how we do debug protocols
<ato> I’ve updated the agenda: https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F#Thursday
AutomatedTester: wilhelm originally added it for the opera debug work and now I've added it for the cdp
AutomatedTester: last night there
was an informal chat on if we were to do this how it would
look. the consensus was to remove it from the working group
charter and move it to an incubator group and build it out that
way
... because while people agree that working towards a standard
for the debug protocol is what we want, the process around what
we want to do in a meaningful way isn't there yet
... which is what's expected when trying to do things through
consensus
... from that point of view we'll be setting up an incubator
group, working with rbyers and others to set this up
MikeSmith: by that you mean a community group?
AutomatedTester: yeah
rbyers: key thing i wanted to see is, we create community groups when we want to collaborate, and the goal is to have a place where we can discuss/share docs/etc.
auchenberg: should we go ahead and make a suggestion here that we create an incubator group and each vendor can say whether they support that?
ato: yes and mike will talk about this later but that the debug protocol won't be a part of the re-charter
<Zakim> MikeSmith, you wanted to speak about rechartering
rbyers: we see good engineering
as being more important as it needs to come first before the
process/legal stuff
... the DevTools team in particular wants us to focus on
code
jgraham: there's a few things there, one is the debug protocol stuff which is unrelated to the existing webdriver work which there is agreement on that happening separately. and then the other is that mike is saying we could also move all of webdriver into a community group and not have a working group until later when ready to publish. i'm not clear on what the tradeoffs are there on whether we can still have meetings
MikeSmith: we can still have space for those at TPAC
jgraham: my understanding is that the space is limited for that
MikeSmith: no it should be viable. jrossi has a question
jrossi: we'll need to re-charter
MikeSmith: no we just need to extend
jrossi: i view the webdriver work
as very different from cdp. cdp is very nascent, webdriver is
fairly mature from an adopted perspective.
... our perspective would be continue webdriver in a working
group but the DevTools stuff on the other hand is a perfect
example of what should be incubated
AutomatedTester: webdriver has goals to move forward with
jgraham: now that there exists a
spec and it's clear what the requirement is for the editing
process, the group has now learned how to write a spec/tests,
we can potentially focus on a few things that everyone has
agreed to implement in the short term and publish a new rec
more often
... if we believe we can move towards that model then it
doesn't make sense to shut down the working group and then spin
it up again. it just makes sense to publish stuff more often.
personally i'd like to see it go in that direction
AutomatedTester: i'd like to see
it go in that direction for webdriver
... ato
ato: what is the cost from Microsoft to move from a working group to a community group?
jrossi: any group we join
requires a legal process, it's totally doable, but webdriver's
scope/deliverables are fairly straightforward/rubber
stamped
... if there was a good consensus we could make that happen
rbyers: if we do incubation in the wicg there's no additional cost to the user?
jrossi: there is
ato: i think speaking for myself that over these 6 years there has been overhead in having a wg (working group) for webdriver and the value isn't clear to me. however i see jgraham's point that we now have the right information. so if we adopt a model where we can ship faster that seems reasonable. that said the wg model for webdriver has historically been important because we started with this not having all the browser vendors onboard. 6
years ago Microsoft/apple were very different so historically having the patent protection was important. if browser vendors are happy i'm okay to continue the current model, i just want this to move forward and have investment from everyone
AutomatedTester: auchenberg
auchenberg: i'd like to separate the convos here. there's one about webdriver and it's future and one about cdp and it's future. there are a lot of people here for cdp incubation. my suggestion is we move the incubation work for cdp forward and then allow the webdriver work to continue after that.
AutomatedTester: so rbyers wants the vendors to agree
rbyers: i just want to see who want to work in the wicg
ato: i'd note this group doesn't have everyone who's interested in working in the wicg. we have a DevTools team and we don't want to speak for them.
rbyers: i just want to see who has interest
jgraham: the point is to get the official consent
brendyna: Microsoft agrees
RESOLUTION: CDP is going into an incubator group/CG. There is sufficient support from vendors and independent parties.
rbyers: Google agrees
titusfortner: Sauce agrees
boazsender: Bocoup agrees
rbyers: next step is get a repo created and continue the convo on an email thread
JohnJansen: do community groups have chairs?
rbyers: no, wicg does but not per subset
brendyna: one last question, do we have Mozilla interested/
AutomatedTester: i think so but
since Harald isn't here I don't want to speak for him
... the general consensus is yes but need to verify
auchenberg: any comment from apple?
brrian: i'd need to ask the DevTools team
AutomatedTester: fair, we got the two we needed and we can move forward
jgraham: we can move forward now
rbyers: second one was about the process for other groups to ...
ato: one second, can we have a resolution on what we're doing with this wg
JohnJansen: yeah
jgraham: it sounds like Microsoft want to keep publishing recs and that therefore means there has to be a wg
JohnJansen: we care about recs
<AutomatedTester> RRSAgent: make minutes
jgraham: so does it make sense to continue the wg or drop out and spin something else up
JohnJansen: keep the working group, don't drop other things. we should extend the charter in a way that allows apple/Microsoft to continue their participation in a way that it's royalty free
AutomatedTester: are we happy to do the extension? Microsoft has agreed, Mozilla will, my sense is apple needs it
brrian: we need it
AutomatedTester: that's fine
jrossi: do we want to re-charter for what we do after webdriver
jgraham: MikeSmith what's the difference between extending and re-chartering?
MikeSmith: we need to extend
first, then get to rec, then declare victory. that's separate
from re-charter
... after we extend, if we're going to have another charter. we
can't extend the same charter we've had for 6 years.
... i'd also say it's not a given that if we decide we want to
re-charter, that the w3c will say we can re-charter
... we'll need the w3c to decide if it's something we want to
re-charter for
... w3c will take into account if vendors want to continue, and
if we do then that'll be take into account
AutomatedTester: how do we do
webdriver extensions for other wg's to add to webdriver for
testing
... will it be better to have a wg for that scenario or will
this fizzle out and hope you speak to the right people? how
does that play out?
MikeSmith: it doesn't make any difference. e.g. in reality the technical group for html has been outside of the w3c so it's quite possible to have solid technical communication about work that isn't in the w3c. whether it helps across wg's i'm not sure it makes a difference.
AutomatedTester: what i mean is, potentially there could be no one to do those technical reviews for other wg's
jgraham: the question is does the
work depend on there being a wg?
... i think the answer is no
AutomatedTester: WebVR as an example. will it be easy for a technical review to have a wg to reference?
MikeSmith: i don't think that will matter
AutomatedTester: if it doesn't matter then that is fine
jrossi: I've been in groups before where there were questions about the state of other specs
MikeSmith: the current policy does favor on normative reference/specs published in w3c space. the gist is from the w3c side the group doesn't have to be in complete agreement that a wg is necessary but we need a least a couple of people feeling strongly about a wg being necessary to tell us that
AutomatedTester: okay, we at
least got 2 groups that could say that. apple want an
extension. we want to move to a level 2 and have a list of
goals. to me i don't think re-chartering sounds like a bad
idea. we have goals we can work towards/ship. i agree with
jgraham that we'd want to do things faster but to our defense
we've been learning a lot along the way and i think we can now
work faster moving forward.
... so i guess if people are happy we can put it to a vote
JohnJansen: we should vote on the extension and then the re-charter
jgraham: does anyone reject a renew?
RESOLUTION: Agreement to extend the charter until the specification goes to rec
<ato> +c
AutomatedTester: we'll have a mailing list later about re-charter
jgraham: sounds like though in the room that's what people want
JohnJansen: from a policy perspective does MikeSmith take an action?
MikeSmith: ... yeah
<scribe> ACTION: MikeSmith to follow up on renewing the charter
jgraham: the sense in the room is we're on board for re-chartering even if that's later
JohnJansen: next thing we need to
do is decide if we can get to rec in 3 months or 6 months
... last time was 6 months and that was long
jgraham: think we're running out of good will
jrossi: I think that's why having a re-charter plan will be meaningful
ato: as long as the w3c is okay with us then we can continue otherwise we need to go somewhere else
MikeSmith: we're happy to
continue having the wg but it's been delayed for quite a
while/longer than it should have but i blame myself for not
driving this.
... I need to come back with an answer and say we have strong
support for keeping the wg going and that we won't make the
same mistakes
... but we got that so we're okay
jgraham: specifically for charter extension do we need to pick an amount of time?
MikeSmith: we should say 6 months, plh may have problems but we have 4 weeks after we return to cr and then more time after. realistically we need more time
JohnJansen: plus holidays, 6 makes sense
jgraham: practically, to set the
right exptectations around decisions. that gives us a maximum
window of time to make adjustments like getting implementations
done and/or removing stuff from the spec
... seems like 2 months of just process, which means that, with
buffer, probably anything we're talking about should be
implemented in the next 2 months
... so in regards to removing things or can we do it, it will
be can we get it done by end of the year. mid january
... if this is a thing implementers are going to do quickly
then we don't need to remove it, otherwise we do
JohnJansen: we also need to fix bugs in the test which cause us to think we're failing things we aren't
jgraham: for the details i agree, for other things like user prompt handlers, when people have this conversations had elsewhere, the framing people need to keep is that we have 2 months
<boazsender> scribenick: boazsender
rbyers: the next item is, as
we're pushing for web-platform-tests be the only way we do
integration tests, and the biggest blocker has been automation.
and we finally have a way to call testautomation.click() and it
works. now we want to open the flood gates, to give a path for
new features in chromium to use webdriver extensions.
... so the advice we've given for now is, look what we did for
click, coordinate with BTT group, and write a web driver
extension spec.
<scribe> scribenick: clmartin
rbyers: wanted to make sure this group was okay with that
AutomatedTester: this group agreed in TPAC/japan that this group wants to do that
boazsender: wanted to do?
AutomatedTester: the idea was the group would do extensions to webdriver. obviously some things are webdriver specific that possibly people will need help with but ideally just sending it to the mailing list and we can help will get things into somewhat of a process without it being to heavy handed. ideally it won't be heavy handed at all.
rbyers: i'm hoping to make this work effectively that we're close to telling every engineer that you can't make changes without wpt and we'd need webdriver to be ready
jgraham: i think this is a question about the policy here. if it doesn't make sense for web developers to use they should use dom apis, but we're also bad at what web developers needs. Bluetooth has a dom api but there are use cases to test Bluetooth that maybe should end up in webdriver
clmartin: if it's vendor specific it can also go behind a vendor specific webdriver command/capabilitiy
jgraham: right but it's more about it if it makes sense for webdriver
ato: for existing features finally webdriver is coming into play for automating wpt
<boazsender> https://github.com/w3c/permissions/pull/151
ato: i think the next question is what to do for other wg's who need to test their spec. e.g. webdriver doesn't have a great story for permissions. bocoup had an example of an improvement for the permissions spec. it's very exciting.
<boazsender> https://bocoup.github.io/permissions/index.html#automation
ato: been trying to follow up
with bocoup as there is some confusion about how they hook into
webdriver. it's in the spec that webdriver wants to allow other
wg to add it in their own spec
... we have a job to do on following up with wg who want to use
webdriver that we have the primitives necessary to adopt
webdriver.
... so far we've had issues around things like errors
<kereliuk> ^the bocoup link is out of date from the current automation branch on bocoup github
ato: had a meeting with WebVR and they seem to have a fairly solid testing strategy as fair as i could tell and nothing that conflicts with the way webdriver works so they're going to drop by tomorrow
AutomatedTester: boazsender ?
boazsender: the thing that we experienced making this proposal was a lack of clarity on where it should happen. so an agreement from this group on how to do this it would unblock a lot of other people doing this. yesterday in the testing breakout Vincent was asking similar questions on Bluetooth/usb.
jgraham: i think the agreement
exists in principal, the idea that people should be able to
extend it in their own specs isn't something people have been
opposed to.
... are you asking for documentation?
ato: the way that the terminology
around extension commands is currently phrased, it's about
vendor extensions, but in another chapter we talk about
extension commands in a different context to be used
... we need to tighten that up as well
simonstewart: the original idea
was to use a vendor prefix for experimental commands, and then
move it to the spec when you're ready
... however it's also totally legit to do that in your own
spec
jgraham: this should be clear in the spec, you must not use the extension point because it's for vendor specific thing
boazsender: for
documentation/direction. even as early as the wicg people
should be thinking about how they create a testing interface
for that
... it'd be great to hear an example
rbyers: everything that comes will be standardized but part of incubation is we want it to be safe to fail. web Bluetooth is chrome only right now but we want to be able to fail but still be plausible to be a spec
simonstewart: so if web Bluetooth is a spec then define it there, if it's chrome specific then use the vendor specific
<scribe> ACTION: simonstewart to clean up documentation on how to hook into the spec.
boazsender: and documentation for how to think about testing your spec
clmartin: we care about this looking holistic/uniform across the api
simonstewart: i'll clear that up in the spec
AutomatedTester: ideally if a chromium engineer wants to do web Bluetooth and they write a command and JohnJansen says it needs work we can iterate on that
clmartin: what about if there's a disagreement
AutomatedTester: w3c should have process to settle disputes reasonably
<boazsender> RRSAgent: make minutes
AutomatedTester: should we break for lunch?
lunch time!
<ato> Scribe: ato
plh: Besides two features, my understanding is that you are almost ready to move to recommendation.
AutomatedTester: Yes, but not everything is green.
plh: There are edge cases, sure.
AutomatedTester: For all of those we have bugs on file, and we expect people to implement it.
plh: So besides these two features you would be able to move to rec?
JohnJansen: I believe so.
plh: If it makes sense to you, I can try to make sure it makes sense to the director as well.
AutomatedTester: We got some tests that are failing, but we have bugs on file that show that people are going to do that implementation.
ato: Was there agreement over
lunch what to do about these features?
... Do we rip the features out? Would that make it easier to
move to rec?
plh: I think you should publish
without these two features.
... I am not asking you to remove it from the master
branch.
... In the mean time you continue work on v2.
AutomatedTester: We do want to have those features.
plh: I would like the master
branch to be the v2 right away.
... Can you publish an editor’s draft of v2 next week?
AutomatedTester: Yes.
plh: It is independent from the
CR goingto rec.
... You can publish another rec right away, if it is supported
by vendors. It would not take you two years.
simonstewart: If we drop these two features, we could move to rec right away?
plh: Yes, this means the
specification is not at risk.
... In two months you would get another approval from the
director.
... Then you would get your recommendation.
... You will keep focussing on master and the features you want
to have in the browsers.
AutomatedTester: If we are dropping these features and it kicks off process, it is going to be two months/four months?
plh: I am going to give you precise dates.
clmartin: If we remove something from the CR, does that retrigger the CR process?
plh: I will ask legal.
MikeSmith: Is it really a legal thing?
plh: Yes.
MikeSmith: We would prefer not to go back to CR.
plh is showing a presentation with an astounding amount of detail.
<plh> https://w3c.github.io/spec-releases/milestones/?cr=2017-11-16&noFPWD=true
AutomatedTester: Thank you, but I’m not after specifics.
plh: 60 days exclusion opportunity last date with a PR 12/14/2017 will be January 15 2018.
<scribe> ACTION: plh to check with legal if we can move to PR.
simonstewart: Answer today would be nice.
plh: I will check.
AutomatedTester: The next thing, if we go through this process, we also need our charter extended.
plh: If you can guarantee how
long it will take, I am willing to argue you should get the
extension.
... I wanted to know what guarantees you can offer me.
... I have invested interest in making this successful.
AutomatedTester: We want an
extension so we can get to recommendation.
... When, say we remove these two features, what are the
potential next sticking points that might come up?
... Not knowing enough of the process, history has shown us we
have gotten stuck.
plh: You will have to make a
choice, do you address any incoming comments in level 1 or you
do so in level 2, the master branch?
... My suggestion is you address them in v2.
AutomatedTester: We want to get patent protection.
plh: But if v2 is upcoming, that
is what they will see.
... The bottom line is, in the worst case scenario, you have to
go through the CR period again, and we should extend the group
until the end of February.
... The best case, mid January.
AutomatedTester: Saying we do all these things, will the dates be moved back because of Christmas or Thanksgiving or things like that?
plh: The tool is smart.
... Publication moratorium is built in to the tool, so it
pushes the different dates back.
... If I extend the group, the next question is going to be
what is next after that?
AutomatedTester: We have a set of
goals we want to hit.
... We have been slow in the past, but this is because we had
no idea what we were doing.
plh: I would _love_ if you could
go to rec every six months.
... Can we produce a CR, or can we go forward with the
PR?
... That is what we need to figure out, and I will do that for
you.
AutomatedTester: We need to finish this off, but we have general concensus that the WG will move on.
ato: Let us discuss the technical options.
<AutomatedTester> scribenick: automatedtester
scribenick automatedtester
jimevans: what does Geckodriver do when there is a prompt?
ato: at the moment it doesnt act. We have return an error when a prompt is hit
jimevans: I can move the IEDriver to match how GeckoDriver does things
jgraham: We need to update the spec to say return an error when a prompt has hit
RESOLUTION: We rip out web window, web frame et al. from the level-1 branch.
ato: for the frame/window returned from executescript we can remove since no one has implemented it
<ato> ACTION: jimevans to adjust IEDriver to match geckodriver’s behaviour.
<ato> ACTION: Adjust wording of user prompt handler to match behaviour of geckodriver so we can move to rec.
jgraham: it feels like we can have a patch by the end of today on this
RRSAgent: make minutes
<JohnJansen_> scribe: JohnJansen_
<AutomatedTester> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F#Thursday
<simonstewart> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
simonstewart: break up the agenda
easily
... changes to actions
... things not controversial
<scribe> ... new features
UNKNOWN_SPEAKER: let's do new
things first then uncontroversial
... or easy first
cl
clmartin: easy first!
simonstewart: ok
... lists agenda items
ato: new tab command
ato: users of selenium are
working around this currently
... geckodriver does not allow keyboard input to affect the
os
... alternative "window.open()" does not actually open a new
window
... so we need a primitive for enabling users to open a new
TLBC
simonstewart: people want to do both
<scribe> ... (new window, new tab)
jgraham: agree there are some
cases where you might prefer one or the other
... difficult to know how to talk about that
... say, a browser that does not have "tabs" or "windows"
simonstewart: suggest if you don't have tabs/windows ignore the command
ato: good idea for this
... what do you return for a browser that has neither?
simonstewart: if you are stuck with TLBC, return unsupported command
clmartin: and give them back a
window handle id
... we have hacky workarounds
ato: volunteering to do the PR?
<scribe> ACTION: clmartin submit a pr for this
<ato> ACTION: clmartin to draft PR on New Window command
<ato> Scribe: ato
AutomatedTester: This is from clmartin.
clmartin: we have a vendor
specific command for this today.
... How can we clear out data for a fresh session?
... Because, with Edge, we can’t do that.
... We have ability to nuke the entire session.
... But that is too crude.
<JohnJansen_> ato: more things than history that you want to delete
<JohnJansen_> ... cookies is covered already
<JohnJansen_> ... but not all browsers support all these features that people might want to delete
<JohnJansen_> ... and allow browsers to add new things to the list of things
<AutomatedTester> scribe: JohnJansen_
clmartin: or maybe just have history
simonstewart: you have cookies, maybe then cookies should be in this general delete as well
ato: I don't think including
cookies in this is the right approach
... even delete all cookies is just for domain
jgraham: localstorage as well
clmartin: starting point would be
doing what happens when users click "Delete History"
... goal is to be in a clean state
... not mimic user clicking the button
ato: new browsers may not even have this command
jgraham: is the problem that Edge does not support clean sessions
clmartin: I've seen that users
want to have a clean session at the start, plus also clear the
session in the middle
... edge does not use a clean session, it uses current user
jgraham; new private window as well
<Zakim> plh, you wanted to provide a non-answer to the question of CR/PR
AutomatedTester: will you know more tomorrow?
plh: we may not need to go back to CR
simonstewart: how should we tell you
now back to history discussion
clmartin: even if we had multi session, this probably still has value
<ato> https://github.com/operasoftware/operaprestodriver/blob/master/protos/core.proto
ato: lines 13-28 are examples of
different things we'd clear
... i think we need ot have an api that is vendor
specific
... if we have specific endpoints for each type of data
... I'm a bit skeptical about that because we could add 15-20
commands
clmartin: yeah, it's better to have a command that can be browser specific
jgraham: usecase: just start a
test with a clean state
... do people have specific use cases that are different?
jrossi: our UI does not allow granular deletion
clmartin: we would match what the UI does
ato: this is not really about acting like a user
jgraham: deleting history is
something that a user can do
... maybe there are wpt cases for this
... delete history, cache in the middle of a test
clmartin: dev wanting to delete a storage mechanism
<jrossi> correction: Edge UI does allow granular deletion. Ignore me :-)
ato: if we are adding this so
edge can delete data, then not a strong use case
... but if there is general interest from devs, then we should
consider it
jgraham: vendor specific tests
that are doing this via vendor api
... maybe google has layout tests that do this
... maybe this would help them
... could be part of the storage spec
clmartin: i've seen a few people
ask, but that COULD be Edge specific
... definitely a use case for vendors, though
AutomatedTester: concern at the
moment is: no one wants to restart the browser, it's
expensive
... users might call this and expect it to nuke all, but some
state comes through and breaks them without them knowing
why
<scribe> ... new features, for example, that are in a browser but not added to webdriver
UNKNOWN_SPEAKER: restarting is a
good clean start
... could create a bad state
ato: i feel like I'm lacking data
on this
... I'd like to help solve Edge's problem here
... but need more input
... are there tests in chrome?
... are there tests at Moz where'd you want this
... testing a feature, it would be helpful to delete
storage
brrian: these things would be great for WPT, but not necessarily for website testing
juangj: from a selenium
perspective: people are using Edge and attempt to get a clean
state
... very difficult to manage
... tell people to "give up, start a new VM"
... like brrian said, this is subtle
... maybe a product like selenium should not expose them
<ato> Scribe: ato
clmartin: I wouldn’t exactly create a defined list.
ato: There’s a simple solution: You add a getter returning a list of topics you could delete.
jgraham: Potentially this is a
Selenium question.
... If we think it is required in WPT, we could expose it only
through the protocol but not in Selenium.
... Option 2 is a case where we don’t add a WebDriver command
for testing in WPT, but we invent some web API privileged
API.
... That is a fine consensus if we don’t think we should expose
this to developers due to the reason it could be a
footgun.
... We shouldn’t reject a feature that is useful for testing
browsers because we think it could be confusing to a subset of
users.
... We should assume that it is a good idea to add new
commands, but only if those are useful to developers.
clmartin: I’ve seen devs come ask for things I was surprised about.
jgraham: I don’t want to haveanother discussion about CDP, but perhaps the lesson from to be learned that people want low level access.
JohnJansen_: We have implemented
this feature in a vendor specific endpoint.
... We have web devs who want it and people who will use
it.
... I don’t see the harm is that someone uses it
incorrectly.
... We not allow people who do understand it use it, and let
those who don’t be surprised.
AutomatedTester: It’s a foot gun and this is what I want to prevent.
jrossi: At a granular level?
JohnJansen_: The only way to clear state in Edge is to nuke all cache.
jgraham: Are doing this at the start of a test?
JohnJansen_: No, people are also doing it in the middle of a test.
ato: Yes, and I guess that would be the exact use case one would have writing WPT tests.
brrian: Why not just open another window?
JohnJansen_: There are certain
types of cache that exist globally.
... There is persistent user data as well, which is not web
platform things.
AutomatedTester: I have never seen a request like this before.
jgraham: This is a request, to be fair.
AutomatedTester: In 10-15 years I have not gotten this request.
ato: There are multiple data points here.
jgraham: For all we know it is
exposed in CDP.
... It is true that web apps are becoming more complicated.
brrian: I don’t understand why you could not rewrite the test to open a new session that is clean, and you log back into Sharepoint, and you do the whole thing again.
[jgraham explains a testing scenario]
AutomatedTester: OK, let’s file an issue/PR for this and continue the discussion there.
JohnJansen_: You should include concrete examples.
brrian: For validating utility I
want to see concrete data points.
... Alternatively we can offer a clean session
capability.
<scribe> ACTION: clmartin to Write issue/PR about deleting history &c.
simonstewart: The way Selenium
works, not WebDriver, is that we have a thing that looks at
what you pass in to Element Send Keys, and if it is a file then
it uploads that file to the Selenium server.
... Send Keys does not do a file upload in the spec.
... You want an upload file command moves the file to the
server.
... Then one to upload it to the driver.
... For this existing WebDriver session, upload file.
... We want a generic command to upload an apk to the remote
machine.
... I believe safaridriver has it.
brrian: Of?
AutomatedTester: No one has done
it except the Java/Grid.
... It sends it to the specific machine.
... But none of the remote ends have historically had to
know.
simonstewart: It is common behaviour for the intermediary nodes, so we can put in specification text that is specific to that node.
clmartin: My concern is that you don’t type into <input type=file>.
brrian: But you also don’t type a file path.
AutomatedTester: It’s because
it’s an <input>.
... Each OS will open its own widget for <input
type=file>.
simonstewart: I think I would like to have something in there that for intermediary nodes.
AutomatedTester: I’m not saying no.
brrian: safaridriver is the only
thing that can put a file on that device.
... I don’t think we could run the Selenium server there.
ato: Good use case.
AutomatedTester: Yes.
... And we need to do more mobile friendly in the future.
simonstewart: You would like to have equivalent of <input type=file> Send Keys endpoint to upload a blob.
ato: I like the idea of the Element Send Keys blob.
simonstewart: The payload to Send
Keys is {"text": "…"}.
... We could have {"file": […, …]}.
... I think this covers brrian use case.
brrian: Current semantics is that you make multiple calls to Element Send Keys to select multiple files.
<brrian> "titusfortner"
AutomatedTester: The question to
Sauce was: a lot of this is going to be intermediary node
heavy.
... What would you want?
... This will influence you quite heavily.
titusfortner: It’s not really any difference.
AutomatedTester: Then the other use case is to upload resource files, such as apks.
ato: Is that something people actually use?
titusfortner: Yes it is.
ato: Putting in a file transfer
protocol in WebDriver sounds very scary.
... For Android I would use an adb debug bridge to move the
file.
... I don’t know about iOS.
boazsender: Uploading a file to a file element sounds like something the driver could do.
juangj: If you just want the file to exist on the filesystem, it does not sound like WebDriver’s job.
<kereliuk> juangj
JohnJansen_: I don’t think we should be uploading arbitrary files to an endpoint.
AutomatedTester: People are not
sure about intermediary node endpoint for uploading arbitrary
files.
... If there was a /upload.
simonstewart: How do you get the apks to the machine?
titusfortner: I think we would prefer that to not be a concern of WeBDriver.
juangj: [agrees]
simonstewart: Maybe what we want is to have interop between browser stack and sauce.
jgraham: Cloud provide API spec
is not really related to WebDriver.
... The browser driving protocol is our domain and I think this
requires different expertise.
tboyles: If you gave us a
standard, we would definitely use that standard.
... We have worked around it so it is not something that blocks
us.
... We don’t need this today. We have solved this problem.
simonstewart: Array of file blob and file name.
ato: Just to be clear, we can’t change the current behaviour of Element Send Keys passing a file name to <input type=text>.
brrian: If you write your test in Python, it can encode it or it cannot.
JohnJansen_: Break?
Yes.
<scribe> ACTION: simonstewart to write proposal for Element Send Keys to take new "file" field.
<clmartin> scribenick: clmartin
ato: it came up in the wpt
testing session that webdriver should have a separate command
for cutting/copying/pasting in the browser
... if we want to be more mobile friendly then we need to
handle text selection more nicely
jgraham: implementing selecting
text will be a disaster because it's complicated
... is there an existing platform api for selecting text so
that the minimal change would be to copy the current selected
text
... and a command for pasting it from the clipboard
ato: what you're suggesting is have some script that uses the platform apis
jgraham: yes
AutomatedTester: so like dom range?
jgraham: yes
... ideally we do this without creating an alternative to the
DOM range API.
ato: Add a whole new range of serialization methods for different types of nodes.
jgraham: The point is you pass in I want the selection to cover this point in this node to this other point in this other node
titusfortner: Is it relevant that we have an atom for selecting text?
jgraham: Yes
titusfortner: We have an atom implementation in the ruby atoms
simonstewart: What does it look like?
titusfortner: Let me check
<titusfortner> https://github.com/SeleniumHQ/selenium/blob/master/javascript/watir-atoms/watir.js
ato: So the thing we want to make WebDriver do is take that selection (whatever selection is made in the browser) and put that on the clipboard.
simonstewart: I guess you want to post the keyboard and get the keyboard.
ato: It's not necessarily related to clipboard.
simonstewart: Clipboard*
... So posting would be "I want to modify the clipboard" and
getting would be...
jgraham: That's the wrong model, you don't want to know the value of what's on the clipboard
simonstewart: Well you want to do copy/paste right?
jgraham: Yes exactly. So you should do two commands. One is to put something onto the clipboard, and the other is paste where we put it into something.
brrian: I've asked ryusuke to
come over here to describe the ask.
... When you're copying you're copying everything whatever the
OS does.
jimevans: Clipboard formats are
not fun.
... We should not expose the clipboard formats
<inserted> gsnedders: there's also "paste & match style"
jgraham: Is what you're saying is that there are multiple kinds of paste.
ato: So how do you trigger those?
simonstewart: If you do apple-v paste you get one kind of paste, if you hold different keys you get other pasting formats
ato: But the use case is that they have an action sequence you can push the keys you want to validate
JohnJansen: When you paste into a spreadsheet you get options on what you want to paste.
jgraham: So it sounds like there is browser specific options when you go to paste
JohnJansen: Yes it is browser
specific.
... We have a clipboard API for testing. Our browser inspects
what is being pasted and tries to determine how you'd want to
paste that.
... So you may want to paste a number vs. a string, or raw
value vs html.
jgraham: Right, the difficult part is there might be multiple possible paste options.
JohnJansen: In Edge you get a table and the table changes size based on how complicated what you're pasting is.
jgraham: So the options are actually context specific.
JohnJansen: Yeah
gsnedders: It turns out browsers are complex
titusfortner: *laughs*
jgraham: I'm not sure if anybody implements the paste part of that
ato: It is entirely possible that we want to hook in at a deeper level.
jgraham: Yeah but that API is, clipboard access turns out to not be very secure
ato: brrian For that to be shippable, you'd have to null it out before you start the session.
brrian: If you want access to the
real clipboard that's very dicey.
... and how do you abstract that in a way the browser works,
idk
titusfortner: Is there a reason we can't just do cmd+c and cmd+v
jgraham: The problem John is describing is that the clipboard isn't as simple as you like.
gsnedders: Also which clipboard
titusfortner: I get what you're saying, if you want to emulate what a user would do then that wouldn't work.
jgraham: A vendor could do that if they want to.
tboyles: Would it make sense to do that?
ato: If something does ascii now and we release a new version that's differnet that's a problem
brrian: We're talking about clipboards
rniwa has joined the room
ato: We were talking about how different clipboards on different browser on different OS's are different
rniwa: correct
ato: John was talking about how copy/pasting depends on the context you're in.
JohnJansen: I was saying that we
shouldn't try to implement this as it's more complex than
simple copy/paste
... We also have security alerts when copying/pasting
data
... Makes it hard to standardize
jgraham: but is it impossible?
JohnJansen: No it's just code but
jgraham: But we've agreed that
browsers can be different.
... Edge could pick a default option and go with that
JohnJansen: Yeah we can do that
ato: I was curious about preventing business content from getting copied/pasting. Is that something that gets prevented at the os level?
rniwa: Content editors have the
requirements that they can copy/paste stuff from Microsoft
Word. For those you can't mimic the content that Word
generates.
... With clipboards there are things that are very different
between platforms. In Windows you have one impl. I don't know
if Windows has a way to archive, store images.
... On a Mac this is different. Not only do we support plain
HTML but we also have an archive format that can contain all
the data for the file which means the mechanism by which things
are serialized can be very differen.t
... So if you're using WebDriver to test copy/paste behavior
it's important this works.
... If you're just copying within the page you almost don't
need an API for this.
simonstewart: So you do need to modify the system clipboard?
rniwa: Right
ato: We brought this up because you brought this up on Tuesday (rniwa)
jgraham: I think things outside
of the browser window are out of scope
... You have the use case you want to copy from Word, I think
that's valid but standardizing that via WebDriver isn't going
to work.
simonstewart: Setting the caret position might be helpful.
AutomatedTester: You can do that
ato: We can do the select via the DOM API's
simonstewart: What we see are users using the browser automation, then swap to a lower level automation, then swap back
ato: Right but we still need a way
simonstewart: So what you'd do is use it to copy then use a different tool to paste
ato: If I were to implement in geckodriver then I'd look at what copy does and I'd abstract that away, I'd call that endpoint
simonstewart: I agree with
jgraham and ato but I'm saying this is one way people would do
it
... Everything we do today allows you to test in parallel. The
minute you start operating on the system clipboard you break
that.
ato: I wouldn't expect geckodriver to work like that
jgraham: It's not about that
gsnedders: It's about global state
JohnJansen: I confirmed any web
content, copied html content and tried to paste it to
Microsoft.com and I got a warning
... If i paste the same text into a word doc in the browser it
maintained the formatting
... It's very complex
rniwa: In general the two use
cases are:
... 1 is copy paste in the wpt
... at the end of the day if we don't support any ability to
test the system clipboard, then all authors that work on web
properties that utilize the clipboard then they wouldn't be
able to test it.
simonstewart: They will be via other tools
rniwa: I would prefer the approach that by default you use a synthetic clipboard, but provide some way to test the system clipboard. One tricky thing with the system clipboard is that there is an order of types. What the browser exposes and what is the default behavior would be different. Adding the ability to order these things in the system clipboard would be complicated.
jgraham: Do we think the use case
of a synthetic clipboard makes any sense
... It sounds like the system clipboard is complicated but some
allow you to create a per-app clipboard
rniwa: The system does
simonstewart: Does Windows?
JohnJansen: idk
gsnedders: Idk if Linux test environments do that
rniwa: For global no, but regular copy paste would
jgraham: If that exists then it
would make things easier.
... If you want to interact with the system then you'd need a
vendor extension. It's not worth spec'ing that.
ato: So I just checked on MDN and gecko has this concept of logical clipboards. A subset of those are mapped to the system clipboard.
JohnJansen: In the browser?
ato: Yes
jgraham: Not expose to content.
rniwa: It would be nice to trigger copy/paste without having to hard code the command (shortcut keys)
jgraham: Well that doesn't work.
rniwa: Right, so it would be good to...
ato: I agree it would be useful to add to WebDriver. I sense there's an action coming on that we need to do some more research.
JohnJansen: Yeah we need more research. I'm hesitant to access the clipboard right now.
rniwa: The system clipboard?
JohnJansen: I'm not sure.
gsnedders: If we do nothing then people will keep using ctrl+c
brrian: It doesn't work anymore
gsnedders: So it only works in some browsers
jgraham: From a security point of view if that does to work in MicrosoftWebDriver then you add a capability
AutomatedTester: Are we all agreeing that if we're going to do this we need to do more research?
ato: I will file an issue
<ato> clmartin: Thanks!
AutomatedTester: It's 3 minutes
past 4, how's everyone doing?
... What can we handle?
simonstewart: The two additions to the actions API which don't feel terribly controversial
ato: Who added that?
simonstewart: I did
ato: If the element you want to reach is outside the viewport youc an't do that
simonstewart: Right what youw ant
to do have is have it scroll into view so you can interact with
it
... or even if not that the ability to scroll the view port
jgraham: For elements it can work like the existing click command so for the start you can find where it is and scroll that amount
simonstewart: The problem is imagine you're doing drag and drop
jgraham: That will work, before
you do scroll into view, you ask where the element is on the
page which has a set of coordinates and you say in order to
have that in view you need to scroll by x and y and that's how
much you scroll
... Because as you scroll you can also get scroll events
simonstewart: But we do want to be able to scroll?
jgraham: Yes
ato: What kind of primitive?
jgraham: A scroll primitive that works like point to move in the sense that you either provide it an offset relative to the top of the viewport or an element
simonstewart: So this is exactly like the origin astuff we have
jgraham: Like input type scroll
rniwa: One question, why does scroll to not work in this case?
simonstewart: The actions api is a sequence of events you fire off simultaneously so you don't pause in the middle of that
rniwa: I see
juangj: What I hear jgraham saying is that the primitive is we'll compute and do that and if it doesn't result in it coming into view that's okay
ato: Yeah
jgraham: That's how the existing stuff works so I would expect it to work like that
simonstewart: When we do a pointer move action we have an origin
<AutomatedTester> https://w3c.github.io/webdriver/webdriver-spec.html#dfn-dispatch-a-pointermove-action
jgraham: Right I'd expect it to be the same
ato: What kind of value or metric would we be passing that
simonstewart: We'd probably use the null input device
ato: Right but how much do I scroll
jgraham: It works like something,
you give it an x/y or an element
... and it computes the amount for you
ato: So the x/y calling is fine, but in the case of an element you'd have to invoke the is pointer interactable
simonstewart: no
jgraham: no
... You just query the layout for where is this element and
scroll to that position and if it moves during that then bad
luck
simonstewart: I'm on board with that
ato: that works
jgraham: You have to choose where
you want it to end up
... In the element case you copy the semantics of scroll into
view as closely as possible
gsnedders: Which is what click already does
simonstewart: Yeah, we'd want element click to be implementable on top of the actions api
titusfortner: Does scroll into view move it into the center
ato: Because!!!!
jgraham: Okay, but...
... Lots of other words
... Do what the semantics are and stick to what is already
being used for other thing
titusfortner: I just want to avoid static headers/footers if they exist because sometimes scroll into view will be hidden
AutomatedTester: If that happens we can't solve that
titusfortner: Well offset, if we have an offset then that's good
ato: How do you do that as an
atomic action
... You can do it to an element but by an x/y there's nothing
in the api that allows that
simonstewart: You treat it like a pause
ato: How do you move those few extra pixels
jgraham: Scroll into view does that
juangj: There's a way to do that
jgraham: regardless it's possible to scroll in the web browser by an amount
ato: But that's different. First you call the dom api for scroll into view
simonstewart: there's nothing conceptually difficult here right
jgraham: the DOM API may or may not allow thos eoptions, then you'd call the underlying API your scroll into view api is using
simonstewart: Yeah scroll into
view with the element as the origin as 0,0
... If you called scroll into view with an origin that was
relative you need an x,y
ato: But it doesn't support that
simonstewart: I need a new name, scroll action, with an x/y
jgraham: Just call it scroll
ato: We can scroll by line but not by pixel in gecko
AutomatedTester: You can scroll to a point, scroll to, and pass an x/y
ato: Yeah scroll by offset
jgraham: Any other issues?
ato: Just want to make sure we can do this
jgraham: Is it instant or smooth scroll?
simonstewart: It doesn't matter
jgraham: You might get different
behavior
... You might want an option
rniwa: Hit testing will happen in
browsers if you aren't doing an instant jump
... For drag/drop you'd get events
simonstewart: You'd allow scroll options to be passed in
jgraham: Especially if it's already int he scroll into view apis
juangj: scroll into view will give you that
ato: Scroll into view does,
scroll by doesn't
... I think we can do that
An action to add this to the spec
simonstewart: The second
suggestion is wait until interactable
... This is for suckerfish style menus
... When you move the mouse onto the element, hover and then
wait
jgraham: How do you tell when that happened?
<AutomatedTester> https://w3c.github.io/webdriver/webdriver-spec.html#dfn-interactable
juangj: Give a CSS selector to an element you'd expect to appear and wait for it to appear
ato: It's technically possible I
guess
... With a pointer source because we have that definitioin
simonstewart: You do the test
with the pointer interactability
... It would be a pause of an undefined length
AutomatedTester: How long is that
wait
... If that thing doesn't appear
gsnedders: A frame or two
jgraham: It might never appear
AutomatedTester: I don't care how we do it we need a way
simonstewart: 2 ways to do it, 1
specify the timeout and wait for it, 2 allow an upperbound on
wait for interactability
... If you were to put it on the global timeout it would work
that way
jgraham: It uses the implicit wait timeout
ato: No
simonstewart: No that's very different
jgraham: We have a note in the spec that says implicitly wait until it's interactable
ato: That's wrong
simonstewart: That's what it was originally
ato: It would wait for the element to be locatable
simonstewart: We can use the
implicit wait timeout
... But ideally set it to 0
AutomatedTester: Always be 0
simonstewart: But with this it might take a while
jgraham: I'd strongly prefer you can specify a timeout for this action with the global state is the fallback
titusfortner: Interactability is not something you can query right
simonstewart: The spec has some clear definitions on what pointer interactable is
<ato> titusfortner: https://w3c.github.io/webdriver/webdriver-spec.html#dfn-pointer-interactable
rniwa: Does it make sense to wait for an event to be fired
AutomatedTester: What event?
<simonstewart> Interactable definition is here: https://w3c.github.io/webdriver/webdriver-spec.html#dfn-interactable
rniwa: You name an event and maybe while doing the test you need to do a query that takes an arbitrary amount of time and when you're done you fire
<simonstewart> Pointer interactable: https://w3c.github.io/webdriver/webdriver-spec.html#dfn-pointer-interactable
jgraham: WebDriver isn't built to
handle that easily but you could do that via execute
script
... In the middle of an action sequence I don't know what the
use case is on waiting for an event
rniwa: Something about drag and drop
simonstewart: Wrong level, WebDriver is at the browser level
jgraham: I don't understand it but I don't think it should be dismissed
simonstewart: I'm not saying it's not in scope, I'm saying the mental model for how webdriver works doesn.t...
jgraham: What would async drag and drop look like
rniwa: Drag and drop we don't do this, but let's supposed we did. In that case maybe in a drag start you'd add something, then the browser would make a callback that I've added it, but
jgraham: What something?
rniwa: Like actual data to drag. In that case you'd want to wait for that thing to be done before it drops
jgraham: How would that work if a user drags/drops?
rniwa: There's an idea of a
promise
... We might not have it in a browser but it's conceivable
simonstewart: So why wouldn't
WebDriver use that, it's supposed to be a user
... Drag and drop should work and if it doesn't there's a
bug
jgraham: If you drop something that needs to wait on data it should get a promise back
rniwa: This isn't a good example
simonstewart: I can see wanting to wait for a mutation event
jgraham: interesting!
ato: Yes
simonstewart: This is another
primitive
... Which we may have consensus around
titusfortner: We can't wait for interactable outside of specific commands
jgraham: The theory is you can do this via the actions API
titusfortner: I'm thinking of the concept of a string of stuff but in actions
ato: Do we think that it's a useful primitive
simonstewart: Yes
jimevans: I think it's amazingly useful but we don't have it in place
ato: I wrote this thing, there are dragons
simonstewart: Add a definition for pointer interactable
AutomatedTester: Clay got lost
ato: The spec says you can set the timeout to 0 and wait indefinitely
simonstewart: That's fine
juangj: You'd expect most callers to set an explicit timeout
ato: With an unlimited wait they'll hit the HTTP timeout
simonstewart: Right
ato: I was thinking it was
unrecoverable
... But it isn't
juangj: What we're saying is if you don't set a timeout we hit the implicit wait timeout why not jus tmake the timeout necessary
simonstewart: There's a different
between should and must
... The spec should just default to the implicit
juangj: Why can't the spec require it
simonstewart: Nowhere else do we require it
ato: What do you mean by the upperbound?
simonstewart: A timeframe that youc an wait
ato: If it can be indefinite why would you have an upper bound
AutomatedTester: Where is it indefinite?
simonstewart: If you don't set an implicit wait timeout...
AutomatedTester: If implicit wait isn't there, set it to 0
ato: I was talking about script timeout
juangj: If you do this implicit wait timeout falls to 0 and the default timeout on wait to interactable is 0 but a 0 timeout isn't userful
simonstewart: No but there's a mechanism to override that
AutomatedTester: And client bindings can set that for them
simonstewart: Not everyone sets it to 0, if you see it set to 30 seconds hteen
juangj: The purpose of this primitive is that running it once isn't useful
simonstewart: Click in actions
just does it
... We have a new action, wait until pointer interactable, and
that waits for a set amount of time for an element to become
pointer interactable, by default that time is the implicit wait
timeout, but you can set an upperbound independently.
ato: Can we get the resolution on
whether the is pointer interactable primitive is...
... I've tried implementing keyboard interactable and haven't
been able to
simonstewart: How come?
ato: It's hard
... Had to dig through a lot of specs
... So the question is, wether you want the command for whether
it's interactable in a keyboard and click, or independent
titusfortner: I don't need what I was asking for
ato: The intention of "is interactable" is can I, using the keyboard in any way interact with an element
titusfortner: Well yeah the idea was clicking on it
ato: I could see a use case for someone
simonstewart: I can see a wait command
ato: If you're working on google ads I could see this being useful
titusfortner: Sometimes I have
users that would like to automatically retry when it fails,
others don't
... When I was coming up with a solution it was tough
... If we had a wait for interactable that would be good
simonstewart: Would that be pointer && keyboard or pointer || keyboard
AutomatedTester: An object would be much better
ato: It's not bad but it's more
complicated than it ought to be
... Could see two separate commands for pointer/keyboard
simonstewart: But then webvr
AutomatedTester: No but people throw it into a lambda
jgraham: I don't understand the use case for knowing is this thing... presumable the next thing you're going to do is interact with it with the keyboard or something else. but not at runtime
simonstewart: Yes you're right, I
think AutomatedTester is quite elegant
... Because it allows for other variations
jgraham: I think that's a good
reason not to do it like that
... I don't want this thing to have to fire up the VR subsystem
to check if it was VR click interactable
... When you haven't been doing anything with VR up to that
point
AutomatedTester: So that one is
easily solved by checking "is subsystem x enabled" no
... But if you're in a VR mode and ask if it's interactable it
would be possible to do that
... And you can gate it on the subsystem being running
jgraham: That's a lot of work for the browser
simonstewart: It is a separate endpoint, it's doing a full HTTP roundrip just to say yes/no
jgraham: I feel like if this was
a thing you think it would be extended, i feel like telling
people "oh yeah just put it in there" seems like it doesn't
scale
... I don't want to say "doesn't scale" but "doesn't scale"
simonstewart: This is where David's point comes in, if you aren't using a subsystem then don't run those checks
jgraham: Then it pushes
complexity onto extension spec authors to properly define
that
... A specific endpoint solves the use case
simonstewart: You're suggestion is a separate endpoint for each type?
jgraham: Yeah
juangj: You know what you want to do
ato: It's also more in style with the other web element specific methods
simonstewart: Is displayed,
enabled
... attribute as well
jgraham: There aren't other cases like that
simonstewart: Which way would
people prefer?
... Should we take a vote?
... Is there a problem with multiple endpoints?
No one disagrees
simonstewart: Do we add the endpoints at all?
ato: I think if we're exposing them through the Actions API, I guess it would make sense to also expose them as endpoints. No one has asked us though.
titusfortner: I have a use case. A transition is happening, try to click it, it's not interactable, it throws an exception. In Watir we abstract that at a high enough level that if we try to do that it's complicated. If we let the user specify it works better
<AutomatedTester> RRSAgent: make minutes
ato: Do we think it could be
confusing to test this in a way that's interactable?
... I could see someone going "I want to click this button but
it's disabled"
... But the is interactable will return true
simonstewart: Oh that's fine
ato: But do people understand that difference?
simonstewart: I see what you're saying
titusfortner: But it's in the spec what it means
yeah
simonstewart: Imagine you're not a good developer
titusfortner: Such a stretch
simonstewart: and you what
developers always do and they don't read the docs, and you have
the option between interactable and not
... If you named the method ...
titusfortner: You can click a disable elemenet
element*
ato: I don't care what you expose
it as
... I like juangj idea doing it in a restful way
... You could do AutomatedTester's thing as well
jgraham: I vote against clever
juangj: Coming back to the question whether it satisfies your use case, waiting for it to be interactable, that's not what this would do. It would just return if it's interactable.
titusfortner: That's fine
ato: I'm in favor of adding this
RESOLUTION: add interactable check command
<AutomatedTester> RRSAgent: make minutes
This is scribe.perl Revision: 1.152 of Date: 2017/02/06 11:04:15 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Succeeded: s/re./rec/ Succeeded: s/a/the/ Succeeded: s/thigns/things/ Succeeded: s/stuff/agenda items/ Succeeded: s/illegal argument/unsupported command/ Succeeded: s/shoudl/should/ Succeeded: s/users/users/ Succeeded: s/users/user/ Succeeded: s/start/part/ Succeeded: s/boazsender/juangj/ Succeeded: s/keyboard/clipboard/ Succeeded: i/Is what you're saying is that/gsnedders: there's also "paste & match style" Succeeded: s/doesn't scale but doesn't scale/"doesn't scale" but "doesn't scale"/ Present: jgraham ato jimevans clmartin soareschen AutomatedTester alrra simonstewart JohnJansen juangj MikeSmith rbyers auchenberg Found ScribeNick: clmartin Found ScribeNick: boazsender Found ScribeNick: clmartin Found Scribe: ato Inferring ScribeNick: ato Found ScribeNick: automatedtester Found Scribe: JohnJansen_ Inferring ScribeNick: JohnJansen_ Found Scribe: ato Inferring ScribeNick: ato Found Scribe: JohnJansen_ Inferring ScribeNick: JohnJansen_ Found Scribe: ato Inferring ScribeNick: ato Found ScribeNick: clmartin Scribes: ato, JohnJansen_ ScribeNicks: clmartin, boazsender, ato, automatedtester, JohnJansen_ Agenda: https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: adjust ato clmartin handler jimevans mikesmith of plh prompt simonstewart user wording WARNING: Possible internal error: join/leave lines remaining: <scribe> rniwa has joined the room WARNING: Possible internal error: join/leave lines remaining: <scribe> rniwa has joined the room WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]