<gitbot> [13webdriver] 15shs96c opened pull request #1144: Remove the ability to return a window or frame from execute script (06master...06to-pr) 02https://github.com/w3c/webdriver/pull/1144
<boazsender> /list
<kereliuk> ScribeNick: kereliuk
AutomatedTester: We have a number
of items today that are in our backlog from yesterday. Lets do
the heavy ones first and then we can get to the other
ones
... Not in the agenda, but maybe WebVR later and they might
come
jgraham: There was some talk of
this but I don't think a room of 20 people is the best way to
do this
... we had a conversation with Mike whether to keep doing a WG.
There is a desire to keep that going rather than I community
group
... it seems like it is worth considering if there are any
changes we should make so that we aren't doing what we are
doing now. Asking for a lot of charter extensions
... it would be interesting to get to a place where we can
"push button publish"
... being w3C rather than WG has a lot of overhead, but some
people like it
... how do we mock up feature propositions so that we can write
6 and only 2 get implemented so we just move the sections that
don't get implemented
... this might not work, especially for people who write
extension specs
... this would still allow you to do clean up work and rephrase
things without it being a waste of time
ato: I think you covered the
important points. For definitions... I don't care if they are
left in
... Its complicated with a lot of maybe overhead but for the
sortof changes we are going to make now maybe its worth doing
things in a div or invisible class
jgraham: We need some way to mark this up so its automatic
simonstewart: ahh feature toggling using CSS
jgraham: I don't know either, I feel like the choices are probably do something like this and see if it works or give up the idea that we can publish stuff on a different time scale
simonstewart: I think its worth giving it a go,
<clmartin> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
jgraham: a problem with the w3C process is that if you add N features and not everyone implements things in that order it doesn't work
ato: maybe its worth experimenting with, either we have the situation where we manually change the spec, or the css styling works
simonstewart: Some of this stuff lends its self really well to be wrapped in a span or a div
ato: we just have an integer and everytime we increment
jgraham: what is the resolution here? wrap features in tags
RESOLUTION: When adding new features make them togglable
simonstewart: ahhhh... logging
<AutomatedTester> RRSAgent: make minutes
simonstewart: you run a test, the test fails, you want to look at the logs to understand what happened
brrain: what logs
simonstewart: the intermediate nodes, output from geckodriver
jgraham: I'm not super concerned
from getting stdout from geckodriver...
... it probably makes more sense to think about actual requests
from users to get data that they can't otherwise yet
simonstewart: this is rehashing a
discussion we had about... 5 years ago in boston
... there is usually some sort of intermediate node that people
are using
... can I get logs at a certain time, can I get logs from a
certain node. if we can find a way of querying those two axes
then we can get everything. The way selenium did it was you can
query the remote end and ask what log type you want. each
intermediate node adds the logtypes it provides and you give a
request.
clmartin: do we concatenate logs in one master file? Or...
juangj: you request a log type and it will give you all of the logs of that type from that time
simonstewart: there are
facilities in selenium to merge logs
... michael extended this logging to do performance testing for
chromedriver
AutomatedTester: People have asked for that in geckodriver
simonstewart: yea.. that and for monitoring health
junagj: intermediate logging goes elsewhere for us anyways, so not a webdriver thing
jgraham: there are multiple different use cases and I think conflating them to the same case might not be the best way forward
clmartin: if there is no intermediate node you just get it from the server. thats what we should actually spec
simonstewart: we should define some log types
jgraham: perf logging, I don't even know what that means
simonstewart: console logs from js, is that a better example?
ato: well.. there is no spec
simonstewart: all I'm saying is
there is going to be this standard subset of things that people
can use. What does logging look like if you are doing standard
logging entries
... I want the ability to specify log type
brrain: what do you mean
logtype...
... if you are saying where did this come from... I think thats
more useful
simonstewart: log type can be as
in log source
... for example, for the selenium server, there might be, if
you started geckodriver give me the console output from
geckodriver
... you end up with a set of .. a list of node names and log
types
... name and type are the two dimensions
... performance is going to be very browser specific
<simonstewart> Example of Chrome doing performance logging: https://sites.google.com/a/chromium.org/chromedriver/logging/performance-log
AutomatedTester: the performance one, the way the data is formatted and returned, we can't tell chrome "this is how you need to format your perf data", you want memory pressure, you want everything and these all come out of the browser in very browser specific ways. The data will come out in this shape and I think we need to focus on that
simonstewart: standard shape of the log, name, maybe log level.
AutomatedTester: I just want us to not go off into the weeds and talk about performance
Simonstewart: this lets them see post run of webdriver and see whereproblems are
clmartin: if you add a verbose flag you get the output of that
simonstewart: there are things like memeory pressure and where you store them, selenium takes the end
tboyes: this is kinda like a buisness way, the more structure we give it the easier it is in the future
juangj: if you have multiple parties in the logs then its difficult to coordinate them. Different tests wanted to see the logs for firefox to see if there were js errors. We wanted to look at an intermediate node, but we can't have the intermediate node to spy without cleaning state
jgraham: some sort of ring buff
that would have a maximum size
... this should be a single endpoint that returns all the logs
or nothing?
... if there are basic things everyone wants, we could have a
basic way of saying was there was any logging of this type
clmartin: get log type, get named
logs, get logs
... get logs pass in an array of the log types you want and get
logs of all that
jgraham: I don't know the use case for this, its asynchronous too
<simonstewart> For reference, ato found the minutes from the last time we spoke about this: https://www.w3.org/2013/06/13-testing-minutes.html#item02
jgraham: we could have
log/console and that returns a very standard thing
... You can probably represent it ..
... are there multiple use cases here
simonstewart: have a handful of logs types that we can share
juangj: there are some logs that
are for human consumption only, but there are others that are
for machine consumption where the machine makes decisions based
on that
... thomas said the other point that this is part of
intelligence, if there was a good way to do this it is useful
to users
... if there is a use case I don't think its a problem to
have
brrain: are we not entertaining duplex
simonstewart: we have never discussed that in this working group
brrain: some logs just aren't turned on by default
ato: there are certain types of
performance logs that would be quite verbose
... if you have an API that clears say an array buffer then
that is problematic
juangj: what we ended up doing to work around that, if an intermediate node wants to store the log its up to it to maintain it up to that point
jgraham: sounds potentially, you need a way to explicitly turn logging on when you start the session
simonstewart: there is enable a
specific form of logging, and disable logging all
together
... lets say you write a java test and the only thing you are
interested in is is there a log on the console
... you can just say, what type of logs do you have and it will
say ["console"] and it thats your answer
jgraham: if the default is off for everything that forces you to have some coupling,
simonstewart: I think the default position is capture a reasonable amount of logging
jgraham: if passing in console turns off all the other logging you want then that doesnt sound good
tboyles: you might want to make it opt in by default
tbouyles: many patterns I've seen are, you see what is available and then you iterate through that
jimevans: maybe we return a list
of available log types as part of the return of a new
session?
... these are the things you have available to you
simonstewart: what im trying to get at is that the implementation should know what types of logs its capbaple of producer whether there is content or not
whoops thats jimevans
clmartin: I don;t see, we already have capabilities and endpoints, I'm wondering if there is an advantage to having it in capabilities as opposed to an endpoint
ato: interesting point, if you are a tooling vendor and you want to trace logs of geckodriver then you want to have that when the session is starting. I like james's idea but..
jgraham: I'm not sure what problem people are thinking about. For certain log types you want to specifically enable those, for others you don't want that. If you ask for a specific log type and that log type doesn;'t exist
clmartin: I would really default
to having logging off entirely
... they have to explicitly worry about space
JohnJansen: if your looking for the logs its probably because something broke and you didn't expect it ahead of time
simonstewart: after a fatal series of crashes trying to figure out whats wrong and you didn't realize you had to turn logging on
clmartin: is it reasonable to expect people to rerun with the logs
simonstewart: I don't think anyone is saying keep the logs indefinitely
jgraham: I think there is general
agreement that storing logs has to be effecient
... how do you deal with some logs have to be specifically
enabled and some logs are on by default
ato: what sort of things do you want on by default
jgraham: sauce labs might automatically provide the commands available or something
ato: that sounds like a job for
the intermediate node
... so they are allowed to change
<AutomatedTester> RRSAgent: make minutes
simonstewart: there are implementation independant logs that are on by default
juangj: lets say saucelabs wants
to turn on certain logs on by deafult
... your tests wants to be performant and turn off all other
logs, how do you make sure you don't turn off the other
defaults in a bad way
jgraham: these are the log types I want to specifically configure and you get back the log configs for the type
siri: okay I changed the volume
<AutomatedTester> RRSAgent: set logs world-visible
ato: I would make the argument
that that sounds like a complicated API
... I think that passing in the logging config on session
creation is the right thing to do
... I didn't quite understand the problem you were mentioned
jason
juangj: there was a concern that if your tests said give me only console logs, that we might inadvertantly mean you should turn off all the intermediate node logs that you might want later, but not now
jgraham: instead of passing in a
list of logs you want to make or you pass in a list of
paramaters for each log
... there are two types of logs, standaradized log types and
then maybe some intermediary nodes have whatever
ato: why does it need to be configured
jgraham: something might not enable console logging
<AutomatedTester> RRSAgent: make minutes
simonstewart: you might not neccissarily want to capture trace level
ato: waht are the usecases where you want those, I think you mostly just want the fatal logs
jgraham: having all log types is not controversial in the world of loggin
<AutomatedTester> meeting: Browser Tools- and Testing WG meeting, Friday 10 November
simonstewart: I think James;s
point is accurate. There are some logs that are genuinely
configureable and others
... generally when you have alogging API they allow you to
filter logs at a certain level
juangj: I have an intermediary node where I could see us controlling the verbosity of the log
<AutomatedTester> RRSAgent: make minutes
simonstewart: I think what you end up with, if you passign in session capaibilities, you pass in a list of names of log types you want, and then you pass in config options that are log specific and deafult to a specific config
jgraham: for the specific case of
a driver one can imaging that we can standardize something
around the specific semantics around that
... there are certain values that are say standard for console
logs, ...
... there should be two or three things that are controversial
and we can standardize tha
... and there is some stuff that we can't standardize, maybe we
should standardize those similar to extension capabilties
simonstewart: I think we need names of log types
jgraham: if its not defined for common stuff you have to go to every driver and figure out how it does it
ato: the java bindings, console, performance, and you know the defaults of their config, then you have 5 different ENUMs
simonstewart: there has to be some sharing
ato: for geckodriver there is
trace, debug, config, that type of thing
... don't map to java api cause java is missing one of them
clmartin: couldn't we offer per log log level for people who want it.
jgraham: i'm not sure level makes
sesne for a lot
... it feels like it might increase the complexity
simonstewart: i think we are in
agreement it would be nice to have a logging mechanism for
identifying log sources, and then I think there is tentative
agreement which each source needs to be configured in some way,
but there isn't agreement in how that should be
configured.
... lets set this aside for now, cause this is a good place
jgraham: it seems liek there is a reasonable amount of agreement that there should be persistant logging up to some threshold
juangj: if you are looking at js logs, some people will want the 'best effort' for the amount of logs
simonstewart: next step is, we agree there is an endpoint for querying the log sources
jgraham: no, I think you should configure it in a new session and it should return the log types that are availabale
ato: I do agree that that is a
sensible way
... you pass in a difctionary with log type or log source and
when the session comes back it comes with the same difctionary
with either the dictionary in use or false if not
simonstewart: I think each node in the chain would need to tell you what logging its using
jgraham: it seems bizzarre to me that I got a response for a bunch of logs I don't know what they are
ato: we don';t have any great
examples of this, but geckodriver has a vendor specific thing
for checking accessibility checks
... we are passing in the log config, matching on that
simonstewart: I say you only return the log types that are enabled
jgraham: I think its not even true that a given system would know the log types it supports
simonstewart: it is correct
andreas that we should return the cofigure log types and people
should know what those are
... there is a specific subset that is implementing specific
logs that are enabled by default
clmartin: I think we need a log
type endpoint
... what if we want to change something midtest
juangj: its not clear that is even possible
simonstewart: I think something
we do need is the ability to reset storage
... maybe not if you are logging and its affecting the
performance of the browser, and you are measuring the
performance of the browser
juangj: one concern I have about putting the log thing in the capabilities is that it does have to be exact
jgraham: we only do it at the top level
simonstewart: if you as for a specific log type and that log type is not available then you shouldn't be asking for that logtype
juangj: what if I want to turn on some sort of sauce logging. I don't want to have to make a copy of my capabilities for every environment I'm running in
jgraham: so, the way you get
around that in the existing spec is you put each logging config
into a first match and for every test
... I agree there are API designs that could make this less
painful
simonstewart^ not jgraham
jgraham: there are two cases, someone requests a log type thats targeted at my local machine. Ignore it thats sort of irrelevant. What if I want the memory log type or whatever and I don't support memory logging. I think its reasonable to fail the session in that case
simonstewart: if using grid which
scales out automatically to sauce labs when it needs to
... I rescind my initial position and I agree with you,
James
<AutomatedTester> RRSAgent: make minutes
<gitbot> [13webdriver] 15shs96c opened pull request #1145: Remove the configurability of the user prompt behaviour (06level-1...06pr-user-prompt) 02https://github.com/w3c/webdriver/pull/1145
SribeNick: kereliuk
simonstewart: before the break we
came to an unprecedented agreement. Lets bask in that. One of
the requierements from the beginning is to access a particular
log on a particular node for a particular source
... We have yet to decide whether we want a specific endpoint
for configuring logs, or if we want to return it in new session
capabilities
jgraham: I don't think it always
desireable to expose the internal configuration all the
time
... also, it doesn't make sense to me that you could return
more stuff here then you can configure in the first palce
ato: this sign tha tyou have
here, you were mentioning the example of sysout
... if there is a chain of node and they all have stdout
capabilities, then you couldn't enable the standard logging
effort on one of these things
... if these source types are unique but carry the same
name
... another thing I don't understand is what is the use case
for... do we have more log sources on intermediary nodes
... I would have thought that the intermediary chain, if I run
my test on sauce, sauce returns some additional logging service
but I don't care where that come from I just think thats
something that is offered to me which I use
simonstewart: sauce would
introduce themselves not as a chain, but an item in the
list
... thenmaybe the end node would be there too
... maybe the grid node that was being called to selenium
ato: well.. this selenium node offers 3 different types of log sources, I'm not sure
simonstewart: the selenium server has a list of commands comming into it, but it will be capturing stdout and stderr
ato: that could be from the
driver itself
... wouldn't it be a better design to just say this selenium
server offers 3 different logs types but it says nothing about
where they come from
simonstewart: occasionally you do
need the deatil of where things come from
... if I just want stdout, I want the accumulated stdout for
everything
jgraham: that doesn't really make
sense anyways
... its going to be all out of order though,
... what I'm not sure about is... does it makme sesnse to have
like very general catagoriues that you can enable that affect
multiple pieces of the chain
... if you want the sauce stdout you pass in sauce
simonstewart: it gets complicated cause you can end up with a chain like...
*proceeds to draw 8 rectangles with arrows forming a chain in a snake like fashion*
simonstewart: this is not unusual
for a cloud provider
... there are 3 things here, getting logs and often they are
separated by a third party source
jgraham: are there any use cases
for me specifically to say "I don;t want everything with sauce
and I want this specific node"
... makes more sense to think of sauce as a black box
simonstewart: lets say your tests
are running slow and you want to find out why
... you get your logs and you see how long it takes to get
through each node, since you can step through everything you
can see where your bottleneck is
jgraham: I think you can get that
anyway, the question is whether you need the things to be
individually accessible
... If in this example you said hey I want selenium logs it
would give you the first two bits there
... from the point of view of logs you would just say hey I
want the sauce logs
simonstewart: in the new sessions capability, the thing I want is to go to a node on the chain and say "give me the log from you"
jgraham: different services, and
you might have a number of them, would expose their logging in
a way that is specific to that service
... I want the sauce stdout log and it may be an alias of those
3 things
... When you enable it that might make sense for enabling
specific services
... sauce could provide multiple services and
configurations
... it could have a filtering configuration
... if conceptually its one thing
simonstewart: they are separate
jgraham: the main problem I see
with this design is if hypothetically for some reason you say
there is no way to distinguish
... there would be no way to distinguish between two
seleniums
... the configuration that you pass into new session doesn't
know about the 2 nodes
... if you wanted to see each service on the chain and you had
their UUID, you could give it that and the server would be able
to say "hey this is logging for me"
simonstewart: sometimes this is implemented with selenium will start up a random docker container when it needs i
jgraham: its only a problem if
its the same type of thing at two points
... if your solution doesn't work for the configuration
simonstewart: so you would
suggest having well known service names for each of the things
in green (on the board visually)
... so this would have one service name, another would have
another name
jgraham: I think in the end nodes specifically the service name and the other name would be the same
ato: on the top line thats the chain you already know something about so you can make some educated guess about what your doing
simonstewart: I guess that
works
... here, the only problem would be, if sauce labs were using
selenium grid internally is they would have duplicate nodes
jgraham: I think how you would solve that-
simonstewart: oh you would just make it their problem
jgraham: yes
simonstewart: :)
... in selenium server you have something json coming in and
json coming out, but there is no guarentee that they are the
same format
ato: how would you distinguish the nodes you know about
simonstewart: this would have a well known server name, sauce type for in and sauce type for out
ato: the protocol has no
restriction for number of nodes you can have in sequence"
... how would you identify them
simonstewart: you could give the
service an arbitrary name, in selenium we do something
... I think you can order these in a nice way
ato: if we are using the whole colon thing as a namespace how would... lets write some json. I've been working on.. lemme share this
<ato> https://public.etherpad-mozilla.org/p/WebDriverLogging
ato: this is wrong obviously, but
I'm trying to figure out what you need
... *walks up to the projector* I was in a room with 7
engineers once and we couldn't figure out how to do this
simonstewart: this isn't too far
from what I was suggesting
... what I would do is aggregate by service nam
... i think this is conceptually identical
jgraham: I think thats
conceptually nice for nodes , and incredibly ugly , the empty
string is your key rather than...
... it can't be geckodriver, the whole point is this has to be
standardized
simonsteward: the endpoint is what we call the end node in the case
jgraham: I'm not too sure it
is
... i guess it like, doesn't matter if this is abstracted out
in APIs people actually use. The protocol could be arbitrarily
ugly and it wouldn't matter
simonstewart: what would you change?
jgraham: remove one level of nesting and use some string concatenation
ato: I hear you saying words but I don't understand what you mean, what would be different from this
jgraham: a log of cake... is that
a swiss roll?
... the cheese proposal is what I was thinking of
... conceptually I guess the advantage of the other one is that
you can imaging arbitrary nesting. You could collapse them into
one proposal. If selenium wanted to read one specific key and
it had the internal bits be objects after that then that could
work
simonstewart: the cheese proposal is something you are happy with
jgraham: yea
simonstewart: who doesn't love cheese!
jgraham: correct me if I'm wrong, the realistic situation that we should optimize. I am using sauce and sauce has some config, or I'm connecting to browser stack and you know broadly what you are connecting to. I think the case where there is some ineternal topology that you don't know then that is super complicated and there is an information flow problem
ato: there is a specialized case that simon was talking about;. you have a preset environment wheer you have 2 nodes
simonstewart: dealing with an unknown topology, is there a mechanism for querying what is the topology of the nodes
jgraham: my fairly strong
preference is that that is out of scope for the moment
... the way you would have to solve it is there is another way
of dealing with logs where you can't be that specific until
you've started the session
simonstewart: lets go with the
cheese proposal; agreed? cool
... I think you would get back the list of configurable loggers
that are available on your session. by the time this comes back
you will have spun up
... if there is a logger type that sudently becomes available
mid test cause you had some weird test then this is what james
is saying about things being addewd in later
... you can have sensible defaults for each thing
jgraham: we shouldn't requiere that there are logging services that you didn't require in your logging capabilities
ato: would you expect it to be a
best efforts sort of thing
... lets say we have a third node called fish. This fish node
it does some sort of very bizzarre protocol conversion thing.
We have fish:logConversion: true
simonstewart: if you had this and
nothing understood fish:logConversion, then the new session
would not have a fish:logConversion thing in it
... if I asked for fish:logConversion and it wasn't available
whats in the response
jgraham: if something was asked
for and wasn't available then it isn't in the response
... if something fails to configure then we return an error
ato: the node is not required to return
simonstewart: i think you could do that in a different, log types and log sources. You could have log types and log sources and lists
<AutomatedTester> RRSAgent: make minutes
simonstewart: what does the log
type mean, what does the log service mean
... if you want to config loggin for a service on the chain
then you need to read the docs for that
jgraham: we can return
geckodriver: {level: info:} in the response or something
... get logging is a get with a list of service names
RESOLUTION: new session will have a new capability called logging which is a dictionary of servicename:sourcename and the value of that is arbitrary. The return value is the list of loggers servicename:sourcename which are available for your session. Sessions are free to ignore names that they don't identify. If you have servicename:sourcetype and a configuration specified and that confiuguration fails, return an error. And we have an endpoint
to get the list of serivcename:sourcetypes.
simonstewart: we may need an endpoint that clears logs
ato: the existing selenium serice
could clear the log and the existing service could try and get
those logs, so we should make the clear explicit to be more
helpful
... GET /session/{session ID}/logging/console
... DELETE /session/{session id}logging {"console":
"fish:logConversion"}
jgraham: would it make any seense
to have a wildcard thing for match
... for example geckodriver logs, you have lasjd:std and
aoiuhskdjhaskjdha:std in selenium and you want all the standard
out
juangj: this is strictly for convenience right cause you already know all the log types
simonstewart: we can probably thrash out this when we do the reviews
ato: what were the different endpoints again?
jgraham: one for getting the logs of specified types, and one to clear the logs if that is a supported operation. I don;t think failing to clear a log should be fatal
Simonstewart: are we cool with this?
ato: so we only actually need 2 endpoints
jgraham: one edge case. what if I
pass in SE docker code on sauce
... the question is what if I pass in no I don't want a
specific type of log, and that log doesn't actually exist do we
fail the session?
simonstewart: no
jgraham: good
<AutomatedTester> RRSAgent: make minutes
<clmartin> AutomatedTester: juangj ato and I are going to be late. We’re okay with you starting without us. Didn’t realize the In-N-Out was so packed 😅
<AutomatedTester> clmartin: how long are you expecting to be?
<clmartin> AutomatedTester: We are almost back so less than 10 minutes, we opted for Hawaiian instead
<simonstewart> Warm pineapple on pizza? Bletch
<AutomatedTester> heathens
<clmartin> What? No Hawaiian food
<simonstewart> That is ham and pineapple on a pizza
<clmartin> So chicken katsu and kalua pig with Mac salad and rice
<AutomatedTester> pineapple on pizza == heathens
<clmartin> No pizza was consumed
<AutomatedTester> RRSAgent: nolisten
<boazsender> ScribeNick: boazsender
<ato> Scribe: ato
simonstewart: Switching into browsing context, this should be in the spec.
jgraham: That’s not really possible to spec.
simonstewart: There’s at least
three different driver implementations that do that.
... Android, Firefox, ???
jgraham: We’re not writing an Android spec.
simonstewart: It would be useful to have a gateway to leave the WebDriver world.
jgraham: geckodriver can switch
contexts into chrome.
... It was probably a mistake to expose that.
... If it turns out that there are some like prior art that
fits a really specific pattern, then maybe it should be in the
spec.
... You specify the mechanism for switching into this void.
simonstewart: People want to be
able to get into the chrome.
... You’re in WebVR, you want to go into a browsing context,
into another WebVR contxt.
jgraham: You could have an
endpoint that was reserved for doing this thing.
... But I don’t see the value of this.
simonstewart: You get context handles in Appium.
jgraham: We also can’t test a feature like this.
boazsender: Permissions wants to interact with the browser UI.
<kereliuk> ScribeNick: ato
brrian: In Safari we ship a
prototype of a permissions endpoint.
... I don’t care how this would go into the spec.
... I don’t have a demo, but I have a draft blog post.
simonstewart: Lots of these
things share the same why.
... Geolocation, WebBluetooth (cancel/select
device/accept/close), camera (accept/deny), microphone
(accept/deny).
boazsender: Popups, sync
jgraham: I’m not sure this is so
uniform.
... Geolocation might allow you to enter an address you want to
be seen at.
... Thinking about the UI level is differnet.
... This is not about yes/no, but selecting a feature set or a
specific input source as well.
brrian: End of day, user
media
... Every browser has a different set of policies.
... Driven by browser product stuff.
... Wary of enumerating all these things that will
change.
... We prototyped something proactive.
... Nothing will block.
simonstewart: Everything else in
the spec has been reactive.
... Alerts.
brrian: I went for a proactive approach because I want to avoid browser type problems.
JohnJansen: Is it a capability?
<brrian> https://gist.github.com/burg/17a6f362fedfe6ee91354d197b415736
JohnJansen: You want to test that the dialogue pops up and that it has the right string.
jgraham: I don’t think people testing here cares about whether the alert pops up or not.
JohnJansen: You have to know at least once.
brrian: WebRTC folks care to emulate what a user does.
simonstewart: You go to one site,
allow it, then another site and you want to disallow it.
... How would you write that?
brrian: This is get/set
permissions at runtime.
... I’m going to allow it when it happens, and this is global
for the session.
... You set a flag on the session.
... Am I simulating?
... Make a choice based on this policy.
... You don’t interact with the popup directly.
simonstewart: The spec is mostly
reactive.
... You switch to an alert, you switch to a window.
jgraham: The UI implementation is irrelevant.
brrian: That was the independent conclusion as well.
jgraham: I think it helps to
think in terms of a DOM API making a permissions reques.t
... A browser must find out a permissions request so it can pop
up a UI.
AutomatedTester: We could make it
work per domain.
... It would solve simonstewart’s use case.
clmartin: We do different domain
logging
... We seem to be doing different things for different
features.
jgraham: Per domain is not the
right granularity.
... It is possible to grant a permission request once.
brrian: That is a policy per
browser.
... That is specific to browsers.
jgraham: Baking in assumptions that browsers currently make is not going to work in the long run.
clmartin: Did you say that the
WebRTC WG were looking for an on/off switch?
... They weren’t looking for a per-domain thing.
soareschen: Currently our main
target is WPT and interop testing.
... Our main concern is to shut down the notification.
... So it does not ask for permissions.
... Always allow and always deny would solve our current use
case.
... There might be other use cases that you might need to
consider.
brrian: Safari is written in such a way that the only real way we can do this is at the request layer.
jgraham: Maybe we get to hear what the Bocoup model looks like.
boazsender: There is a PR.
<boazsender> https://github.com/w3c/permissions/pull/151
<boazsender> https://docs.google.com/document/d/1Oe4VhgdFnZ6ID3WGyG97n_b1khvYsRcX7T4ddNcyJ9A/edit#heading=h.xws79928vxeg
boazsender: My colleague Mike has
documented three approaches.
... Passive, reactive, proactive
jgraham: The passive one sounds a lot like the one brrian was talking about.
boazsender: Proactive, being
directly setting the state.
... Reactive being interacting with the UI that the browser
presents to you.
simonstewart: Reactive is best.
jgraham: What will a reactive UI look like.
simonstewart: Switch to
permission, or grant permission X
... Yes, no, whatever
... We don’t care how a user prompt is presented.
jgraham: Permission prompts can come up at any time.
ato: So an error on the next command?
simonstewart: Yes, like the user prompt handler.
clmartin: I’d prefer to avoid something that goes down the route that is browser specific.
simonstewart: I’m trying to come
up with an API that is good for test author.
... It fits the mental model that the rest of the API uses.
jgraham: The WebDriver model is a poor fit.
simonstewart: Do you want to have
a bidirectional protocol, where the remote end can start
sending messages back?
... You would have a listener and send back an event.
tboyles: This comes up a lot.
jgraham: I don’t have any
objections to that.
... But we should not block this feature on that.
brrian: If you don’t understand how getUserMedia works you will have a hard time anyway.
jgraham: I think internally for
Firefox, you set a pref to say “always grant this
protocol”
... I think it would be easy to implement on top of the
prefs.
brrian: In the prototype some of
these things are capabilities.
... You can’t do WebRTC over HTTP
... I don’t want to gate your testing on having a reasonable
test serve.r
simonstewart: How would you do
WebBluetooth?
... Or just pick a camera and a microphone.
jgraham: I would assume there is one fake input source.
brrian: From a security PoV we could not enumerate attached devices.
jgraham: [leaves to talk about WebVR]
boazsender: Could we standardise the options? Could that ever be standardised?
<simonstewart> https://webbluetoothcg.github.io/web-bluetooth/#permission-api-integration
brrian: There is a question whether it should be part of the permissions spec or the webdriver spec
<AutomatedTester> https://w3c.github.io/permissions/#status-of-a-permission
simonstewart: We could model ourselves after the permissions API
<simonstewart> https://developer.mozilla.org/en-US/docs/Web/API/Permissions
AutomatedTester: In the spec, there is a permission state that we then, could, like, and then it talks about camera, microphone, that could be extendable in modes.
simonstewart: Maybe we model
ourselves after the permissions spec I linked to.
... The actual JS API.
... I don’t know how geolocation works
<simonstewart> https://w3c.github.io/permissions/#permissions-interface
brrian: We’re talking about hooking up a device.
<simonstewart> https://w3c.github.io/permissions/#permission-registry
brrian: If you’re not allowed to
use this device but this other one, then that would make the
command more complicated.
... But if we’re only using mock devices, and you could not
switch, then it would be easier.
simonstewart: We could ask the permissions API to put a hook into WebDriver, so if your browser implements the permissions API you would get these new WebDriver commands.
clmartin: Yes, we talked about this yesterday.
AutomatedTester: We would be technical reviewers of their spec before they would go to rec.
simonstewart: My suggestion would
be to go to the permissions WG and ask them to add it as an
extension of WebDriver.
... They can ask us for advice, but we don’t know all the
details.
... We could use brrian’s example as a strawman.
<simonstewart> https://github.com/w3c/permissions/issues
<scribe> ACTION: On ↑ file an issue to provide a webdriver endpoint
<scribe> ACTION: Start a discussion based on brrian’s proposal for that bug
<juangj> Scribe: juangj
<AutomatedTester> RRSAgent: make minutes
simonstewart: there are all kinds of auth mechanisms but typically what people are asking about is a basic user/password dialog
<simonstewart> https://gist.github.com/shs96c/10d24dd2e51651f12a8f2785934b98bc
simonstewart: (waits for the
internet)
... typical test (see above) switches to the auth dialog that
you expect to be there, and then plugs in credentials
... 99.9% of sites use user/pass but the Credentials object is
supposed to know how to serialize themselves in case you need
other stuff
jimevans: microsoft screwed us on this; the auth dialog is no longer automatable. it's not a typical dialog and you can't inspect it
simonstewart: you can see an
example of this dialog if you try to do something on the w3c
website requiring login
... user/pass would make almost everybody happy, but i would
like it for this to be extensible for other auth mechanisms
JohnJansen: haven't seen this as a focus myself because internally, if we ever hit this, it's a 2-factor auth prompt so it's not really automatable
jimevans: there was a demo of this at SeConf!
simonstewart: (describes the rube goldberg automation that was demoed)
<jimevans> https://the-internet.herokuapp.com/basic_auth
simonstewart: endpoint named
"/authenticate" probably; GET would say what fields are
required, POST would send the credentials
... model it after dialog confirm
<scribe> ACTION: simonstewart to spec this out, requiring user/pass for basic and digest auth, and leaving open questions of how to provide other fields for more complex auth
AutomatedTester: looking at the agenda, the remaining item is Presentation API, but the person who knows about that has headed to the airport
ato: if we follow what we decided re: Permissions, then we should discuss with them (rather than on our own)
<kereliuk> https://www.w3.org/wiki/WebDriver/2017-TPAC-F2F
AutomatedTester: we'll follow up
with them, tell them what we suggested for Permissions and help
guide them
... other topics?
tboyles: we had somebody asking
about asynchronous new session creation. has this been proposed
before?
... you'd send a request to create a new session and it would
return immediately. then you'd poll it to see when the session
was created
ato: that would be different from
the rest of the existing API, in which all commands are
synchronous
... it's not a bad idea and perhaps the API should have been
this way
tboyles: it's true this may be a little self-serving because of our (sauce's) use case
simonstewart: back in the day,
New Session used to use with post-redirect-get
... POST response would redirect to an endpoint that returns
the capabilities of the current session
kereliuk: that's already implemented in chromedriver, but it's not in the spec
<simonstewart> https://en.wikipedia.org/wiki/Post/Redirect/Get
ato: then it should be in the spec. if you're doing something different the second time an endpoint is hit, then that's not conformant
kereliuk: ... i'm not sure where exactly this is happening in chromedriver. let me check
simonstewart: do we want to use something other than HTTP/JSON as the protocol?
ato: that would be nice. but with
the CDP discussions going on, the right thing to do is probably
wait for that to evolve
... WebDriver seems to fit right now in a spot that's one level
above what CDP is concerned with. the target audience is
different
... two-way communication would be nice, as we've discussed
before, but for the short- to medium-term we have so many
features on our plate that it's not realistic to tackle this.
by the time we do have the time, we will have more clarity re:
CDP
AutomatedTester: agreed this
would be nice. i know simonstewart has some concerns about
memory pressure this may create on intermediary nodes
... best to do the things we've talked about today and
reconvene in, say, 6 months over email or whenever to discuss
again
ato: i sense we will be
discussing this in Lyon at next year's TPAC
... let's say that CDP becomes something and is implemented in
different browsers. i think what that means for this WG is that
CDP offers the right sort of services to allow something like
WebDriver to operate on top of it
<AutomatedTester> https://github.com/WICG/devtools-protocol
<scribe> ACTION: simonstewart to add GET /session endpoint
<simonstewart> Thans
<simonstewart> Thanks, even
ato: (discussing webdriver as a
web IDL)
... questions of how to translate http semantics like
get/post/delete into this api
simonstewart: postMessage
ato: you still need to define a serialization for this
simonstewart: legacy firefoxdriver had done something like this (is looking for the code)
ato: this all would seem to require a lot of plumbing
AutomatedTester: concern about
security implications of making this a web api
... it would have to only be enabled when automation is
explicitly enabled
ato: two use cases here: 1. accessing the api from web content, 2. accessing webdriver primitives from the context of dev tools
<simonstewart> https://github.com/SeleniumHQ/selenium/blob/master/javascript/firefox-driver/extension/content/dommessenger.js
ato: was this using the js atoms in the background?
(some musing about how this could be implemented)
ato: don't actually think this is seriously worth discussing though
kereliuk: clarifying previous post/redirect/get -- i don't think we actually do the redirect. we just have the GET endpoint
<AutomatedTester> RRSAgent: make minutes
ato: this is the most peaceful meeting we've had
AutomatedTester: visibility
<AutomatedTester> RRSAgent: make minutes
<AutomatedTester> RRSAgent: publish
<AutomatedTester> RRSAgent: publish minutes
This is scribe.perl Revision: 1.152 of Date: 2017/02/06 11:04:15 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Succeeded: s/drop testing/ interop testing/ Succeeded: s/aproaches/approaches/ Present: AutomatedTester jgraham simonstewart clmartin ato brrian juangj jimevans JohnJansen titusfortner tboyles kereliuk Found ScribeNick: kereliuk Found ScribeNick: boazsender Found Scribe: ato Inferring ScribeNick: ato WARNING: No scribe lines found matching previous ScribeNick pattern: <boazsender> ... Found ScribeNick: ato Found Scribe: juangj Inferring ScribeNick: juangj Scribes: ato, juangj ScribeNicks: kereliuk, boazsender, ato, juangj WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 WARNING: No person found for ACTION item: on ↑ file an issue to provide a webdriver endpoint People with action items: simonstewart start WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option. WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]