Browser Testing & Tools WG, F2F, TPAC 2014 -- 30 Oct 2014

<lukeis> #34

<selbot2> 3

<selbot2> simon.m.stewart open/AcceptedSupport BASIC and Digest HTTP authentication - https://code.google.com/p/selenium/issues/detail?id=34 [Type-Enhancement Priority-Medium Component-WebDriver ]

<wilhelm> scribe: mdas_

<jgraham> s/.*//

<mdas> scribe: mdas_

<mdas> AutomatedTester: The current state of the spec is turning it into a spec instead of a story

<mdas> AutomatedTester: we're not missing many API calls, just a trivial handful, and we have action items from previous meetings we need to verify as done

<mdas> wilhelm: We have 85 open issues, which should we discuss?

<mdas> wilhelm: any that aren't listed?

<simonstewart> https://www.w3.org/Bugs/Public/show_bug.cgi?id=20860

<mdas> simonstewart: some bugs are level 2 and don't need to be addressed

<mdas> jgraham: if you try to implement the current spec, what you send over the wire is unclear

<mdas> jgraham: I think there's work needed there to pin down what should be sent over the wire

<gitbot> [3web-platform-tests] 5Ms2ger opened pull request #1326: Extend the Node#cloneNode test. (6master...6clone-prefix)https://github.com/w3c/web-platform-tests/pull/1326

<mdas> ato: we also need to figure out some details of the schema and how to define these things, we previously concluded that webIDL was not appropriate

<mdas> ato: we need to figure out what is appropriate that is less cumbersome

<mdas> sam_u: We need to discuss screenshots

<ato> sam_u: About the agenda topic Data Model: http://lists.w3.org/Archives/Public/public-browser-tools-testing/2014OctDec/0021.html

<wilhelm> Scribe: mdaas

<mdas> Scribe: mdas

mdas: we need to clarify some parts of the touch spec

<gitbot> [3web-platform-tests] 5Ms2ger opened pull request #1327: Extend the Element#tagName test. (6master...6tagName)https://github.com/w3c/web-platform-tests/pull/1327

auchenberg discussing developer tools

<wilhelm> http://www.w3.org/2011/08/browser-testing-charter.html

<ato> lukeis1: https://twitter.com/sarah11918/status/526827265813123074

wilhelm: http://www.w3.org/2011/08/browser-testing-charter.html chapter 1-2 says we can explore overlapping interests in other WGs

auchenberg: a few years ago we started RemoteDebug, which unifies interfaces to browsers
... fixing the gap between the editor and the browser, and remote debug tool bridges that gap. Chrome has their own, Opera has their own, Mozilla has their own...
... we want to achieve a unified interface for browsers
... we wanted to use the existing chrome remote debug protocol to start standardizing on
... status of remote debug: Node Inspector is being reused across platforms, and Microsoft demoed remote debugging
... editors are adding devtools, and discusses other implementors
... what's next? start using remote debugging tools and map out the differences between them before unification
... is there room for this concern in this WG?

AutomatedTester: I really like is how we communicate from teh local to remote end using HTTP endpoint, Mozilla's implementation is based off remote debugger, so that level of communication would be great if standardized
... the other parts of what WebDriver does seems out of scope for remotedebugger

jgraham: does anyone work on browser devtools?

JohnJansen is the only one to raise his hand :)

<gitbot> [3web-platform-tests] 5dannin opened pull request #1328: Add blackberry driver (6master...6master)https://github.com/w3c/web-platform-tests/pull/1328

jgraham: we don't have many people who work on it so this may not be the right group
... people may not be interested in standardizing something they don't know about/charter does not cover

auchenberg: we want to iterate on two separate issues: protocol and the API. tooling vendors are interested in using remote debugging but roll their own thing. They don't have the time to implement one protocol for each browser and don't have control over how the browser changes so they have to play catch up
... maybe we can build in some stability into the protocol to enable browser agility (so they can make changes), but we need to get the conversation started

<ato> jgraham: You said exactly what I was going to say (-:

jgraham: I think devtools has been an area where browsers competed against each other. You need to make the case that interoperability is more valuable.

discussion about competing browsers' devtools to compel users to use the browser

AutomatedTester: we should unify on the communication across browsers.

auchenberg: yes, agreed. How people innovate on top of that is up to them
... is there room in this charter for this concern?

simonstewart: start a community group?

auchenberg: can it be joined under this charter or is it out of scope?

simonstewart and ato: first you need to get a community and find a shared goal to unify and evolve from there

AutomatedTester recommends writing a strawman and discusses its benefits

new process discussion

MikeSmith: Trying to spec publishing better, for webdriver, we can adopt the new process if we want

MikeSmith explains a very hand-wavy old process and hand-wavy new process

MikeSmith mentions 'agile'

wilhelm: the biggest issue is disconnect between TR and the Editor's draft
... how can we move from here? kill the TR bit or put Editor's draft there?

MikeSmith: we have flexibility there, I will figure out what we have to do but we should be able to change it to reflect what we want them to implement now
... updating TR takes too long and would be out of date, so we tell people not to read the TR. we want to change this so implementors won't be confused

JohnJansen: the TR will still exist with the new process but it will be easy

quick vote to see who is for/against

everyone is in favour

resolution: adopt new process

Discuss moving decisions to mailing list

AutomatedTester: we should move discussion to mailing list since we only meet every 6 months or so
... we would get more engagement from community

seva: do we need a voting process in the list?

jgraham: the way it 'works' in other WGs is if people seem to be reaching agreement, you don't need a vote, you reach consensus

simonstewart: we can try it!

AutomatedTester: it would be valuable to get more community members' voices heard outside of bugs

lukeis: no response is tacit agreement?

<MikeSmith> +1 to not voting and to lazy consensus and to, if/when more formal decisions are needed, moving such decision-making to mailing list

JohnJansen: yes would work that way

resolution: we will use mailing list

<wilhelm> http://lists.w3.org/Archives/Public/public-browser-tools-testing/

data model

<ato> Context: https://www.w3.org/Bugs/Public/show_bug.cgi?id=26147

jgraham: the context is that I've been trying to implement the current spec to make a proxy between the protocol and marionette (mozilla webdriver implementation)

<ato> Uh, wrong link.

jgraham: at the moment, the spec is not implementable

<ato> http://lists.w3.org/Archives/Public/public-browser-tools-testing/2014OctDec/0021.html

jgraham: there are a lot of things missing, and some things that are there don't make sense
... it should be a goal of this meeting to really nail this stuff down

ato: it's also very difficult to write down the algorithms without the data model in place
... all the current command algorithms don't say anything of the return/response values, http response codes, which headers should be sent, etc...
... we can start working from http://lists.w3.org/Archives/Public/public-browser-tools-testing/2014OctDec/0021.html

jgraham: I don't think we need to decide spec text, but we need to decide how we want to represent our spec (ie: what to use instead of webIDL)
... how about section 2?

<simonstewart> https://dvcs.w3.org/hg/webdriver/raw-file/default/webdriver-spec.html#commands-and-responses

jgraham: webIDL is used here, but we're sending HTTP requests with json in the body and the webDIL to json is undefined
... the spec should be clear that you don't send body with GET requests

<scribe> ACTION: clarify that you don't send body with GET requests [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action01]

jgraham: name and session ID for reqs that take body is pointless since they are in the path and you can't send it in GET, so we should pare it down just to parameters

<seva> http://www.w3.org/2014/07/07-testing-minutes.html

<ato> JohnJansen: http://sny.no/bttmin

<wilhelm> https://www.w3.org/wiki/WebDriver#Minutes_from_meetings

<scribe> ACTION: make the body of the requests just the parameters object and nothing else [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action02]

jgraham: empty json object if no parameters?

simonstewart: yes

<scribe> ACTION: clarify how to send no parameters [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action03]

simonstewart: how about if we want to add implicit parameters?

jgraham: doesn't matter for GET

simonstewart: perhaps using headers or querystring then
... path matching and query string

ato: there is a limit to the length you can send over the wire
... we can't refer to url spec since there may be intermediaries that don't respect the spec

simonstewart: that's a problem for those intermediaries if they don't follow spec

seva: the advantage of shoving parameters in querystring is that it works with GET?

simonstewart: yes.

jgraham: there is asymmetry between req and resp, since response has no query string

MarcFisher: we should move to POST, why have them split up?

simonstewart: we use the different kinds of reqs so we can redirect them appropriately
... our API is REST-ish, and the req types adds clarity

jgraham: I see both sides of this, superficially, thes pec looks nice and looks RESTful, but not quite

simonstewart: that's why it's REST-ish

jgraham: it's easier to implement if everything is a POST and has a body, but I also agree that it looks like a nicer API as REST-ish

simonstewart: I want to avoid breaking all existing implementations. Changing to POST is a massive breaking change

ato: also I think if you examine existing implementations, most of them are utilizing the JSON object, so I'm not sure if it will be such a performance problem not to do only POSTS

simonstewart: the command object allows you to specify any fields in the dictionary, how can we preserve the capability of adding additional metadata? We used the HTTP verbs and json blobs

ato: can we send params through headers?

jgraham: if your'e doing this, sending through headers feels HTTPish but feels like a terrible idea

simonstewart: we can leave additional data and metadata to headers to Level 2

ato: we should specify if the intermediary can change the data it's passing

MarcFisher: say IE doesn't support xpath, we can build an intermediate node which would see XPATH and translate it to somethine IE could understand
... so there is value in changing the data

simonstewart: proxy/shims do this

jgraham: I don't think you have to say anything, the proxy implements both sides of the spec (local and remote ends)

MarcFisher: yeah we shouldn't really address intermediate nodes in the spec

jgraham: agree

simonstewart: I'm happy to take that language out

ato: we mentioned it first just to show it is possible

jgraham: i think it's fine to say it exists, but we don't need to define it

ato: we have a previous decision to add a section of uses of webdriver, so we should add it there

<scribe> ACTION: ato to add intermediate nodes as an example use case [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action04]

ato: how do we standardize on json presentation?

jgraham: there are multiple 'standards'

ato: TC39? ecmascript one?

<scribe> ACTION: ato ask Anne if TC39 should be used [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action05]

jgraham: are we assuming the path in the request is an absolute path, or is it just random prefixes?

AutomatedTester: I remember raising a bug, you can put anything in front if it, and hope the vendor supports it

simonstewart: you can host it on any server you have, it's relative to the base path of the server used

jgraham: for every connection, you need to know the base path

MarcFisher: yes that's always going to be the case, you need to know the address and port your'e connecting to. Adding more segments to that seem trivial

ato: selenium,chrome and maybe firefox uses different urls

jgraham: by hosting webdriver url on a host that has other endpoints, it seems worrying

simonstewart: ultimately it's a webapp that controls the browser
... it's up to you where you want to host that webapp

jgraham: we don't want to encourage incorrect setups

simonstewart: we don't want to prevent users from setting it up the way they want

jgraham: that's fine, it should just be clear in the spec

<scribe> ACTION: add recommendations in the spec about per server path prefix [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action06]

responses

ato: AutomatedTester proposed return value from new session command will have value set to dictionary with SessionId and Capabilities

AutomatedTester: should we get rid of status?

jgraham: no status makes sense
... can have more more info in status field than status code

<simonstewart> https://code.google.com/p/selenium/wiki/JsonWireProtocol#Response_Status_Codes

simonstewart: json wire protocol (https://code.google.com/p/selenium/wiki/JsonWireProtocol#Response_Status_Codes), there is an attempt to use HTTP status code the way it should be used

ato: I don't think this is good

discussion about status field vs status code

simonstewart: not finding an element is interesting. One status code, but two different status fields, either "I didn't find an element" or "malfolmed selector"
... status code is used to know success/fail, and field is used for throwing to user

jimevans: the value for failed commands if defined such that the the 'value' value of the object should contain the message

simonstewart: are we saying that we always return 200 for success, and use codes for problem, and we get rid of the value field and use the body

MarcFisher: two mandatory fields: type and message

ato: any non 200 code should be error?

simonstewart: 302 is redirect

jgraham: there's HTTP level stuff and webdriver level stuff on top
... at the http level, if you get a redirect, then you should follow the reidrect and make a new req
... you should do what http says to do

simonstewart: new session used to redirect to new sessionid
... we simplified that

ato: so we should have status codes for each status defined in the spec?

MarcFisher: we should define what the local end should do with that status code

simonstewart: I think what JSON wire protocol does is enough

jgraham: for each error, we define the status code and on receiving that code we shoudl define how it should be handled

simonstewart: I think we can just say this in a preamble, not for each
... like, 'oh i got a 501' well you should get an error code with the message. if you don't, do some default

jgraham: I think I agree
... in section 2.4 there's a list, each item in taht list should ahve an http status code to it

simonstewart: I think it should be 501 for almost all of them

<erabonza> http status codes: http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6

jgraham: if found elements are in the url, then we send 404 for unfound elements

<Yves> nah, it's at http://tools.ietf.org/html/rfc7231#section-6

AutomatedTester: what about stal eelements?

wilhelm: It's error code 410

jgraham: right, so we need to decide what the codes are for each error

simonstewart: yes, and if we get a non 200/300 code, we consider an error

<scribe> ACTION: assign status code to each error [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action07]

<Yves> and of course https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

<scribe> ACTION: explicitly specify that webdriver implementation is expected to own all paths under its prefix [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action08]

ato: how do you handle a paramater that is wrong? a wrong type for example?

simonstewart: server error?

MarcFisher: we have error code for illegal argument

ato: we shoudl generalize so all command objects should be handled with these pre-steps, then algorithm, then serialization definition.

jgraham: yes, if a remote end gets a request, it should run these steps, if there's an error, produce X response, otherwise handle response and serialize it and push out

<scribe> ACTION: ato to define the pre-steps, error handling, algorithm and serialization definition for all cases in general [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action09]

headers

simonstewart: we should do cache control ones

jgraham: agreed. Did we agree that everything int eh spec should be under cache-control or whatever the header is?

simonstewart: yes.

<Yves> if you need http-related stuff reviewed, please ask me

ato: what about metadata headers? if a proxy modifies a value

jgraham: I think we will leave that for now
... we need to define what *has* to be sent in the headers
... and let additional headers be sent

ato: which headers do we want to enforce?

jgraham: content-type, cache-control

<Yves> perfect!

<scribe> ACTION: specify which headers we expect: cache-control and content-type [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action10]

resolution: remove response body and just send back dictionary

ato: what do we do to send back a primitive?

simonstewart: use 'value' key

resolution: use 'value' key for primitives

simonstewart: are we happy with the way we encode elements?
... we currently use ELEMENT
... how to differentiate a webelement or a dictionary of elements?

jimevans: in previous version of spec, 'element' was mapped to id, which was problematic

ato: seems liek circular problem, not sure how to guard against this

jgraham: the right way is to when you createa a session, it sends back the key you have to use for the element

simonstewart: why not use the same guid?
... so you're saying "here's a random string", why don't we generate that guid now, and be done with it?

jgraham: that works

simonstewart: are we happy with 'ELEMENT' being that key?

jgraham: that's nto a guid, it's human readable string
... using a guid means no human generated key will try to use that key

<jgraham> 7bf8701e-6066-11e4-a52e-4f735466cecf

simonstewart: but we can pick any 36 characters we want
... why not use human readable one if it has equal chance of being generated?

jgraham: entropy affects it
... there is no advantage to having human readable strings
... I have been manually typing this, i would happily deal with the fact that it's a 36char string

seva: how about some human readable part?

<ato> ELEMENT-7bf8701e-6066-11e4-a52e-4f735466cecf

<jgraham> I think the conclusion was element-6066-11e4-a52e-4f735466cecf

resolution: use element-6066-11e4-a52e-4f735466cecf instead of 'ELEMENT'

<JohnJansen> agree

now everyone is happy.

<gitbot> [3web-platform-tests] 5Ms2ger opened pull request #1329: Add some attributes tests. (6master...6attributes)https://github.com/w3c/web-platform-tests/pull/1329

<MikeSmith> Ms2ger, 'Attr.ownerElement' is deprecated and has been removed from DOM4 (http://w3.org/tr/dom).

<Ms2ger> MikeSmith, lolwat

<MikeSmith> heh

<Ms2ger> I guess it well might still be removed from tr/dom

<Ms2ger> MikeSmith, a thought for future f2f meetings... Put "review wpt PRs" on the agenda

<MikeSmith> we still can do that

<MikeSmith> for the webdriver meeting, you mean?

<MikeSmith> oh

<MikeSmith> yeah, should have done for webapps I guess

<Ms2ger> Well, I want people to review my tests :)

<MikeSmith> yeah

<Ms2ger> Not sure if this room is particularly well-suited for that

<Ms2ger> Excluding present company, of course

<MikeSmith> well, on that note, for that "Non-HTML element with upper-case attribute" test it would be nice to know which of those assertions is/are failing

<MikeSmith> is it odd for me to expect that?

<Ms2ger> MikeSmith, no, you're right

<MikeSmith> also I'm surprised how many of these tests fail in webkit

<Ms2ger> Ha

<Ms2ger> Well, Servo crashes, so there's that

<MikeSmith> and wondering why webkit fails 8 of them and blink only fails 4

<MikeSmith> heh

<MikeSmith> sadly for me it just times out

<MikeSmith> ERROR:js::rust: Error at http://web-platform.test:8000/resources/testharness.js:1842: TypeError: w is undefined

<Ms2ger> Yeah, you need our th.js fork

<MikeSmith> but I'm running a build from probably two weeks ago or longer

<MikeSmith> ah OK

<jgraham> Ms2ger: I think we have tried putting testing on the agenda before and basically the room emptied

<MikeSmith> yeah I seem to remember that phenomenon as well

<jgraham> Turns out that there are lots of people who like having "deep thoughts" but don't like doing actual work

<Ms2ger> Ha

<jgraham> At least during f2f meetings

<Ms2ger> MikeSmith, your assertion messages have arrived :)

<MikeSmith> Ms2ger: thanks will review it on critic now

<MikeSmith> btw do you guys do the thing of pulling the pr/NNNN branches?

<Ms2ger> Occasionally

<MikeSmith> OK I'm just wondering what the proper way to do it with get when fetching the subsequent commits

<Ms2ger> git fetch origin; git merge --ff-only origin/pr/foo?

<MikeSmith> Ms2ger: yeah that's basically what I do

<gitbot> [3web-platform-tests] 5sideshowbarker closed pull request #1329: Add some attributes tests. (6master...6attributes)https://github.com/w3c/web-platform-tests/pull/1329

<gitbot> [3web-platform-tests] 5sideshowbarker pushed3 new commits to 6masterhttps://github.com/w3c/web-platform-tests/compare/bea430c61dc7...710be6f97d3e

<gitbot> 3web-platform-tests/6master 4887cb97 5Ms2ger: Add some attributes tests.

<gitbot> 3web-platform-tests/6master 4d4569c7 5Ms2ger: Add messages.

<gitbot> 3web-platform-tests/6master 4710be6f 5Michael[tm] Smith: Merge pull request #1329 from Ms2ger/attributes...

<Ms2ger> MikeSmith, so, while I have you around... ;)

<astearns> Ms2ger: give him a few minutes - he just set his laptop down and closed his eyes for a bit

<Ms2ger> Ah

<Ms2ger> astearns, in that case, want to do some reviews?

<Ms2ger> Dammit astearns!

<jgraham> Ms2ger: Apparently everyone was out partying last night, so trying to get useful work out of them today might be hard

<Ms2ger> I assume not the kind of partying Hixie would do?

<Ms2ger> (As in, drink water and blog about HTML parsing)

<jgraham> I don't think it was that kind of party, sadly

<gitbot> [3web-platform-tests] 5mbrubeck closed pull request #1319: fix pointer types detection placement (6master...6submission/Microsoft/PointerEvents-FixPointerTypesDetectionPlacement)https://github.com/w3c/web-platform-tests/pull/1319

<gitbot> [3web-platform-tests] 5mbrubeck pushed8 new commits to 6masterhttps://github.com/w3c/web-platform-tests/compare/710be6f97d3e...f9c7b19be305

<gitbot> 3web-platform-tests/6master 45534d32 5Evgeny Agafonchikov: Fix placement of detected_pointertypes update...

<gitbot> 3web-platform-tests/6master 4d53e544 5Evgeny Agafonchikov: Fix placement of detected_pointertypes update...

<gitbot> 3web-platform-tests/6master 4b6ab2da 5Evgeny Agafonchikov: Add missing code for detecting of pointer types

<wilhelm> Scribe: seva

<scribe> scribe: seva

how to accelerate specking progress

simonstewart: proposes to try to set aside dedicated time for spec work

AutomatedTester: can easier justify dedicated time lately better than before
... should be easy to do, at least before the end of Q1 2015

everyone looks happy

simonstewart: there are very few points of contention left

<scribe> ACTION: Simon and David to setup dedicated time [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action11]

Having capabilities be required when passed i

AutomatedTester: there's been a lot of cases when desired capabilities passed are in fact required

simonstewart: no, they are often in fact desired. you don't get everything you desire for. sometimes results are unexpected for the users, yes
... desired capabilities were designed when .. it was actually fine for the user to get something similar to what they asked for
... there's still a need for desired: I want Firefox 33. but there's only 34 available. 34 is fine

AutomatedTester: making one extra trip to the remote end to confirm what was asked for earlier looks wrong

simonstewart: no, that's fine. highly concurrent systems do that
... before there were not required capabilities, just desired
... one doesn't need to use desired capabilities. if everything is required, use required

<MikeSmith> Ms2ger: was sleeping and eating. you were pinging me about a review? Or Denis is doing it?

<denis> MikeSmith, I'll take care of https://critic.hoppipolla.co.uk/r/3022

ato: David started the discussion in the mailing list, but only a few people took part

<MikeSmith> de

simonstewart: i want to be able to checj out 6 month old tests that hard code Firefox version 30 and run the tests
... people sometimes overpopulate the caoabilities; use both desired and required browserName. and that works because of the fuzzy matching

ato: getting back Firefox when you asked for a different browser - seems unnatural

simonstewart: then require Firefox
... you can also ask for either Firefox or Crome

Chrome

simonstewart: we should first match on browserName. then on platform. then maybe browserVersion

lukeis: with Grid, there's resource allocation aspect. you need to know/fix the platform first ?

simonstewart: a shim (chromedriver, iedriverserver, etc) can know what version of the browser is installed without starting the browser
... other than Java, language bindings don't have required caoabilities - just desired.
... people could check what they get after they got the session started; they don't

lukeis: discussing some aspects of language bindings and disired/required capabilities...

<scribe> scribe: both desire and required caps stay. language bindings will add features to make those easier to use..

<scribe> scribe: seva

resoution: both desire and required caps stay. language bindings will add features to make those easier to use..

AutomatedTester: if we keep both I would like proxy capability to always be required

MarkFisher: actual capabilities that are returned by the driver should return proxy if it's set

lukeis: proxy is not returned, in fact, in the oss implementation

MarkFisher: it should, according to the spec, right?

MarcFisher: it should, according to the spec, right?
... there can be multiple ways to match a subset of a set of desired capabilities
... the best way to specify what the driver should prefer - not specify that

simonstewart: disagrees. says we can mandate certain preferences

MarcFisher: I don't think spec should be more prescriptive that it is now. it suggest, non-normative, that certain standard caps should be considered more important

jgraham: desired caps are an attempt to reduce allowed non-deterministic behavior

MarcFisher: I believe one things is worth including. for every desired cap, the actual capabilities returned MUST have some indication

lukeis: there are some common capabilities that are returned by the oss project. we should list them in the spec so that vendors can implement them

MarcFisher: it is counter productive to try to mandate a list of caps that MUST be returned
... the list will be growing all the time

simonstewart: if a user sets a desired capability, then in the returned set of caps there MUST be a capability with that key and a value that indicates what has been done there
... is the user also specifies a required cap. then the returned set f caps is a union of hte two sets (desired and required)

JohnJansen: why return the required caps? user knows them because they requested them

ato: let's discuss the nested objects/arrays in the capabilities
... Chrome uses chrome: {}

AutomatedTester: proxy is already proxy: {...}

sam_u: we have a couple of use cases for that
... why not have them?

resolution: yes. support an arbitrary json object for capabilities object

jgraham: you can now use null as a value indicating ... something was not set?

simonstewart: for a desired capability that was requested but wasn;t set, the remote end MUST return either 1) the other value it was set to or 2) null (if it was not set) 3) {} if it was set (but requested not to)

<simonstewart> For every capability defined in the spec, we should return the current value in the session. For capabilities not defined in the spec, remote end implementors can choose to either return the current value, or "null" if not set or the empty object ({}) if set.

<simonstewart> The returned capabilities for a session is a dictionary containing keys for every requested desired and required capability (value set as above) plus (optionally) extra information

resolution: ^

willhelm: do we not have anything non-deterministic left in the spec?

ato: yes we have that left, thats the point of desired capabilities

willhelm: then we should say "do whatever"

MArcFisher: we do

everyone: let's take a break!

Finalise Wire Protocol Endpoints

<gitbot> [3web-platform-tests] 5zcorpan closed pull request #1322: Add tests for the metadata properties on DOMImplementation#createDocument. (6master...6createDocument-meta)https://github.com/w3c/web-platform-tests/pull/1322

<gitbot> [3web-platform-tests] 5deniak closed pull request #1323: Update datalistoptions.html to the current spec. (6master...6datalist-options)https://github.com/w3c/web-platform-tests/pull/1323

<gitbot> [3web-platform-tests] 5deniak pushed4 new commits to 6masterhttps://github.com/w3c/web-platform-tests/compare/26af8561ff83...b7aa726452c4

<gitbot> 3web-platform-tests/6master 4c57cdc0 5Ms2ger: Update datalistoptions.html to the current spec.

<gitbot> 3web-platform-tests/6master 4079e5e5 5Ms2ger: Add a test for HTMLCollection#item to datalistoptions.html.

<gitbot> 3web-platform-tests/6master 43f13a75 5Ms2ger: Remove datalistelement.html....

The New Process. The new way of the spec to the world

Philippe: every single commit, if you want to, can go into the working draft (TR)

our spec in not on githug

we are using mercurial

Philippe: you can install a hook n your side. and notify us on every commit. we will then fetch your document and publish it
... we will retain the last version of every day
... the group has to make the decision: how often to update the doc
... you may decide "every time the editor wants"
... we simply give you a n URI and you can ping it any time. we will go fetch the document etc
... and publish to the /TR

lukeis: do you have a tutorial on good spec writing?

Philippe: no. I can think about it if you want

simonstewart: an editor support group? 12 steps?

Philippe: not yet.

MikeSmith: there's a mailing list ... but that's about technical aspects of spec writing

<AutomatedTester> sam_u: https://treeherder.mozilla.org/ui/#/jobs?repo=mozilla-inbound look for Wr jobs

MikeSmith: there's a guide/survey of API design mistake that were made in the past. not very useful for this group

willhelm: I suggest we make a desicion on this. That the editors push to /TR when they want

everyone: yes

JohnJansen: I would like to be able to review complex changes first

AutomatedTester: we could create a fork..

simonstewart: under github/w3c

<gitbot> [3web-platform-tests] 5jacobrossi created 6submission/Microsoft/PointerEvents-ExplicitDone (1 new commit)https://github.com/w3c/web-platform-tests/commit/a1b64945c240

<gitbot> 3web-platform-tests/6submission/Microsoft/PointerEvents-ExplicitDone 4a1b6494 5Jacob Rossi: Add explicit done call to avoid timeout

<simonstewart> MikeSmith: please can you create a GitHub project under the w3c namespace for the webdriver spec?

<gitbot> [3web-platform-tests] 5zcorpan closed pull request #1327: Extend the Element#tagName test. (6master...6tagName)https://github.com/w3c/web-platform-tests/pull/1327

<gitbot> [3web-platform-tests] 5zcorpan pushed3 new commits to 6masterhttps://github.com/w3c/web-platform-tests/compare/b7aa726452c4...490c952f58ab

<gitbot> 3web-platform-tests/6master 432d5020 5Ms2ger: Add more empty lines.

<gitbot> 3web-platform-tests/6master 4bb8e600 5Ms2ger: Add some Element#tagName tests.

<gitbot> 3web-platform-tests/6master 4490c952 5Simon Pieters: Merge pull request #1327 from Ms2ger/tagName...

<gitbot> [3web-platform-tests] 5jacobrossi opened pull request #1330: Add explicit done call to avoid timeout (6master...6submission/Microsoft/PointerEvents-ExplicitDone)https://github.com/w3c/web-platform-tests/pull/1330

<scribe> ACTION: MikeSmith: please can you create a GitHub project under the w3c namespace for the webdriver spec? [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action12]

JohnJansen: propose use a different irc hash, not #testing

everyone agrees; tomorrow

resoution: ask editors push to /TR as often as they want. with John's caviat

(complex changes won't push without a review: announced in the mailing list)

Screenshots

<JohnJansen> John's Caveat: for most (almost all) changes, just push; for anything deemed 'complex' or 'controversial' please have a review before publishing. Also, always send mail to the list.

<gitbot> [3web-platform-tests] 5jacobrossi closed pull request #1330: Add explicit done call to avoid timeout (6master...6submission/Microsoft/PointerEvents-ExplicitDone)https://github.com/w3c/web-platform-tests/pull/1330

sam_u: spec says that driver should pretend there's an infinitely large monitor
... it is hard to implement in chromedriver
... what's the most correct thing to do
... resizing will trigger a nunch of events etc

<simonstewart> https://code.google.com/p/chromium/issues/detail?id=45209

sam_u: it is sometimes better to show the screenshot of the actual viewport. debugging invisible (off the viewport) elements
... I don't know if this is feasible to implement.

AutomatedTester: one other aspect there are plugins (different discussion)
... we are trying to see what user sees

plugins can both modify dom and overlay elements

AutomatedTester: plugins should be black. accessing flash/plugin content from the page is a attack vector

simonstewart: one can always make a screenshot at the OS level
... but this is out f scope of this spec

<gitbot> [3web-platform-tests] 5jacobrossi opened pull request #1331: Fix inconsistent event param (6master...6submission/Microsoft/PointerEvents-EventParam)https://github.com/w3c/web-platform-tests/pull/1331

JohnJansen: I was worried about scrolling into view elements that aren't in there at the test time

<gitbot> [3web-platform-tests] 5lukeis opened pull request #1332: using wptserve now instead of our own server (6master...6wptserve)https://github.com/w3c/web-platform-tests/pull/1332

JohnJansen: sometimes I just need to check that the scrollbas os certain length; or how much fits in the viewport
... and not the entire page

jgraham: increasingly we see pages that look different in different viewport size
... width and height

JohnJansen: what you want is NOT the full page. example: iphone changed screen size

willhelm: I see 2 distinct use cases
... 1) I want full screenshot 2) I want viewport

jgraham: a "full page" is a fictional thing. it does not exists

most people nod

<MikeSmith> simonstewart, https://github.com/w3c/webdriver

<gitbot> [3web-platform-tests] 5jacobrossi closed pull request #1331: Fix inconsistent event param (6master...6submission/Microsoft/PointerEvents-EventParam)https://github.com/w3c/web-platform-tests/pull/1331

<gitbot> [3web-platform-tests] 5jacobrossi pushed2 new commits to 6masterhttps://github.com/w3c/web-platform-tests/compare/76e0fde3e1d4...b924bffd5f35

<gitbot> 3web-platform-tests/6master 459f4007 5Jacob Rossi: Fix inconsistent event param

<gitbot> 3web-platform-tests/6master 4b924bff 5Jacob Rossi: Merge pull request #1331 from InternetExplorer/submission/Microsoft/PointerEvents-EventParam...

<simonstewart> MikeSmith: Thank you. I'll figure out how to get our existing hg repo there

<simonstewart> I assume that AutomatedTester and I have write permissions?

<MikeSmith> yeah

<MikeSmith> I can copy the sources and history over

<MikeSmith> if that's OK with you guys, i'll go ahead and do it

ato: infinitely scrolling pages, like twitter.com

<AutomatedTester> MikeSmith: yes please

<MikeSmith> hai

sam_u: what's going to be less surprising?

simonstewart: users are surprized when they see a small window

jgraham: they should not be. they may be upset.

ato: we support element screenshots. what is element is larger than screenshot?

JoshJansen: scroll to 0,0, take screenshot of the viewport. thats inthe spec, no?

jgraham: based on the doc position before css transforms... ?

AutomatedTester: for element that;s bigger then the viewport. we scrool to 0,,0 of that element

JohnJansen: scroll as scrollIntoView in the spec prescribed

AutomatedTester: then take screenshot of the viewport

err.. of the element, but not larger than the viewport

screenshot of the element cropped to the getClientBoundingRect()

resolution: ^

jgraham: if element if not at all in the viewport after scrollintoview, then the result screenshot is 0x0 png

willhelm: if element is partially in the viewport, then the result screenshot is the visible part of the element

<scribe> ACTION: make some prose out of the last ~20 decisions scribed above [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action13]

<Automate_> ACTION: Create algorithm for "for element that;s bigger then the viewport. we scroll to 0,0 of that element. If the element can not be fully scrolled take a screenshot of the element that is in the viewport" for takesElementScreenshot [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action14]

(regarding screenshots of the full page and of element)

jgraham: on full page screenshot : noone want to implement, and it is wrong. so we settle on the viewport only

<AutomatedTester> ACTION: takeScreenshot to only return viewport of the browser [recorded in http://www.w3.org/2014/10/30-testing-minutes.html#action15]

Automatedtester: (has a little notebook where he wrote all screenshot related decisions down)

seva: proposes something else other than 0x0 image for invisible element's screenshot

<scribe> Done for the day!!!

RRSAgent: bye

- DRAFT -

Browser Testing & Tools WG, F2F, TPAC 2014

30 Oct 2014

Attendees

Contents