W3C

WebDriver F2F, June 13th 2013

13 Jun 2013

Agenda

See also: IRC log

Attendees

Present
Mike Smith, Philippe Le Hegaret, Michael Klepikov (Performance Logging only), Wilhelm Joys Andersen, Jim Evans, John Jansen, Andreas Tolf Tolfsen, David Burns, Simon Stewart, Ken Kania, Marc Fisher, Gregory Dennis, Eran Messeri
Regrets
Chair
Wilhelm
Scribe
Wilhelm, Eran, Ken, Andreas

Contents


State of the union

Simon: The spec is the laggiest part of the work we're doing.
... We just had SeConfg. Just before that, Google released ChromeDriver 2.
... Adds support for mobile.
... Mozilla announced Marionette.
... It will be in Firefox 24.
... We announced Selenium 3, where the project gets rid of the old APIs.
... The spec does not reflect the current work.
... We content on everything but the wire protocol.
... The test suite also requires work.
... Every implementation uses the Selenium project test suite currently.
... We made the wire protocol mandatory.

David: Mozilla has an implementation of touch we'd like to discuss.

Simon: People are extending Selenium to test native mobile. This work falls outside the scope of this group. See the Selenium project.
... Other big news: Opera is now using the Chromium engine.
... ChromeDriver 2 supports Opera, no?

Andreas: That's the plan, but it's not yet ready in Opera 15.

Simon: Does anyone have expertise on the accessibility APIs? If not, I suggest we wait with this discussion until we have the expertise present.

Performance logging

<Simon> Relevant part of the spec: https://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html#logging

<Simon> It's currently empty :)

Michael: What I'm proposing is to adda a field to log entry called data, which is arbitrary JSON.

<Simon> Current OSS webdriver wire protocol: https://code.google.com/p/selenium/wiki/JsonWireProtocol#/session/:sessionId/log

<Simon> Representation as Java: http://selenium.googlecode.com/git/docs/api/java/org/openqa/selenium/logging/LogEntry.html

Simon: (Refers to the links above.)
... Looking at the interface definition from Java, it sounds like you're proposing we add another method to that.

Michael: Yes.

Simon: It would be a map of string to object.

<Simon> Suggestion: adding a map/dictionary of string->object

Eran: Question on context: Does the log entry makes sense with data and nothing else?

Michael: Yes. It's not really for human consumption.

Eran: This will be distinguishable from the log type.

Simon: Overview of how we do logging in the OSS project: The problem we're trying to solve is that there are multiple sources of messages. Local side, browser.
... There are also intermediate nodes in the chain (f.x. SauceLabs).
... There are 3-4 different sources of log information. At the end of the test run, you should be able to get all these logs.
... We have log types.
... Exposed through an API.
... "Get me the logs of this particular type."

Michael: I'm working under the assumption that these are browser specific.
... I'm just requesting an [opaque] data field.

John: This seems right to me.
... A new data field makes sense to me, at a high level.

Eran: We would find it useful for getting profiling data.
... It seems related, but might not be in the same format as this proposal.

Simon: We should probably leave the data format undefined for now.

plh: The web performance working group has a HAR format.

Michael: I have reservations about that.

plh: You should raise these with the web performance WG.
... We are working on diagnostic APIs and the ability to do some logging. Early in the process.
... There is work happening, within the browser itself.

Simon: We can already return logs from the J2E server. Those formats are not well defined.
... There is value in standardizing specific formats. I don't know if we can do that here.

plh: What logs do you have?

Simon: At what point was the flake in the flaky test introduced?
... We wanted logs from each moving part.
... We could do an aggregated log, or display separately.
... Useful diagnostic tool.
... JS errors is a classic example.
... Not every browser exposes this.
... We need an API to query available log types.
... Do we have standardized names for log types?
... What if there are clashes?
... Should we circle back to this later?

David: Yes.
... I don't see it being too contentious.

Simon: The proposal is an undefined JSON blob.
... We should put thought into how to name log types.
... Do anyone need thinking time?

Jim: Yes.

Simon: We'll revisit this after lunch.

Visibility of elements

Simon: Greg has been working on the browser automation atoms.
... One of these is isdisplayed.

Gregory: I've been working on this for a year.
... Overflow is complicated.

<Simon> The Automation Atoms on the selenium wiki: https://code.google.com/p/selenium/wiki/AutomationAtoms
... The relationship between visibility and interactability.

<Simon> The current source of the atoms: https://code.google.com/p/selenium/source/browse/#git%2Fjavascript%2Fatoms

The spec currently says that to be interactable, you must be visible.
... I'm not sure that's the case.
... I don't think we should put that in the spec.
... One issue is opacity.
... In the current JS implementation, a transparent element is not visible.
... But it is interactable.
... These are two separate questions.
... (Quoting the spec on visibility.)

<David> https://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html#determining-visibility

Gregory: I wonder if we can get away with being a little vague.
... An element is shown if a human user can see it.

Gregory: Does something have to be scrolled into view to be visible?
... The atoms say this is not necessary.
... THere are practical benefits to this.
... "Is something visible on the page?"
... There is a moral argument against it in that a user can't actually see it.
... UI sometimes make scrollable things that don't have scrollbars.

<Simon> (the main benefit is that a test can make assertions about something being visible regardless of the size of the viewport of the browser)

John: It's not binary.
... This thing is on the page, but not visible.

Simon: "Could it be made visible?"
... "Is it within the viewport?"
... Finding the viewport is difficult.
... Is a spec defining a viewport?

John: CSS is attempting it.

David: getboundingclientrect is based on viewport.
... It must be defined somewhere.
... CSS3 Transforms depend on this.

Gregory: The closure implementation of getting the viewport is quite accurate.

David: scrollx, scrolly tells you how far you've scrolled.

<John> this might help: http://dev.w3.org/csswg/css-device-adapt/#the-viewport

Simon: I'm happy to rely on this definition.

<Simon> Sounds like we mean: http://dev.w3.org/csswg/css-device-adapt/#actual-viewport

<Simon> Wilhelm^ http://dev.w3.org/csswg/css-device-adapt/#actual-viewport

Gregory: If you want to say, "is it scrolled into view?", that's a separate question.

Andreas: It's also impractical if you want to determine if a button is clickable.

Simon: "Can I scroll this into view?"

Gregory: Interactability vs visibility.

<Simon> My interpretation so far: we need "can we move this element into the actual viewport?" (the current "is shown" behaviour) and "is the element currently within the actual viewport?" (which isn't currently defined)

Marc: From the point of view of test authoring, we need to be able to determine if we can interact with an element.
... (Discussion on opacity and bubbling.)

Jim: Should we define interactability, and expose it?

Simon: Yes.

Marc: The criteria for sendkeys vs click may be different.

Gregory: It might be interactable for one type of action, but not the other.

Marc: You could tab into a field that is hidden.

John: You may even be able to tab into a disabled element.

Marc: Handling of disabled elements vary between implementations.

Gregory: If something is typable or not clicking, clicking would just be a noop.

Simon: We seem to have inconsistent behaviour between browsers.
... Standardizing this will be tricky.

Gregory: Can we define this by definiting a minimum?

Simon: What are the meaningful conformance tests we want added?

Gregory: You could create a test suite with the minimum conditions we want to impose.
... Implied tests, "if this is true on this browser.."
... Capabilities?

Simon: I want a meaningful list of tests to add to the spec.
... "Could you interact with the body?"

Gregory: Not if it's zero size.

Simon: Wouldn't the body fill the whole viewport?
... (Yes.)

Simon: What about a disabled input element?

Marc: You can click on it.

Simon: You could fire a click on it, but it would be meaningless.

Marc: Doesn't it fire events?

David: I think it ignores the events.

data: text/html;charset=utf-8,<input%20disabled%20onclick%3D'alert(1)'>

Eran: So when is an element is not interactable?

Simon: If it lies under another totally transparent element and it not in the tabindex.
... Unless the above element bubbles events.

John: ARIA.

Andreas: Do we want to expose both?

Simon: I don't know if there's any value in exposing interactable? to a user.
... (Tabbing to invisible elements.)

Marc: From the user point of view, they're clicking on the underlying element beneath an invislbe element. The intermediate element may complain, depending on implementation.

Gregory: A user may want to interact with a transparent element.
... Use case: UI with transparent element that turns visible onmouseover.

Simon: The action element lets you specify an x/y coordinate to interact with.

Andreas: From a test author's view, I wouldn't want to rely on interactable? to be exposed. I would try to click the element, and see what happens.

Marc: People expect to be able to click something, but there may be a transparent element in the way. ChromeDriver throws an exception.
... That might be undesirale behaviour.

Simon: There are two versions of click: Actions API, WebElement.
... "Do what I say" vs "Do what I mean".

Marc: There's different behaviour between Chrome and Firefox.

Simon: For some of this, you need access to the render tree.

John: The differences between browsers mean that one of them is not compliant.

Marc: In Firefox, this was like firing the event directly on the element.

John: Which is mandated by the spec?

Simon: Probably Chrome.

John: We've been looking at this in IE.
... Certain version of IE handles this differently.
... I just want to make sure we are basing tests on specs, not browser implementations.

Simon: How do we define interactability?

John: (Describing a bug with floats and fractional pixels.)
... I need the distinction between immedately visible and requiring scrolling to be visible.

John: One of these tests should pass, and the other should fail. (visible / interactable)

Gregory: "The browser window size is different, my test breaks"

Marc: (Refers to CSS spec on viewports.)
... Scrollable divs are handled differently.

Simon: I'm glad this is easy.

<David> http://www.w3.org/TR/CSS21/visuren.html#viewport

Simon: So we're using the CSS 2.1 definition now?
... How does that handle frames?

ACTION: John to ask the CSS WG which definition of viewport, etc., we want to use [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action01]

Simon: Given sane definitions here, this is further complicated by: z-index, tabindex

Marc: If we're not going to expose this to the user, we can talk about clickable and typable as separate properties.

Simon: (Explains why disabled elements are clickable in the WebDriver API.)
... "Would a user be able to interact with this element?"
... If any opaque element is in the way, the element is not interactable unless you can tab to it.

Marc: "Given the element was enabled, this is the element the event would go to"

Gregory: I thought I understand what I wanted...
... Now I don't know anything anymore.

Jim: "It's all edge cases!"

Andreas: There is a list of edge cases we could make test against.

Marc: Maybe you could only interact with elements you could add an onlick handler to.
... The actions API doesn't care about interactable.

Simon: Users don't know how browsers work.

Marc: What elements can't you attach handlers to?

<Simon> http://www.w3.org/TR/html401/sgml/dtd.html#events

Wilhelm: I'm not sure this is relevant anymore... What do current specs say?

<Ms2ger> I can definitely tell you HTML4 isn't relevant anymore

David: The OSS project has users that still test against IE6.

John: It's not necessarily a browser legacy issue that should be ignored.

Wilhelm: We should just write the tests and extract the spec from that.

<Andreas> https://dvcs.w3.org/hg/webdriver-test

ACTION: David Burns to lead the work on writing tests on visibility/interactivity [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action02]

Jim: The spec says the "do what I mean" APIs should delegate down..
... WebElement.click method should check for visibility, interactability, scroll into view, use advanced user interactions ...

Simon: That was the intention. There are edge cases.
... (Selection of multiple elements.)
... This brings us to ... partially visible elements in the viewport.

Visibility

Simon, first question on the agenda: what is considered visible? not the same as being interactable

David: ideally something you could see on the screen
... If an element has text and is the same colour as the element, is it visible?
... An element with the same colour as the element behind it
... Also problems with opacity and partially hidden elements

Simon: being visible means that if I were to take a screenshot of the dom, is there a sequence of actions that would make this element visible

Marc: do you only mean the global scrollbar?

Simon: want to take out of the equation clever things like endless scrolling
... the common use case: usage of web driver as a service, no access to the machine, can't set the resolution

Marc: agree, be resolution-independent as possible
... scrollbars internal to the page feel different to the window scrollbars

Gregory: people create scrollable elements without scrollbars, which is a problem

Simon: we're not going to handle that

Marc: we should stick to standards when it comes to respecting scrolling mechanisms

Simon: otherwise we'll have to run every code path hoping one of them will scroll

Gregory: users don't care.

Wilhelm: we should define explicitly that we don't handle crazy custom scrolling cases

Simon: agree
... already defined this for taking screenshots

John: when scrollbars aren't visible until you hover, web driver should understand that the scrollbars exist ever if not displayed

Marc: it is a rendering choice of the browser

David: main use case from firefoxos is z-inedx. Trying to avoid doing a lot of the animation to conserve power. Apps developers overlay items.
... automaters have a hard time figuring out if the element at the bottom of the stack is visible
... we live in a 2d world and trying to solve a 3d problem in a 2d world
... if the top-level one has no opacity and you can see through it, is it visible?

Simon: we'd be in a far better place if we did this work 3 years ago

David: in firefox we have no access to the render tree

Simon: the implementation should be able to access the rendering engine to figure it out

Andreas: if we go this path, can we use a definition from another spec?

Simon: feels like it should be defined in another spec

David: if it affects rendering, it probably should be

John: mentioned relevant parties on Chrome/Accessibility

ACTION: Simon to continue conversations with html5/css spec committees [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action03]

Simon: this feeds into things like our implementation of getText, which tries to be consistent across platforms
... textContent doesn't capture it because there's no consistency between browsers.
... if we could ask an element if it's visible getText would have been much faster

John: Is it a bug if an element says it's visible but it actually isn't?

Simon: yes

John: the test would have to prove the visibility of the element

Marc: we're not worried about rendering bugs in a web driver test. We use screenshot comparison tests for that.
... our web driver tests tend to be more functional

John: Are screenshots brittle?

Marc: yes, but not done for most projects

Simon: should we add takeScreenshot to WebElement?
... multiple voices: yes, please

Simon: example for a partially-hidden element is a div within a div. the outer div is partially visible.

David: also caused by css transforms. Could affect visibility during the transformation

Simon: to avoid getting a screenshot mid-animation people should use waits
... trying to click on an element that's constantly in motion can't be done completely "black box".
... results taken during a transition will vary and that's ok, as they should be accurate to the time the screenshot was taken

David: main issue is z-index
... and partially hidden elements

Marc: overflow considered partially hidden

David: implementation in marionette: check all 4 corners of the element as well as centre, see which returns true. If any returns true (or the centre)

John: if the element is very big the centre of the element will also be off

David: 10k by 10k element will not be visible

Gregory: why isn't the element visible if the element and the viewport intersect in any way?
... generalise this with overflow calculation: check if the element intersects with its ancestor elements. it can be generalised in that way
... is an element necessarily visible if it has a descendant that is visible?
... Google maps UI, the map is 0 by 0 elements, has tile children with positive size, but the map is 0 by 0
... so the atoms say you have a positive size is if your descendants have non-zero child.

John: don't think we can say it's visible

Gregory: but you can get events

John: but they're actually interacting with the children
... divs with content that comes from a hidden frame

Gregory: element which is negatively positioned in the viewport but has a child within the viewport

Simon: there's a limited set of elements that can get keyboard input

Marc: the advanced interactions API doesn't care about interactability.
... only have to determine coordinates to click at

Gregory: if they were to click on the map, they would not be able to.
... map tiles are not known in advance, only want to click on the map
... that x,y may not be interactable

Marc: the element is used as a reference, to calculate position

Gregory: this is a distinction from the javascript implementation, where the element provided is the element to be interacted with

Simon: do what I mean vs. do what I say, Interactions API is do what I say
... many of the limitations originate from the original implementation of the iedriver

Jim: now we do fairly complicated calculations to figure out what the screen coordinates we should be clicking at are
... based on element coordinations on the screen, including iframes
... and we can't use the atoms because if there are cross-domain frames the atoms would not be able to get the document elements

Simon: what does it mean for visibility?
... the intuitive definition is "i can see some part of it on the screen"

Andreas: if any of it is visible

John: what about the children being visible but not the parent

Wilhelm: we should write tests

John: makes sense that the parent is not considered visible even if it's children are (unless it's visible by itself)

Marc: the visibility of the children does not affect the visibility of the parent

Gregory: they want the map to be considered visible if its children are visible

John: css3 regions are relevant as well
... Animation should be drawn only if the element is visible, no reason in doing it otherwise

<John> http://dev.w3.org/csswg/css-regions/

Jim: anchor element where the entire of it consists of a span element

<Simon> plh: can we please get a link to the web performance spec where this is defined?

Jim: the span element was considered to have 0 dimensions

<plh> Page Visibility API
... but it only defines for the document, not the elements (yet)
... we'll need to do this for the elements in order to refine requestAnimationFrame

Gregory: I have a case where the element is negatively positioned outside the viewport, but has children in the viewport

Gregory: should the same exception we make for size should be made for negatively-positioned elements?

Eran: does the problem not stem from the selection of the element to be interacted with?

Gregory: the event handlers are on the map, tiles amount and naming is unknown

Wilhelm: is visibility an academic problem?

David: what opacity do we set to say something is visible? currently anything above 0
... that should be higher

Marc: you can click on something with 0 visibility

David: the lowest visible value is 0.05 (with human eyes)

Simon: select arbitrary limit for the visibility
... white text on white background is still visible, just hard to read

Marc: if you care about actually being able to see something you should be doing image analysis

<plh> ACTION: Simon to change link from DOM2Core to HTML5 in Section 12, Cookies [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action04]

David: element with visibility hidden?

Simon: we've covered it in the Selenium IRC channel

ACTION: Simon to link to the discussion on elements with visibility hidden [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action05]

Simon: if the size of the element is positive then disregard visibility hidden, because it affects layout

David: from a visibility point of view, it's not visible, but asking other questions (size) should be correctly returned as they affect layout

Marc: is displayed returns false , but get location and get size return correct values?

David: yes

Marc: but it's not interactable?

David: yes
... if display:none then it's totally different

<David> http://eideticker.mozilla.org

Timeline for spec work

<Simon> http://www.w3.org/2011/08/browser-testing-charter.html

Simon: According to our charter, we were supposed to be finished by February .. 2013.

John: We should put some amount of thought into it, and make it as real as possible.

Simon: It should have some bearing of reality.

plh: The product is your spec.

Simon: We're doing our spec backwards. We have implementations, trying to go back to a spec.

John: By making a concrete schedule, you need to have make tough decisions. Limit the scope.

Simon: We've done a good job of limiting scope so far.

ACTION: Group will meet in the #testing channel on July 12th, to work on spec and test suite together [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action06]

RESOLUTION: Group aims for LC just after TPAC: 22nd of November, 2013

Jim: CR in March 2014?

<John> http://dev.w3.org/csswg/css-regions/

John: The current spec does not use plinss' tool for mapping tests to chapters.
... What it doesn't tell you how many tests you need for that paragraph, but it tells you how many you have.
... Every statement needs >0 tests.

ACTION: John will share annotations on needed tests [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action07]

Andreas: We could create empty tests and fill them in.

Simon: I'd prefer a todo list.

John: This approach is useful for TTWF.

ACTION: John to investigate feasibility of running the Python-based test suite at MS [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action08]

Simon: PR in April.

RESOLUTION: WG would like to meet at TPAC in China in November

<Simon> Current touch https://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html#touch

Touch events

<David> http://nakubu.com/site_media/webdriver-spec.html#interactions

David: with firefox OS, first target is mobile
... we've looked at open source project, it assumes one finger; nowsdays multitouch is important
... examples of garage band, microsoft surface; there's a need to emulate multiple actions

<Gregory> the atoms TouchScreen abstraction: https://code.google.com/p/selenium/source/browse/javascript/atoms/touchscreen.js

David: in our implementation, we combine actions into one batch for transmission
... we do time slicing; you can create chains of different lengths, if you want chains to be of the same length, you can send a no-op
... This is our first implementation. There's no regard for pointer events. For instance, we don't handle tilting

Gregory: I'd argue this should be just for touch. Pen, etc. is separate

Simon: Agree.

<Simon> http://www.w3.org/TR/pointerevents/#pointerevent-interface

David: there is no JSON wire protocol for this yet, since we are using an internal protocol
... should there be separate implementations for touch and for click?

Simon: we should redirect based on the underlying device
... if you do need to distinguish between click and touch, you could use lower level action apis
... but the default would match the system

John: can you override it?

Gregory: if override requires to go through the advanced interactions API, that's a bit much?

Jim: i'm less concerned about the high-level interface; we should have a single endpoint called input which takes an array of actions

<Simon> ArtB: yes it would

Simon: let's finish the straw man for touch before batching

<Simon> David tells us that he's spoken to some of the mozillians involved, but hasn't had their strawman in place for long

David: if its not too contentious, I'd be happy to forward to the other working group, but I'd like our input first
... We have tap on the element; users wanted to know the difference between click and tap
... we've done click in the past and redirecting, but harms readability of tests

Simon: this is a stronger argument to remove click from WebElement

Jim: it would force you to go to the advanced interactions

Marc: that means we'd ignore all the interactions stuff we discussed earlier

Andreas: could we have a state on the web driver itself that says whether it is click or tap?
... if readability is a problem, I would subclass the WebElement instance

Simon: click is very clear about what it means; you could tell someone to click something on their phone; adding an individual tap is bad

Andreas: I agree adding method for tap is bad

Gregory: which of the ones we are proposing does a redirect
... is there a drag that does a swipe

Simon: this is only for stuff on webelement

<Andreas> Isn't that what I proposed earlier? d-: Oh well.

Simon: if you care about the actual events, you should be using advanced interactions

<Vishal> Simon: sorry if this is missing some context, but what's the motivation for keeping two separate levels? those on webelement and interactions?
... Simon explains way actions work currently in open source

<Simon> tl;dr: Actions make use a Mouse and Keyboard abstraction.
... David explains straw man proposal linked earlier

Simon: aggregations of underlying things that could be done on the local end don't need to be in the spec
... I'd like to see the deepest level of abstraction instead of starting the discussion on the high level
... I'd like to see the equivalent of the mouse and keyboard interface

Gregory: why is cancel needed? a user cannot right?

David: mashing your hand against the phone will cancel
... everyone tries mashing their hand against their phonee

<Vishal> if we have a mouse and keyboard, would having a screen (for touch) also make sense?

<Vishal> going forward, screen interactions will become more relevant (and important)

<Simon> Vishal: David is currently describing the touch strawman
... David explains time ticks and necessity of batching for latency purposes

<Simon> So "yes"

Simon: are beats based on time or on command

Marc: if you make it time based, it's hard to make it line up across multiple fingers; action based is easier

Simon: you can imagine causing failures in both pretty easy
... does anyone think time based is a good idea?

Gregory: I'm not sure long press is necessary
... David explains that long press is used to bring up context menu for testing in firefox os

Wilhelm: how would i zoom out in a map application? wait an amount of time?
... Simon describes java interface for mouse

Simon: what are the primitives for touch?

Marc: is velocity part of a move?

Simon: aggregation of underlying primitives should be pushed outside of the spec

Marc: how long it takes for a long press varies on device

Simon: we need to find out what these primitives are for device
... so have a tap which takes a count, ok?
... many agreements

Simon: I think long press also needs to be a primitive
... Simon talks about open source touch primitives

David: I don't think scroll is needed, just a press and move

Simon: I think I agree

Gregory: do you want the ability to specify time in wait, instead of a pause to sync for the tick

Marc: the time in the wait could specify the minimum amount of time of the tick

Simon: I think that makes a lot sense
... I think we've agreed on ticks is a good thing, having taps with a count parameter, long press
... and finger down and finger up, move by offset
... talk about how to do rotation

Gregory: instead of multi-action, another option would be multiple locations in a single action

Simon: might want to do different action types at the same time

Gregory: that is a good argument

Wilhelm: how would rotations work?

Jim: a huge batch of straight lines, which could be composed at a nicer way at a higher level
... many voices agree

Marc: does the mozilla implementation work with multiple input devices at the same time

ACTION: David will update the proposal for touch primitives and send it out for review [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action09]

David: not yet
... David points to place in mozilla straw man that discusses batching actions

Batching

Simon: The need for batching is obvious, ideally for everything.
... We're only looking at user input batching atm.

Jim: From a wire protocol perspective
... You send a command to a single protocol endpoint
... Which as its JSON data takes an array of JSON objects, each of those objects describes an action.
... The format of those actions is ... it takes the name of the action.

<Simon> https://code.google.com/p/selenium/wiki/JsonWireProtocol#/session/:sessionId/buttondown

Jim: Instead of the parameters simply taking the numbers, it would have name, colon, button down, colon.

Simon: It would also need the name of the device.

Jim: I hadn't considered that because up to this point I was only dealing with desktpo types.

Marc: Touch, click would be what it would be otherwise
... I don't think you need a separate device thing.

Jim: touch/press
... All of them are prefixed.

Marc: I think it would've been nice if keyboard and mouse had done that to begin with.

Jim: That's my strawman proposal.

Simon: How would we do it?
... I guess it would be a list of lists.
... For rotations it would be nice to go as soon as you can.

Marc: List of beats, then each beat is a list of actions in itself.

Simon: I'm actually happy with that.
... We've already had this argument some times.

Jim: Ticks, ticks, whatever we're calling it.

John: I'm going to spell check the beats to “ticks”.

Jim: So how would that work, how would it sit for David?

David: Each line as one list, then we slice the lists. So that's fine.

Simon: Cool.

David: That's what I was looking for.

Simon: Does anyone disagree?
... No.

Resolution: We batch with a list of ticks, and each tick is a list of actions.

Resolution: We have a single end-point in the HTTP protocol for actions to be sent to.

ACTION: David finish his strawman proposal. [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action10]

Simon: Do we invite Michael back now for the data in logging discussion?

Marc: Naming conventions for log types?

Resolution: Add data field to a log entry, which should be a map of string of objects (JSON blob).

mklepikov: We agreed to add your proposal. No need for further discussion.

Resolution: Add data field to a log entry, which should be a map of string of objects (JSON blob).

Simon: Topics we could cover if we so wish.
... Modern HTML5 input type, on which there is no concensus.
... Security dialogues.
... Sensors, caches, interaction with sysapps, the test suite.
... I would quite like to either do the new sensors, or the modern input types.

Jim: The input types is the only one I have real input on.

HTML5 input types

<Simon> https://groups.google.com/forum/#!msg/selenium-developers/_JiNc65rOUo/SJxNjCQq7t4J

<Simon> http://www.w3.org/html/wg/drafts/html/master/forms.html#the-input-element

<Simon> http://www.w3.org/html/wg/drafts/html/master/forms.html#attr-input-type

Simon: The type attribute has gone from being very simple to very complex.
... Whereas you could previously only upload one file, you can now upload multiple files for example.
... Some of them just fallback to plain text.
... datetime, datetime-local, range, color pose some problems.
... They probably bring up some native UI. I don't know how IE does it, but Firefox does shadow DOM.
... We need to be able to send data to it.
... The file input element allows you to upload multiple files.

<Wilhelm> http://shwetankdixit.com/testpages/webforms2demo.htm

Simon: So the interesting thing is, if you have an old browser it will fall back to rendering all of these as text inputs.
... We could say just go on and send strings.

Marc: Is it really? The forms themselves just send it to the backend.

Simon: What do we do with datetimes?
... Well you have to do it with an ISO formatted date, then you end up jumping through these hoops, you end up leaving people confused.

Andreas: I like Jason Leyba proposal to use client binding utility classes.

<Wilhelm> http://www.w3.org/html/wg/drafts/html/master/forms.html#date-state-(type=date)

<Wilhelm> Definition of a valid date string: http://www.w3.org/html/wg/drafts/html/master/infrastructure.html#valid-date-string

Simon: What is a valid date string?

Wilhelm: The spec includes saniation for any string.

Simon: We could do that.
... How do we cope with the case where the multiple attribute has been set?

Marc: For file uplods?

Simon: File is one, but I think they can all have multiple.
... Now the problem is that our sendKeys API already takes a list.

<Wilhelm> The multiple attribute: http://www.w3.org/html/wg/drafts/html/master/forms.html#the-multiple-attribute

Marc: It could just send it, and and send it, until you hit clear.

Simon: That might work.

Marc: Just continue appending it on.

Simon: I think they can all take multiple values, although it makes no sense.

Marc: What does that look like in the frontend?

Wilhelm: Several emails make sense.

Marc: Yes, appending to an email field makes sense.

Simon: Should we just say “we will attempt to add an extra element, unless multiple isn't set and the browser supports the new HTML5 _and_, yeah”.

Marc: We use well-formatted strings to send keys as defined by the HTML5 spec, and if the field has multiple attribute it just appends another field, and you could clear it using the clear() API.

Jim: Not all of them accept @multiple.
... For example for INPUT @type="tel".
... It's valid on the INPUT element, but it does not apply.

Marc: Yeah, it's non-normative.

Gregory: What happens with the select boxes?

Simon: Select isn't an input.

Marc: We don't have to worry about select in this case.
... And select. You click on each item.

Gregory: Yes, there are multiple, individual actions on the wire protocol level.

Simon: If multiple isn't supported, append as we do currently with text or unspecified.
... We're going to be providing utility classes.

Marc: So we're going to have to throw an exception if they try to enter another colour?
... Except for password, telephone, text, date, colour.

Simon: Files are handled in a very strange way.
... We upload the file to the remote end, have it return the path …
... Discussion about driver specific INPUT @type="file" implementations.

Marc: Maybe we just throw exceptions for values you can't append to without clearing?
... Presumably all of these can have default values.

Marc: So do you have to clear it before setting it?

Andreas: Yes, replacement makes sense.

Simon: So we do replacements for the ones we can do
... Replacements for dates, times, numeric inputs (number and range), colour, file upload.
... And we do append for text, telephone, url, email, password.

Marc: These aren't supported by all browsers.
... So are we smart about detecting the type?
... The attribute is still there.

Simon: We will do an append, because the input type is text.
... If I go to the demo page with a browser that doesn't support the HTML5 controls, you'd only get text inputs.

Gregory: The property will still be text.

Simon: Yes, that's the difference between attributs and properties.

Marc: And getAttribute would return "color" right?

Simon: yes, because the attribute is defined as so.
... And actually that's what people would actually expect, right?

Marc: I think that's reasonable.
... So.
... Replace or append for send keys depending on the multiple attribute.

Gregory: Can you clarify what happens in the wire protocol?

Simon: Normally we just send the the keys across the protocol.
... Unless the string matches a local file.
... And you have the LocalFileDetector.
... You upload that to the remote end.

Gregory: You'd still use the sendKeys() command for all of these?

Simon: yes.

Jim: Yes.

Gregory: It would be a bad idea to have a select command?

Jim: The proposal for language bindings would be to expose a wrapper element type.
... Which under the covers would call sendKeys with the correctly formatted strings.

Simon: So the idea is that we have as little in the spec as possible.

Marc: It's really that we're taking advantage that the _output_ of the input fields return plain strings.

Wilhelm: Should we validate?

Simon: If a user calls sendKeys() directly they're welcome to send whatever they want.
... But that would be the remote end telling us you can't.
... The language bindings would have utilities thta would make it easy to use these things without messing up.

Gregory: Even if modern browsers implement this would it be posisble for users to attach something else to it?

Simon: Are we content with that?

Jim: I'm happy with that. It's what I wanted to do anyways.
... Support classes, which are more discoverable than role based interfaces.

Simon: Okay.

<Simon> http://selenium.googlecode.com/git/docs/api/java/org/openqa/selenium/security/UserAndPassword.html

Resolution: Default behaviour for something that has a property of type that is text, is to append. If the browser supports the particular type (Fx not doing color), if the input is correct formatted we will set the value to whatever it is, otherwise throw exception. If the value is multiple and the browser understands the concept of multiple, otherwise it will be appeneded unless it's a file input element in which case it will be replaced.

<Simon> Something like that.
... particular type _and_ if the

Resolution: If the multiple property has been set and the browser supports the multiple property, new separate values will be added, otherwise the value will be replaced.

Summary of Action Items

[NEW] ACTION: David finish his strawman proposal. [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action10]
[NEW] ACTION: David will update the proposal for touch primitives and send it out for review [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action09]
[NEW] ACTION: David Burns to lead the work on writing tests on visibility/interactivity [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action02]
[NEW] ACTION: Group will meet in the #testing channel on July 12th, to work on spec and test suite together [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action06]
[NEW] ACTION: John to ask the CSS WG which definition of viewport, etc., we want to use [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action01]
[NEW] ACTION: John to investigate feasibility of running the Python-based test suite at MS [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action08]
[NEW] ACTION: John will share annotations on needed tests [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action07]
[NEW] ACTION: Simon to change link from DOM2Core to HTML5 in Section 12, Cookies [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action04]
[NEW] ACTION: Simon to link to the discussion on elements with visibility hidden [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action05]
[NEW] ACTION: Simon to continue conversations with html5/css spec committees [recorded in http://www.w3.org/2013/06/13-testing-minutes.html#action03]
 
[End of minutes]

</meeting><pub>

Minutes formatted by David Booth's scribe.perl version 1.138 (CVS log)
$Date: 2013/06/14 15:30:06 $