Browser Testing and Tools WG, Monday, TPAC -- 29 Oct 2012

<sms> Hi jari

<sms> I should change my nick

<simonstewart> That's better

<jari> hi

<simonstewart> The room's looking a little bare now

<simonstewart> (physical room, that is)

<simonstewart> Starting in 5 minutes

<jgraham> (does it have the countdown music for the last 30 seconds?)

<jhammel> it should!

<simonstewart> I'll be sure to hum it loudly

<wilhelm> Scribe: jhammel

wilhelm: is the list of would-be attendents publically available?

<wilhelm> jhammel: https://www.w3.org/2002/09/wbs/35125/TPAC2012/registrants

thanks

<simonstewart> We've had our 5 minutes

<simonstewart> And we're off!

wilhelm: For the first session, what is webdriver? What is the state of the spec? What would the attendents like to get out of webdriver?

introductions: Michael Smith, Andreas Tolfsen, Shangyi Chen, Jeff Hammel, Larry Masinter, Wilhelm Andersen, Simon Stewart, [more]
... Byungjung Kim, John Jansen, Ken Kenia

simonstewart: 3 audiences for driving a browser
... 1. people that want to drive a web application end to end
... <aside>original selenium was limited to what you could do in a JS sandbox; but the web has become more sophisticated
... webdriver integrates tightly with the browser; so webdriver + selenium merged to become selenium 2
... webdriver is currently the API of selenium

</aside>

2nd group of people: browser vendors

andreastt: Opera's made a huge undertaking in converting testing to use the webdriver API

simonstewart: 3rd audience: people that are writing specifications on their own who can't write these in pure JS (mostly hypothetical at the moment)
... i've thrown together a quick demo
... walk through demo: first create a browser instance; get a particular URL ...
... webdriver represents a browser instance, but people are interested in elements on a page; [demos how to get an element]
... summary: we're going to google, searching for cheese, then arrowing down for autocomplete
... the ability to do testing across browsers with only changing the driver is very powerful with rapid releases from browser vendors

larry masinter: there are no assertions?

simonstewart: no, assertions aren't part of the webdriver API; this is left to whoever is writing the tests/scripts
... each of one of several languages that selenium has language bindings have their own mechanisms for running tests and doing assertions
... there is a JS binding for webdriver for nodejs
... there are multiple implementations of webdriver; the OSS project is where the webdriver work began
... opera were the first implementators of webdriver; written separately, but using the OSS project to ensure compatability
... Opera owns the Opera driver; Chrome is maintaining the Chrome driver; Mozilla is taking responsibility for the Firefox driver with Marionette

Larry Masinter: You want to test scrubbing through a video; is that doable?

simonstewart: there is a section on dealing with non-HTML content; we currently don't have a great story on how to test
... example: canvas; writing to canvas isn't generally done with testability in mind
... what we want to do with the webdriver API is to bring focus to how to make what is done on the web testable/automatable (social engineering)

wilhelm: isn't the video element just shadow DOM?

simonstewart: one of our topics is dealing with the shadow DOM
... other open questions: SVG, a11y
... a surprising amount of web pages are still JS and HTML

Larry Masinter: having a roadmap of what needs to be done would be really useful

simonstewart: there is a section on extending the protocol; extending the APIs + vendor extensions
... example: wouldn't it be nice to have a API for bookmarks? However this is done differently across browsers
... process: experiment; see what works and doesn't; consolidate on what works
... just the point of getting a standard that works with JS and HTML and works with the various browsers is a huge undertaking
... then figuring out what people really want

Larry Masinter: as a separate item, in the long run, what needs to get tested? Mapping out what all should be tested

Larry Masinter: e.g. performance testing doesn't fit in this framework; but there should be some place where performance testing can be consider (wrt w3c WGs)

Andreas Tolfsen: we can look at the webdriver API as part of that infrastructure

Andreas: we use webdriver to run the tests; then we use another framework actually handle the testing
... webdriver needs to be much more generic than just browser testing
... the screenshot story is a bit complicated
... if we take screenshots externally, should that be part of the spec? otherwise, this could trigger a window repaint

jhammel: so webdriver is fundamentally a browser automation framework, not a testing framework

simonstewart: yes

Larry Masinter: but it is the browser testing charter?

Simon Stewart: yes, we need to ensure that webdriver supports all that is needed to test

Larry Masinter: I wonder if we could do fuzzing

Andreas: it would be difficult to try to put all security testing into webdriver

Simon Stewart: tests that involve using a browser like a user are good candidates for webdriver automation

Wilhelm: Next topic: State of the union: where is the spec right now?

Simon: I've landed a bunch of edits; the specification is moving forward
... the OSS selenium project; 15-100 checkins / week ; most of this is refining capabilities
... (handling edge cases, etc); the broad strokes are fairly well defined
... however, getting the test cases into the w3c test suite and ensuring that the spec is rigorously defined in English is still needing
... some sections are fairly flushed out; some are skeletal
... the implementations are further along than the specification because of a common test suite

Wilhelm: which sections are finished?

Simon: Commands+Responses flushed out ...; Switching windows; Running without focus : spec says you should; ...
... if you can use the OSS tests section 10 makes a lot of sense
... cookies = WIP; ...; Modal dialogues: skeletal; screenshots aren't flushed out
... How we do logging in webdriver: logging is out of process
... selenium grid: driver + client may be running on different machines

simonstewart: vs in process; maybe console.log has all we care about

Larry Masinter: You have logging but no assertions

Simon: Yes; we want to get logs back in a consistent format; the logging API we have lets you get consistent records

Larry Masinter: The logs would include assertion failures

Simon: No; it is only internal logging

Larry Masinter: What do you need to do with the logs that require standardization

e.g. the console of the browser

Larry Masinter: If different browsers log differently, does that matter?

Simon: There isn't a standard for the content of the logs

Simon Stewart: you're probably on a heterogenous network of machines; if a test fails, the user doesn't have any insight into what is going on

Simon Stewart: developers want to take a look at the logs at all intermediate stages; this is what happend locally, on a webdriver server, on the test machine, etc

Simon: we need that format in order to be able to get the logs
... we haven't put assertions in, but we've given you the ability to put assertions in
... its not active logging; its fairly passive

Larry Masinter: roadmap: how do you ensure adequate consistency? are we writing our specs too precisely?

Simon Stewart: i've seen a number of projects that have survived major reworking of the UIs without failing; the key is how you find the elements

Larry Masinter: there may be operations that can only be done with multitouch, etc

Simon Stewart: you can query your webdriver instance for the capabilities it supports

Simon Stewart: e.g. do i support touch actions?

Larry Masinter: would you use media queries to determine what tests are run?

Simon Stewart: no; in the OSS project, you only run tests if the assumption is true

Simon Stewart: we've divorced ourselves from a testing framework, but we've given you the capabilities to create a testing framework; we allow you to query the capabilities without enforcing you to do anything if the capabilities aren't met

John Jansen: How do i easily determine the diff between red+blue?

Simon Stewart: using diff from the VCS; would you prefer a different way?

John: is that pushed decided by the wg?

Simon Stewart: Yes

Wilhelm: is there any particular reason why you care about the blue one over the red one?

John: mostly out of habit

Simon: because of the way we're writing the specification, blue isn't invalid

<simonstewart> http://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html#commands-and-responses

Wilhelm: Blue refers to the published working draft; red is the editor draft

<simonstewart> http://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html

<simonstewart> http://www.w3.org/TR/webdriver/

<simonstewart> jhammel: those links probably won't be in the minutes unless you scribe them

Wilhelm: What do you all want from the webdriver spec? What do you want from this meeting

location of webdriver spec: http://www.w3.org/TR/webdriver/

<simonstewart> thanks :)

andreastt: we want web developers to be able to use opera when testing

Andreas: to ensure that they have site compatability
... last year, we released our HTML5 parser; we also got sites to run our test suite versus their sites

Jeff Hammel: trying to figure out what Mozilla needs to do to get Marionette + webdriver on spec

Shangyi Chen: trying to observe what is the progress is being done

Shangyi Chen: Baidu using webdriver

John Jansen: trying to solve big problems at microsoft; this is one point of input

Wilhelm: last time i wanted to test my browser; today i want to test my sites with various browsers
... want the spec to move forward; want all WGs to test all the specs they're working on
... want to be able to point to a spec to say, 'Use this spec to test all other specs'

Simon: working on webdriver for 7 years now; has become the de facto way of testing web apps
... I want to standardize it; as the web gets more capability, it becomes harder and harder
... need support from the browser vendors; webdriver is one of the tools in the arsenal
... the thing that testers hate doing is duplicating effort; they tend not to use another API once they have one solution working

?: Make sure the spec is understood; the more help we can get from browser vendors, the better

?: figure out the limits of the spec; figure out what can be done to extend these limits

Ken Kenia: primary concern for chrome team is website compatability

Ken Kenia: can test websites across browsers quickly

Ken Kenia: we don't use webdriver as pervasively as Opera

Larry Masinter: Does webkit use webdriver?

Ken Kenia: not currently with internal testing

Larry Masinter: should it?

Simon: there are at least two projects that are based on webkit that need to do testing; so yes
... it simplifies the vendors work; it simplifies the driver's work

Wilhelm: is there anything this WG can do to help with the politics of webkit?

Wilheml: if so we should do it

Simon: webkit is worked largely on by Chromium; though Apple depends on it too
... is there anything we can do to encourage participation by MS?

John: let's talk about that later

<sms> Here we go again!

<wilhelm> Chair: sms

<sms> Chair: simonstewart

<wilhelm> Scribe: wilhelm

<sms> Ohhh... I'm back as sms

<sms> Interesting

Discussion of open issues in the spec

<jhammel> http://lists.w3.org/Archives/Public/public-test-infra/2012OctDec/0019.html

(See list of open issues in the agenda.)

sms: Some parts of the spec is integer-heavy. (Points to error codes.) We could use strings instead.

Interoperability

sms: The spec, as written, does not describe a transport mechanism and how to encode data.
... This is for historical reasons. Different drivers use different protocols.
... It is possible to follow the spec - and not be interoperable.
... Suggestion, listed as a note in the spec: implementations should allow the use of JSON wire protocol.
... It would be possible to write an implementation specific to one language. For example a proprietary connector for mobile.
... How do we allow people the freedom to choose the transport mechanism they prefer? JSON+HTTP may not be the most efficient.
... At the same time introducing a new device/platform/browser must be easy.

jhammel: The goal is to encourage it, not enforce it?

sms: We could say "must".
... This is currently in a non-normative section.
... We could make that spec normative.

wilhelm: Why shouldn't we?

sms: Nokia was against using JSON over HTTP. Maybe they don't mind a shim.
... One suggestion is we mandate that every implementation must have the shim.

(Tangent about malicious commits to the conformance test suite using something else than the JSON wire protocol.)

andreastt: Has other specs faced this problem?

sms: WebDriver is the only out of process spec that tries to have a consistent API.

andreastt: It makes sense for me to enforce this for the spec to be practically useful. It should support a unified transport protocol.

sms: This is a fairly safe tech stack.

<scribe> ACTION: Make the section on the JSON wire protocol normative [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action01]

(Discussion on whether all drivers will be part of the open source project.)

(Discussion on language agnosticism.)

sms: If you go into a Microsoft shop and say "I want to use this Java program", that's not going to work. By having a standalone executable people can choose the bindings they prefer.
... I don't want a duplication of effort in the transport layer. If you use the JSON wire protocol, and have a shim, you can use any langauge or driver.
... The shim should be on the side of the browser vendor.

kkania: Is there a reason for out-of-process instead of in-process executable?

sms: This is a may. Implementors are free to bake this into their browser.

andreastt: These are implementation specifics, no? If you are writing something for your own uses, this doesn't cause any trouble.

kkania: Is there a reason for why we want JSON over HTTP?

sms: It works remotely. It works over every single client language you can think about implementing. Even more than loading a DLL.
... Works independently of bitness (32 vs 64 bit).

andreastt: And this only covers the C implementations.

sms: We settled on JSON over HTTP because all these languages have a HTTP and JSON library. We considered thrift and protbufs.
... I wrote a JS client that used XHR.
... One of the major audiences are people testing their web applications. Many of these people don't have root.
... They might not even allow you to install any software. This rapidly became a nightmare.

andreastt: In the millitary, you may not even have access to the Internet.

jhammel: You might even do this manually via telnet, typing everything.

sms: This is currently covered by appendix E. This will be moved into the body of the spec, as normative prose.

Internationalised input

sms: There are two use cases for internationalised input.
... Am I doing proper roundtripping with international characters?
... I want to make sure my app does the right thing as the user is typing using an IME (or similar).
... If you use sendKeys with international characters the character makes it into the input field, but it is nothing like what the user actually does.

eranm: IME is only used for complex alphabets.

andreastt: We have not done any work on IMEs. We don't check if the keys exist on the keyboard.
... There have been several requests from our users for support of changing the charset of the keyboard. This is a difficult problem.

sms: Mobiles have different input methods.
... Choose a different keyboard, long press on E to get É, etc.

andreastt: On fature phones you have predefined buttons that can be programmed. This also applies to some smartphones. Programmable keys.
... You don't know what's going to be on the keyboard.

sms: The IME stuff we have is very desktop specific and requires a lot of knowledge about the machine you're running on. Is this a reasonable thing to have in the spec?

eranm: Since we've added support, nobody has actually used it.
... There are 1.5 billion people using complex scripts. Solicit input from them?
... Hebrew/Arabic are much simpler. What we have may be good enough.

sms: So we should move IME to non-normative.

andreastt: É is registered as two different characters in the browser.
... Complicating factor: OS specific stuff.

sms: And different keyboard layouts: Norwegian/English/etc.
... Sometimes there will be a mapping to a key, sometimes it won't.
... If there is a character on the keyboard, we can just send the right character.
... In the case where a key is not on the keyboard (shalom in Hebrew) we just stuff in the right characters.

davidburns: We should not go in detail on this.

andreastt: What should the spec cover? I don't think this fits. A library on top of the sendKeys implementation makes sense.

davidburns: This may be covered in the Widgets spec.

sms: I'm hearing: We specificy that you must be able to do internationalised input, but don't specifiy how that is done.

<MikeSmith> btw, somewhat related to this topic, Norbert Lindenberg of http://norbertlindenberg.com/2012/10/ecmascript-internationalization-api/index.html is here at TPAC this week

eranm: We emulate shift-a for A. Should we spec this?

sms: We also do the alt keys on Windows.

JonathanJ: IME is very complicated.

<scribe> ACTION: JonathanJ to give input on non-latin character input [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action02]

<scribe> ACTION: davidburns to look into how the DOM spec handles keyboard events and internationalisation [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action03]

andreastt: The current IME section is... nothing.

sms: There is a series of commands for handling IME in code. Document these as non-normative.

eranm: Two ways of using IME: Enable it in the OS, input latin characters. Or give me a list of IMEs and interact with them. We should document both, but the first is definitely non-normative.
... We should listen to people who actually use this.

ARIA locators?

sms: An accessible web is a good web. Should we add ARIA as another type of selector?
... I don't know if the ARIA specs have any APIs that browsers should implement.
... Is there an equivalent to querySelector for ARIA? I don't think there is.

JohnJansen: I haven't seen one.

sms: Given the way we implement XPath, CSS selectors, etc - by delegating to the browser - and there is no API for this in the browser, let's not add this.

Shadow DOM

sms: This'll be fun.
... Is everyone familiar with what the shadow DOM is?

davidburns: Shadow DOM is an object model that is hidden away from view. In a <video> tag, the browser generates a number of hidden elements (play button, etc.) you may want to interact with.
... How should we access these elements?

<sms> http://dvcs.w3.org/hg/webcomponents/raw-file/tip/spec/shadow/index.html

davidburns: If we query the DOM, the browser won't give us the shadow DOM elements.

sms: A shadow host appears as a single element in the DOM. You find a wealth of badness beneath it. It nests - turtles all the way down.

andreastt: Is there any way to expose that in the browser today?

sms: The render tree expands everything out, but not in the DOM.
... How do we want to present this to users?
... Chrome just switched this on.

davidburns: It's behind a command-line flag.

sms: No, it's on by default.

jhammel: Does each browser have a different shadow DOM for <video>?

sms: Facebook like button is a good candidate for shadow DOM.
... My suggestion: If you do a findByTagName within a findByTagName('video'), you query within a shadow host.
... We should follow the model of the JS world.
... Don't cast to a specific shadowElement.
... Does this sound reasonable?

davidburns: Yes.
... There will be a lot more shadow elements in the future. They simplify a lot.

sms: Once you're in the shadow DOM, it looks like a regular DOM.
... Do we want to indicate to the user that the element they're looking at is the root of a shadow DOM?

andreastt: I suggest we don't.

sms: I'm leaning that way too.

andreastt: You could do it yourself in JS.

eranm: If you don't have to return something, don't.

sms: My preference is to not add anything to our spec that we don't need to.

<scribe> ACTION: Write a section on handling the shadow DOM that mirrors the JS APIs [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action04]

jhammel: We should write conformance tests for this.

sms: We should do this for all parts of the spec.

Error codes - strings or numbers?

sms: At the moment we always return numbered codes. We happen to know what it means through constants.
... Great for us, since we are familiar with it. Difficult for others trying to debug the wire protocol.
... The numbers don't make any sense.
... "People who observe the wire traffic would like to understand what is going on without having to look up the numbers". Counter-argument: "They can just look it up, we carry on as now."

jhammel: Mild pro-string.
... They are human-readable, which is good.

andreastt: I'm for keeping the numbers. If we keep the numbers, we should rework whether all of these are neccessary.
... Categorize and put in groups.
... If we transfer strings, that's a lot more data.
... Then we'd need to set a limit here.

sms: You don't want the collected works of Shakespeare as a status code.
... You could use numbers in the scope protocol and translate in the shim.

andreastt: That may be overkill.

sms: If we used strings, we would use the summary field, with spaces.
... The numbers come from the original IE implementation.
... So far, we've just used the next unused number for new error codes.

andreastt: Options: 1. Use strings. 2. Use ints that are grouped. 3. Keep things as they are.

sms: This change would have to be in Selenium 3. Major, breaking change.
... We could keep the old error codes for a while.

eranm: The numbers are used across languages. A string would mean less confusion than ints (which are opaque strings).

jhammel: If we stick with numbers, we should have sensible groupings.

sms: We need to figure out what the groupings are.

andreastt: What happens if a webdriver binding doesn't have the complete list of error codes?

sms: If the local end sees an error it doesn't know, just throw the base WebDriver exception.

andreastt: Are all the errors here fatal?

sms: Yes.

andreastt: How will we treat success?

sms: Empty string or 0.

andreastt: If we decide to go with strings, do we still want a normative list of error messages?

sms: We must have a normative list of error messages. (This should be extensible, with a prefix?.)
... As vendor X, how do I add new status codes?
... Opera has 1000, MS has 2000? Or private use areas like in Unicode?
... The private use area of the unicode that we use for sendkeys to do things like down, etc, overlap with emoji.
... So far we've got away with it.
... We may have a problem.
... (This is a separate issue.)

andreastt: If we use strings, vendors can use an arbitrary strings.
... There may be collisions

sms: This is covered under vendor-specific extensions: -moz, -o. We guarantee to not have an error starting with -.
... We should probably do the same for error codes.

<eranm> wilhelm, can I propose another agenda item? It's related to the next one, about a bunch of html5-related APIs that currently exist in WebDriver and we should decide what we want to do with them.

sms: Vendor-specific extension could be: "-o Bookmark not found", "-o-Bookmark not found".
... Humans use spaces. We should use spaces if we decide to go for strings.
... I don't want to use both strings and numbers: 404 Not found

davidburns: My preference is pruned numbers.

sms: Overlap between vendor error codes is a problem. The client doesn't know which browser it uses.

kkania: I like strings.
... Only objection to strings is performance. I don't see that as a big deal.

jhammel: They are a bit English-specific.

sms: So is everything else.

andreastt: Opera leans towards strings.

JohnJansen: No opinion.

jhammel: Mozilla is on the fence.

sms: I suggest we move to strings.
... That means a new field on the response headers.
... In the spec, it will always be strings.
... If we had numbers, we need to give different numbers to different companies.
... The client side should be as generic as possible, and just works.

<scribe> ACTION: In the common case of success, the status code is left as the null value. If omitted, treat it as null. [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action05]

<scribe> ACTION: Strings replace numbers in errors [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action06]

Lunch! Woho!

Taking shit out

davidburns: We have our own definition of "window". We should align this with HTML5.

andreastt: Chrome operates with browsers as opposed to windows, no?

kkania: It's a separate browser class, not a separate process.

andreastt: In Opera you can have multiple windows with multiple tabs. (This applies to all browsers.)
... A window is a tab.

sms: The example we have now is: 1 OS-level window with 2 tabs, 1 OS-level window with 1 tab (chapter 6.3). Each tab is considered a window.

davidburns: jgraham suggested having a top-level browsing context.

sms: Is that a tab or an OS-level window?
... I think it's a tab. I'm cool with that.
... (Quouting HTML definition of top-level browsing context.)
... I'm happy with window to be top-level browsing context.

<scribe> ACTION: Windows are to be the same as a top-level browsing context (as defined in HTML) [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action07]

<sms> http://www.whatwg.org/specs/web-apps/2007-10-26/multipage/section-windows.html#parent

sms: (Quoting WebDriver spec on ordering of top-level browsing contexts across multiple OS-level windows.)

andreastt: Why is this a should?

sms: It's nice to have, but it's not required.

(Speaking of the ordering of top-level browsing contexts.)

kkania: Why have this clause at all?

sms: Unskilled testers will observe the behaviour and expect it.

andreastt: This is difficult to implement.

sms: We can choose to take as much complexity as possible close to us, and away from testers. This is difficult for us. Given the opportuinty, we should try to make the lives of testers as easy as possible.

andreastt: (Detailing complexities in keeping track of the correct order.)

kkania: For Chrome, this is easy to implement.
... Shouldn't this be a must?

sms: Should the should be a must, or do we drop the clause?

davidburns: I go for the arbitrary option (may).

andreastt: I don't get the logic behind that this is how testers think.
... I believe they expect the order they opened them.

sms: You prefer to take this section out?

andreastt: Yes.

kkania: I don't have a strong opinion.

sms: So we take out this section.
... Window-ordering is non-deterministic.

<scribe> ACTION: Remove the proposed ordering section in 6.3 [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action08]

sms: A set in Java is unordered.
... I would like to take out XPath piece, but I don't think that is a good idea.

eranm: getText?

sms: Visibility and shown text has been flagged for being moved to CSS OM. We need to discuss this with them.

<scribe> ACTION: sms to discuss with CSS WG to hand over visibility to CSS OM [recorded in http://www.w3.org/2012/10/29-testing-minutes.html#action09]

eranm: Handling of invalid SSL certificates.

sms: (Quotes spec chapter 5.2.1.)
... The default is to accept invalid certificates during testing.

davidburns: Should this be true by default?

sms: No, to address our largest audience.

davidburns: (Describing case of compromised mobile device.)

sms: This is a desired capability, not a required capability.
... A sensible default is probably to allow access to insecure HTTPS to accomodate testers without control over their environment.

davidburns: The opposite would be a secure default.

sms: WebDriver itself is an insecure entity.

andreastt: I would expect the browser to behave normally.

sms: Testers doing manual testing against these shitty environments may have saved the override to accept the invalid certificate.
... They would forget about it and expect the test to just work.
... They'd blame WebDriver.

andreastt: The test author may expect that, but not a user.
... How does Chrome work here?

kkania: There is a command-line option for ignoring these messages.

andreastt: So for Chrome, this would involve setting said command-line option?

kkania: Yes.

eranm: Users would see the message and enable the flag, given the opposite default.

sms: We get bug reports on this.

<eranm> wilhelm, my point was it doesn't make sense burdening every user with enabling this flag again and again

sms: The majority opinion is against mine. The default is secureSsl set to true.

andreastt: Our security peoplea are okay with this as long as we're running in an automated test mode.

davidburns: We're fine with it.

kkania: I'm fine with it.

sms: Default is allow insecure and local end can override that.

<jhammel> Scribe: jhammel

sms: exceptions

<trackbot> Sorry... I don't know anything about this channel

<trackbot> If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)

eranm: browser connection: if the browser is offline, it makes sense to include in the standard as this is user-controlled

davidburns: e.g. manifests

sms: some browsers get really upset if you take them offline and try to communicate with them with other processes (i.e. Firefox)

davidburns: haven't tried with marionette, but this may work better

sms: if you start the browser in offline mode you couldn't open a socket

jhammel: this might be fixed

andreastt: is this something we want?
... i support it

davidburns: we support it
... we need to find out where our boundary's finished and where another group begins
... should be supported, since a user should be able to turn off the browser

<plh> s/JohnJansen: How do you know if you're running in an automated test mode?//

<plh> s/andreastt: A flag.//

sms: how does the browser react? how do we continue controlling it?

davidburns: if we can't access them, the offline mode should redirect to those pages

davidburns: the local side should be able to continue to talk to the browser

sms: there's a difference between being offline and being able to reach into the browser

eranm: application cache: the only API we have is to figure out the state of the applicaiton cache

davidburns: the one thing, up for discussion, is clearing application cache; at the moment there is no JS input to clear application cache
... chrome has asked for it and Jonas Sicking has asked for it as well
... same as cookies
... where possible, should delegate to the web apps working group

eranm: so applicaiton cache is out of scope here?

davidburns: yes

andreastt: we have exposed the application cache for opera driver

eranm: next: local storage access

davidburns: i'm in favor of not looking after that; again, can be done with JS
... local APIs could do the heavy lifting, but it is beyond scope of webdriver
... we'd just be executing script with proper context

sms: do we allow them to setup data prior to running their tests? do we treat them like cookies?

davidburns: i'd go the cookie root

sms: example: if you're on google.com you can't setup local storage for bing.com
... but what if you need to?

andreastt: it depends on how the profile is setup

eranm: we allow modifying local storage in the same way we allow modifying cookies?

davidburns: yes

eranm: next: geolocation: api for getting current location and setting it?
... getting it is useful; setting it?

wilhelm: use-case: you want to test your app that checks for restaurants nearby

sms: setting definitely belongs in the standard

andreastt: i'm fine with exposing get as well

davidburns: from an API point of view it makes sense to have both get and set

sms: companies seriously send people out with mobile phones and get them to stand in places, and then get the data from the phone
... it was brought up: "What if you never call set?"
... it seems the actual physical location of the device is what you get back

davidburns: that sounds sane

eranm: if the location is set in a test, the browser should not respond with location confirmation

wilhelm: i load the browser and load the page that wants to find restaurants nearby
... it asks for confirmation and i deny it
... presumedly your browser has some behaviour if this happens

sms: should this be a capability

andreastt: this should not be a capability

sms: how do you deal with a popup: "Maps wants to share your location..."?
... as a user you do know this is coming; should this be treated as an alert?

andreastt: we should treat this as SSL

sms: so that's a capability
... you'd have to restart the browser to test; that seems fair
... if we've set it to false, we say "No you absolutely can't have my location"

wilhelm: it makes sense to mirror SSL
... permissive by default

andreastt: we should still keep the API for getting the settings
... what wil be the result of the actions if you're on a browser without geolocation?

sms: we won't expose that capability
... there will be a capability that says: i won't support this ability
... e.g. null for location

andreastt: we're setting a precedent for how to handle these capabilities [wrt enableSSL parity]

sms: if we're happy with that we should go for it gung ho;

<scribe> new topic: davidburns: Mozilla has a big standing against DB storage

sms: database storage has been deprecated

Philippe: the reason it disappeared is because since Microsoft and Mozilla didn't implement it, they basically killed it

sms: one of the nice things is that we can send pure javascript and have it executed

Philippe: that got replaced by indexdb
... IE10, Mozilla, Chrome are doing it

JohnJansen: it is more performant than other data storage solutions

andreastt: supported in chrome, firefox, and IE
... none of the mobile phones except ???

Philippe: switching to indexdb but not widely deployed yet

sms: i'm for holding off until someone asks for it
... does anything else have anything to discuss today?
... do we want webdriver OSS project to become the tomcat of the webdriver spec?

davidburns: your suggesting the tests be in the selenium project?

sms: they will be in the w3c, but selenium can have more tests
... we could continue the status quo
... agenda for tomorrow: writing tests

andreastt: ... HTML contacts
... canvas, svg, xhtml, xml...

sms: if its covered by the w3c spec it is fair game
... what about plain text?
... maybe text as well
... another thing that we don't handle is pdf

Philippe: [putting text in a <pre>] is part of the HTML5 spec

<plh> [adjourned]

- DRAFT -

Browser Testing and Tools WG, Monday, TPAC

29 Oct 2012

Attendees

Contents