IRC log of testing on 2013-11-12

Timestamps are in UTC.

01:10:40 [RRSAgent]
RRSAgent has joined #testing
01:10:40 [RRSAgent]
logging to http://www.w3.org/2013/11/12-testing-irc
01:11:50 [wilhelm]
Meeting: Browser Tools and Testing WG, WebDriver spec
01:11:54 [wilhelm]
RRSAgent, draft minutes
01:11:54 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/11/12-testing-minutes.html wilhelm
01:12:11 [wilhelm]
RRSAgent, make logs public
01:13:13 [a12u]
a12u has joined #testing
01:14:03 [kawada]
kawada has joined #testing
01:14:17 [simonste_]
simonste_ has changed the topic to: Day 2 of WebDriver: interactions APIs, timelines, v1.1
01:14:19 [ISL]
ISL has joined #testing
01:15:16 [kimwoonyoung]
kimwoonyoung has joined #testing
01:18:40 [wilhelm]
(We're waiting for the last laggards, starting a little later than planned.)
01:18:52 [TieSun]
TieSun has joined #testing
01:19:01 [rhauck]
rhauck has joined #testing
01:20:50 [ShuotaoGao]
ShuotaoGao has joined #testing
01:21:06 [mizuman]
mizuman has joined #testing
01:21:22 [simonstewart]
scribe: simonstewart
01:21:25 [garykac]
garykac has joined #testing
01:21:30 [wilhelm]
Chair: wilhelm
01:21:34 [wilhelm]
RRSAgent, draft minutes
01:21:34 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/11/12-testing-minutes.html wilhelm
01:22:00 [wilhelm]
Scribe: simonstewart
01:22:07 [wilhelm]
RRSAgent, draft minutes
01:22:07 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/11/12-testing-minutes.html wilhelm
01:22:19 [kennyluck]
kennyluck has joined #testing
01:22:31 [simonstewart]
Introductions
01:22:46 [simonstewart]
MikeSmith asks if there's anyone new
01:22:46 [kennyluck]
kennyluck has joined #testing
01:23:07 [simonstewart]
garykac Gary Kackmarcik
01:23:16 [simonstewart]
garykac from Google
01:23:32 [simonstewart]
s/Kackmarcik/Kacmarik/
01:23:33 [zqzhang_]
zqzhang_ has joined #testing
01:23:38 [rhauck1]
rhauck1 has joined #testing
01:23:47 [simonstewart]
s/Kacmarik/Kacmarcik/
01:23:53 [garykac]
s/Kacmarik/Kacmarcik/ ^_^
01:23:58 [simonstewart]
@@ interest in console api
01:24:01 [MikeSmith]
i/Introductions/Topic: Introductions/
01:25:03 [rhauck]
rhauck has joined #testing
01:25:08 [sho]
sho has joined #testing
01:25:21 [simonstewart]
MikeSmith: no-one's working on the console api right now, but if we had an editor
01:25:35 [simonstewart]
More introductions (same as yesterday)
01:26:15 [simonstewart]
wilhelm, MikeSmith, MarcFisher
01:26:31 [simonstewart]
Observer introductions
01:26:32 [MikeSmith]
s/@@/Alois Reitbauer -
01:26:40 [MikeSmith]
RRSAgent, make minutes
01:26:40 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/11/12-testing-minutes.html MikeSmith
01:26:46 [MikeSmith]
_win 27
01:27:00 [MikeSmith]
s/_win 27//
01:27:04 [MikeSmith]
RRSAgent, make minutes
01:27:04 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/11/12-testing-minutes.html MikeSmith
01:27:17 [simonstewart]
wilhelm looks up meeting agenda
01:27:28 [simonstewart]
First topic: interactions
01:27:40 [wilhelm]
http://www.w3.org/wiki/WebDriver/2013-TPAC-F2F
01:28:10 [rhauck2]
rhauck2 has joined #testing
01:28:21 [simonstewart]
garykac: the components that the DOM spec needs.
01:28:37 [simonstewart]
Mouse is pretty simple: just need mouse move, over, etc. Seems straight forward
01:29:08 [simonstewart]
Keyboard events: want to be able to test US ASCII, but also international keybaord. Want to test "given a layout, what events are generated"
01:29:10 [MikeSmith]
s/MikeSmith/MikeSmith, Chris Gao/
01:29:20 [simonstewart]
Being able to automatically change layout is desirable.
01:29:25 [simonstewart]
Also generate dead keys
01:29:33 [simonstewart]
Also creating IME composition events
01:30:20 [simonstewart]
Those are the big things. Undoubtedly need more
01:30:22 [simonstewart]
:)
01:30:35 [wilhelm]
Scribe: wilhelm
01:30:53 [wilhelm]
simonstewart: Mouse is pretty easy.
01:31:09 [wilhelm]
... There's a limited range of APIs that we're expecting to generate.
01:31:19 [wilhelm]
... click, dbclick, contextclick, mousedown, mousedown.
01:31:22 [wilhelm]
... OS-level.
01:31:32 [wilhelm]
... These events are on the other side of the glass.
01:31:41 [wilhelm]
.... How does the browser process all these things?
01:31:49 [wilhelm]
... We discussed yesterday scroll wheel.
01:32:06 [wilhelm]
garykac: Would that include ticks? There's pixels and ticks.
01:32:10 [wilhelm]
simonstewart: Undecided.
01:32:31 [wilhelm]
simonstewart: Keyboard: presskey, releasekey, send a string of characters
01:32:35 [AutomatedTester]
AutomatedTester has joined #testing
01:32:42 [wilhelm]
... Just a charsequence.
01:32:55 [wilhelm]
... With webelement.sendkeys, we allow people to send internationalized text
01:33:04 [wilhelm]
garykac: For Japanese, I don't want to send the full string.
01:33:20 [wilhelm]
simonstewart: The commands on webelement are do-as-I-mean.
01:33:35 [wilhelm]
... The advanced actions are do-as-I-say.
01:33:54 [wilhelm]
simonstewart: Initial implementation just blasted the value in.
01:34:12 [wilhelm]
... To do the second case, there is the ability to have a IME engine in the open source web driver.
01:34:40 [wilhelm]
MarcFisher: There are functions to call, but they are not neccessarily working.
01:34:44 [wilhelm]
simonstewart: IME-handler
01:34:51 [wilhelm]
... Will list the available engines.
01:35:12 [wilhelm]
... Your test will be very brittle, as it relies on a specific IME engine.
01:35:15 [wilhelm]
garykac: Scan code?
01:35:35 [wilhelm]
simonstewart: Initial implementation does some calculation to figure out the scan code. Specify A, queries the OS.
01:35:44 [wilhelm]
garykac: This needs to be controlled in the tests.
01:35:52 [wilhelm]
garykac: I want to test A on a French keyboard...
01:36:07 [wilhelm]
simonstewart: We're not an OS level automation API, but browser. We'll take advantage of the machine.
01:36:36 [wilhelm]
... For testing multiple keyboards, you may need multiple VMs.
01:36:45 [wilhelm]
garykac: That will not work.
01:37:00 [wilhelm]
simonstewart: I'm not aware of any cross-platform IME...
01:37:18 [wilhelm]
garykac: I thought we'd have OS specific components.
01:37:26 [wilhelm]
simonstewart: And we attempt to mask that.
01:37:45 [wilhelm]
simonstewart: ChromeDriver, Marionette inject events right into the event queue.
01:37:58 [wilhelm]
... Looks like it comes from the OS.
01:38:07 [wilhelm]
... FFdriver and IE on Win uses Win32 APIs.
01:38:13 [wilhelm]
... Error-prone.
01:38:23 [wilhelm]
... The OS makes some assumptions about focus of windows.
01:38:38 [wilhelm]
.... For accurate emulation, you need to go through the Win32 APIs.
01:38:48 [wilhelm]
... These tests take a long time to run.
01:39:01 [wilhelm]
... Must be able to run without focus, for parallell processing.
01:39:25 [wilhelm]
... spec has three audiences: People writing tests for their web apps.
01:39:30 [wilhelm]
... Browser vendors.
01:39:51 [wilhelm]
... Spec editors.
01:39:57 [miao-cesi]
miao-cesi has joined #testing
01:40:15 [wilhelm]
... Low-level, cross-platform method is incredibly challenging.
01:40:26 [wilhelm]
... TBH, we don't ahve the expertise around this table.
01:40:29 [wilhelm]
... Maybe you.
01:40:50 [wilhelm]
garykac: Injecting keyboard events into @@ is something that we do.
01:41:01 [wilhelm]
simonstewart: Unfocused windows?
01:41:11 [wilhelm]
garykac: Win yes, Mac no.
01:41:26 [wilhelm]
simonstewart: Mac has a mouse window and keyboard window.
01:41:38 [wilhelm]
garykac: I have less concern about the machine doing something else.
01:41:56 [wilhelm]
... Interop between browser vendors. Difficult to run tests will not be run.
01:42:03 [wilhelm]
... You inject cooked browser events?
01:42:07 [wilhelm]
simonstewart: Depends on the broser implementation.
01:42:21 [wilhelm]
... IE: three methods. Synthetitc JS events.
01:42:25 [wilhelm]
... A library I wrote.
01:42:38 [wilhelm]
... Attempts to put raw events into the Windows message queue.
01:42:45 [wilhelm]
... Scan codes, et.
01:42:47 [wilhelm]
c
01:42:59 [wilhelm]
...3: Use something like sendkeys.
01:43:13 [wilhelm]
garykac: Concern: Translation from OS events...
01:43:43 [wilhelm]
garykac: Given this sequence on Windows, all browsers should get the same.
01:43:48 [wilhelm]
simonstewart: Out of scope?
01:44:06 [wilhelm]
dburns: In the way we insert trusted events into the browser, we depend on library in browser.
01:44:10 [wilhelm]
... Mobile, desktop.
01:44:21 [wilhelm]
simonstewart: Mobile is maybe the best case.
01:44:36 [wilhelm]
simonstewart: Uses accessibility API.
01:44:42 [wilhelm]
... Good emulation by accident.
01:44:45 [glenn]
glenn has joined #testing
01:44:50 [wilhelm]
simonstewart: Suggestions on how this can be done?
01:45:06 [wilhelm]
simonstewart: We don't want to give the browser window OS focus.
01:45:22 [wilhelm]
garykac: There's parts of it that can be used.
01:45:32 [wilhelm]
garykac: Mouse will be useful.
01:45:51 [wilhelm]
garykac: If we're not generating the keys on the OS level, that might not fit.
01:45:54 [wilhelm]
simonstewart: Huge problem.
01:46:03 [wilhelm]
garykac: Huge because it's lots of small things.
01:46:13 [wilhelm]
simonstewart: And you've got at least two axes. OS, browser.
01:46:23 [wilhelm]
simonstewart: IME, third axis.
01:46:32 [wilhelm]
garykac: Different keyboard layouts.
01:46:39 [wilhelm]
simonstewart: French dvorak...
01:46:39 [glenn]
glenn has joined #testing
01:46:48 [wilhelm]
garykac: SOme keys are not exposed...
01:47:01 [wilhelm]
garykac: Writing OS specific tests must happen for this
01:47:19 [wilhelm]
simonstewart: Does Win32 sendkeys do what you need?
01:47:47 [wilhelm]
garykac: We sometimes want to know what the physical key is. Need more information.
01:48:23 [wilhelm]
RRSAgent, draft minutes
01:48:23 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/11/12-testing-minutes.html wilhelm
01:48:52 [wilhelm]
simonstewart: The IME implementation we had worked on Windows and Linux.
01:49:05 [wilhelm]
simonstewart: You could take these from the project.
01:49:12 [AutomatedTester]
https://code.google.com/p/selenium/source/browse/#git%2Fcpp%2Fimehandler
01:49:21 [AutomatedTester]
https://code.google.com/p/selenium/source/browse/#git%2Fcpp%2Fwebdriver-interactions
01:49:38 [AutomatedTester]
https://code.google.com/p/selenium/source/browse/cpp/webdriver-interactions/interactions.cpp
01:50:33 [wilhelm]
simonstewart: *explains code just posted above*
01:50:55 [wilhelm]
simonstewart: *explains relationship between Selenium / WEbDriver*
01:52:02 [wilhelm]
simonstewart: You can use WebDriver to get the browser to a specific palce.
01:52:28 [wilhelm]
garykac: Can it check the OS state?
01:52:31 [wilhelm]
simonstewart: No
01:53:14 [wilhelm]
simonstewart: *proposes a mixture of WebDriver + native OS code*
01:53:30 [wilhelm]
simonstewart: If this was easy, we'd have done it.
01:53:35 [wilhelm]
garykac: I want to reuse as much as I can.
01:53:59 [wilhelm]
garykac: Where are the live, valid sets of tests?
01:54:43 [wilhelm]
wilhelm: web-platform-tests
01:54:51 [wilhelm]
simonstewart: If you have suggestions for teh API, given our constraints.
01:55:05 [wilhelm]
simonstewart: *lists currrent API*
01:55:14 [wilhelm]
garykac: Everything else comes from that.
01:55:28 [wilhelm]
simonstewart: Using PUA from Unicode to specifiy other characters.
01:55:33 [wilhelm]
... Return key, etc.
01:55:39 [wilhelm]
... Based on meaning.
01:55:47 [wilhelm]
garykac: DOM3 har those based on meaning.
01:56:07 [wilhelm]
... UI events has keynames based on the physical key.
01:56:34 [wilhelm]
... This key, Q, on English keyboard, is the same no matter which keyboard layout you have.
01:56:52 [simonstewart]
http://selenium.googlecode.com/git/docs/api/java/org/openqa/selenium/Keys.html
01:57:02 [wilhelm]
... If you have to give the name to a physical key, use UI events.
01:57:36 [wilhelm]
simonstewart: *refers to the key list from the implementation*
01:57:40 [garykac]
UI Events: http://www.w3.org/TR/uievents/
01:58:00 [AutomatedTester]
https://code.google.com/p/selenium/wiki/JsonWireProtocol#/session/:sessionId/element/:id/value
01:58:42 [wilhelm]
garykac: Is this all?
01:58:45 [wilhelm]
simonstewart: Non-printable.
01:59:03 [wilhelm]
garykac: No location field to indicate it's from the keypad?
01:59:14 [wilhelm]
MarcFisher: Just a string.
01:59:14 [glenn]
glenn has joined #testing
01:59:46 [wilhelm]
garykac: *ask about specific keys in the list*
01:59:52 [wilhelm]
MarcFisher: suggestions for improving the API are welcome.
02:00:08 [wilhelm]
garykac: We've worked on speccing the list of keys in DOM3.
02:00:55 [mizuman]
mizuman has joined #testing
02:01:24 [garykac]
DOM3 Events: http://www.w3.org/TR/DOM-Level-3-Events/
02:01:27 [simonstewart]
http://www.w3.org/TR/DOM-Level-3-Events/#key-value-tables
02:01:30 [garykac]
Section 6.3
02:01:31 [wilhelm]
garykac: Section 6.3
02:01:48 [wilhelm]
garykac: I'm going to be merging the UI events stuff into this.
02:01:55 [wilhelm]
... SO that there is one canonical document.
02:01:56 [mizuman]
mizuman has joined #testing
02:02:12 [wilhelm]
garykac: UI events provides more info on how awful keyboards are...
02:02:27 [wilhelm]
... This maps to Windows virtual keys.
02:02:35 [wilhelm]
garykac: Assumes we have a location field.
02:02:41 [wilhelm]
... This is only the non-printable.
02:03:03 [wilhelm]
garykac: Separator?
02:03:19 [wilhelm]
garykac: This also includes old keys.
02:03:30 [wilhelm]
garykac: Media keys.
02:04:24 [wilhelm]
garykac: You may want to refer to this and say "this is where we get our values from".
02:04:31 [wilhelm]
garykac: You may need location feels.
02:04:40 [wilhelm]
s/feels/fields
02:05:11 [wilhelm]
garykac: This completely ignores Android + FirefoxOS, which we also need.
02:05:44 [wilhelm]
simonstewart: So far we haven't needed most of these keys...
02:06:26 [wilhelm]
garykac: Thank you for the information!
02:06:34 [wilhelm]
simonstewart: I've learned a lot about the horror that is keyboards.
02:06:52 [glenn_]
glenn_ has joined #testing
02:07:00 [MichaelC]
MichaelC has joined #testing
02:07:22 [wilhelm]
simonstewart: Reviews of our sections on this would be very helpful.
02:08:14 [wilhelm]
ACTION: Add scrollwheel support
02:08:58 [wilhelm]
Topic PFWG
02:09:05 [wilhelm]
Michael Cooper, staff contact
02:09:11 [wilhelm]
@@, observer
02:09:17 [wilhelm]
@@, with IBM
02:10:06 [wilhelm]
s/Topic/Topic: PFWG
02:10:16 [taocai]
taocai has joined #testing
02:10:28 [wilhelm]
MichaelC: A number of us were here at TTWF. Some of this may be familiar.
02:10:37 [wilhelm]
... We want to sync the accessibility testing with other testing.
02:10:43 [wilhelm]
... THis is an iteration of these efforts.
02:10:57 [wilhelm]
... IN the past few days we've got a better understanding.
02:11:22 [wilhelm]
... Suggested agenda items:
02:11:43 [wilhelm]
... Developing accessbility tests from the point of view of a developer who wants to ensure accessibility in the UA.
02:11:48 [wilhelm]
... We test tools, content.
02:12:10 [wilhelm]
.. We talked about how WebDriver can be used to test web pages in UA to see if accessibility features work as intended.
02:12:19 [wilhelm]
... May depend on things from WebDriver.
02:12:29 [wilhelm]
... How can we use WD to test accessibility tests?
02:12:39 [wilhelm]
... We develop ARIA, which could be tested that way.
02:13:06 [wilhelm]
... Considering accessibility tests that can be done like the testharness.js, reftests...
02:13:19 [wilhelm]
.. If we wish to add tests?
02:13:24 [wilhelm]
wilhelm: tobie's department.
02:13:40 [wilhelm]
MichaelC: Mapping accessibility APIs to testing...
02:13:45 [wilhelm]
... Accessibility vs DOM.
02:14:18 [wilhelm]
Janina: The best defined are the UA interfaces to accessibility API.
02:14:22 [wilhelm]
.. On their way to Rec.
02:14:35 [wilhelm]
... We're looking at that approach for basis for furhter work.
02:14:54 [wilhelm]
MichaelC: What level of shared knowledge do we have?
02:15:27 [wilhelm]
simonstewart: *describes the purpose and functionality of WebDriver*
02:17:32 [glenn]
glenn has joined #testing
02:17:33 [wilhelm]
@@: JS in the browser?
02:17:35 [wilhelm]
simonstewart: No.
02:17:56 [wilhelm]
@@: Are you using Windows automation?
02:18:00 [wilhelm]
simonstewart: It depends.
02:18:22 [wilhelm]
... We need to be able to test browsers without window having focus.
02:18:28 [wilhelm]
... We should be able to background the window.
02:18:47 [wilhelm]
... (Describes the three types of events and their implementations.)
02:20:50 [wilhelm]
Cynthia: Assisstive technologies use OS accessibility layers to interact with f.x. UA.
02:21:13 [wilhelm]
... Old MS API, IBM API, newer MS Windows automation...
02:21:40 [wilhelm]
... Windows UI automation was designed with a couple of ideas that are different.
02:21:50 [wilhelm]
... To be used as a test driver in addition to accessibility.
02:21:56 [wilhelm]
... Combine patterns. Invoke, selection.
02:22:08 [wilhelm]
... "This is an invokable thing, not just a button"
02:22:17 [wilhelm]
... You can combine patterns.
02:23:08 [wilhelm]
MichaelC: The accessibility APIs do similar things, but vary between OSes.
02:23:11 [darobin]
darobin has joined #testing
02:23:25 [wilhelm]
Janina: Linux accessibility layer used for testing Linux desktop.
02:23:42 [simonstewart]
What was the name of the linux project that uses accessibility testing?
02:23:52 [zcorpan]
zcorpan has joined #testing
02:24:18 [wilhelm]
Cynthia: You're doing platform-specific work.
02:24:24 [wilhelm]
simonstewart: Yes, but hidden behind the APIs.
02:25:03 [wilhelm]
Cynthia: We look at the markup in ARIA or HTML and our document, "this document should have these properties".