01:05:21 RRSAgent has joined #testing 01:05:21 logging to http://www.w3.org/2013/11/11-testing-irc 01:06:10 a12u has joined #testing 01:06:23 minami has joined #testing 01:06:40 ChenJi has joined #testing 01:06:53 Meeting: Browser testing and tools WG, WebDriver spec 01:06:55 TieSun has joined #testing 01:07:01 Chair: Wilhelm 01:07:07 Scribe: MikeSmith 01:07:16 ScribeNick:: MikeSmith 01:07:29 Agenda: http://www.w3.org/wiki/WebDriver/2013-TPAC-F2F 01:07:48 Here we go! 01:07:52 cwdoh has joined #testing 01:08:10 Topic: Intro 01:08:23 wilhelm giving introductory comments 01:08:25 testing has joined #testing 01:08:36 jwjang has joined #testing 01:08:47 ijongcheol has joined #testing 01:08:57 simonstewart is here, gives self-intro 01:09:03 jwon has joined #testing 01:09:09 simonstewart is from Facebook 01:09:20 Lei_WANG has joined #testing 01:09:24 brucesolo has joined #testing 01:09:39 AutomatedTester David Burns frmo MOzilla 01:09:41 dom has joined #testing 01:09:48 Mark Fischer from Google 01:10:01 @@ from GOogle, working WebDriver 01:10:08 @@ From SkPlanet 01:10:22 Mr. Lee from Korea, @@ COmmunications 01:10:33 @@ from Korea 01:10:35 first guy, His name is hyunsuk shin. 01:10:47 @@ From Shanghai, Baidu 01:10:55 Denis from W3C 01:11:06 Isobe-san from Japan 01:11:19 Kawada-san from Japan 01:11:31 Minami-san from Toshia 01:11:39 Journalist 01:11:42 @@ 01:11:48 @@ from @@ 01:11:58 Wong Wei from Beijing 01:11:59 kennyluck_ has joined #testing 01:12:06 Erika from Microsoft 01:12:16 @@ from Tencent 01:12:20 @@ from Tencent 01:12:26 @@ 01:12:35 zcorpan has joined #testing 01:13:03 some other folks who are observers from Korea 01:13:24 Kim from Korea 01:13:43 @@ from Microsoft china 01:13:58 Mr. Lee from CSI 01:14:31 Topic: What the hell are we working on and how far along are we? 01:14:50 simonstewart describes the purpose of the WEbDriver API 01:15:04 ... which is it automate the behavior of a user 01:15:23 ... runs out of process 01:15:37 ... and runs outside of the JavaScript sandbox 01:15:46 dom_ has joined #testing 01:15:46 ... and runns "outside the glass" 01:15:56 ... can do cross-site testing, for example 01:16:05 Miao has joined #testing 01:16:19 ... and can test things that are not possible to test just with JavaScript 01:16:28 RRSAgent, make minutes 01:16:28 I have made the request to generate http://www.w3.org/2013/11/11-testing-minutes.html MikeSmith 01:17:26 simonstewart: started with Microsoft driver, then Opera driver, and Chrome driver, with some preliminary support for Android 01:17:58 ... and MOzilla, and also btw, Mozilla has a project called Marionette 01:18:05 Started with Selenium WebDriver, which is home of the current IEDriver and firefoxdriver 01:19:02 simonstewart: last f2f we had was in Boston 01:19:29 ling_chen has joined #testing 01:20:12 simonstewart: we have various language bindings 01:20:38 orange has joined #testing 01:20:40 Topic: Agenda review 01:21:01 http://www.w3.org/wiki/WebDriver/2013-TPAC-F2F 01:21:32 - Shadow DOM - Current spec API vs Future API 01:21:38 - Handling of Accelerometer 01:21:58 - Scroll to Element API 01:22:02 noriya has joined #testing 01:22:06 - Window Management 01:22:18 - Define Interactable vs Visible semantics 01:22:45 Topic: Shadow DOM 01:23:25 mark describes purpose of Shadow DOM 01:24:14 Does anyone have a spare MacBook Air video adapter? Mini-DVI to VGA, I think? 01:24:35 kimwoonyoung has joined #testing 01:24:37 kawada has joined #testing 01:24:47 i have 01:24:51 Definitely VGA. 01:24:53 sorry 01:24:57 Masahiro_ has joined #testing 01:25:00 Mac to VGa 01:25:01 ? 01:25:04 Mac to VGA 01:25:08 Yes 01:25:33 Thankyou, cwdoh! 01:25:47 mark: issue with Shadow DOM is that interacts differently in terms of how we normally do testing 01:26:07 zcorpan has joined #testing 01:26:38 Gao gives short presentation about Shadow DOM 01:27:42 hyeonseokshin has joined #testing 01:27:52 Gao: right now, we have to include a lot of verbosity in our testing code just to be able to get to Shadow DOM content 01:28:13 Interesting point about the cross-shadow DOM interactions. 01:28:26 Gao: even worse for deeply-nested shadow dom 01:28:32 Dongwon has joined #testing 01:29:08 plh has joined #testing 01:29:15 shepazu has joined #testing 01:29:16 Gao shows proposed new API, which makes the testing code look much cleaner and shorter 01:30:09 is the proposal available online for inspection? 01:30:14 Gao discusses pros and cons of new proposed api 01:30:38 sangwhan has left #testing 01:31:37 zqzhang has joined #testing 01:33:17 simonstewart: what happens when you have multiple shadow documents attached to the same shadow root? 01:33:39 mark: you get a list back 01:34:15 mark: most important thing is that a web element, once located, should continue to be usable 01:34:46 ... that's where you really get the biggest advantage in terms of the test size 01:34:54 hooney has joined #testing 01:35:25 ... the switchTo* additions are less importan 01:36:15 simonstewart: we deal with this now by stashing IDs on the equivalent of a Window 01:39:54 rhauck has joined #testing 01:40:32 simonstewart: I don't think we need switchToFrame 01:40:40 ken has joined #testing 01:41:31 Zakim, code? 01:41:39 Zakim has joined #testing 01:41:43 Zakim, code? 01:41:43 sorry, MikeSmith, I don't know what conference this is 01:42:18 sho has joined #testing 01:42:26 Zakim, call wutong 01:42:26 sorry, MikeSmith, I don't know what conference this is 01:43:18 zakim, this will be testing 01:43:18 I do not see a conference matching that name scheduled within the next hour, denis 01:43:22 zcorpan has joined #testing 01:43:37 trackbot has joined #testing 01:43:47 trackbot, start meeting 01:43:47 Sorry, but no Tracker is associated with this channel. 01:43:49 zcorpan_ has joined #testing 01:44:16 zakim, this will be test 01:44:16 I do not see a conference matching that name scheduled within the next hour, MikeSmith 01:44:44 zakim, room for 10 01:44:44 I don't understand 'room for 10', denis 01:44:47 zakim, room for 10? 01:44:49 ok, denis; conference Team_(testing)01:44Z scheduled with code 26633 (CONF3) for 60 minutes until 0244Z 01:45:11 zakim, call wutong 01:45:11 ok, denis; the call is being made 01:45:12 Team_(testing)01:44Z has now started 01:45:13 +Wutong 01:46:35 Zakim, code? 01:46:35 the conference code is 26633 (tel:+1.617.761.6200 sip:zakim@voip.w3.org), MikeSmith 01:47:36 http://www.w3.org/TR/shadow-dom/ 01:47:48 + +1.813.728.aaaa 01:47:59 ken_ has joined #testing 01:48:08 mark: similar to how content scripts work in chrome xtensions 01:48:21 Zakim, aaaa is jimevans 01:48:21 +jimevans; got it 01:49:28 wilhelm, can you please add "drag and drop of elements across frames" to the agenda? 01:49:57 simonstewart: Done. 01:50:01 Thank you 01:51:08 mizuman has joined #testing 01:52:35 simonstewart: we have decided there will be a shadow root, will have getActiveElement 01:52:49 hyeonseokshin has joined #testing 01:53:23 ... weill add an API for finding shadow roots 01:53:56 simonstewart: that was remarkably unpainful 01:54:13 ... I think that's because we don't know much about shadow dom... 01:54:34 kawada has joined #testing 01:54:45 Topic: Acceleromator 01:55:16 AutomatedTester: using the phrase Acceleromator quite loosly hear 01:55:21 s/hear/here 01:55:34 "device orientation" might be a better fit. 01:56:06 zcorpan has joined #testing 01:56:39 simonstewart: orientation of the device in 3D space 01:57:42 roll 01:57:47 http://dev.w3.org/geo/api/spec-source-orientation.html 01:59:18 ACTION: barstow start a CfC to publish LCWD of DOM Parsing and Serialization 01:59:18 Sorry, but no Tracker is associated with this channel. 01:59:30 ooops 02:00:42 RRSAgent, drop action 1 02:01:47 MikeSmith: mobile-jsonwp-spec google group 02:01:49 discussing alpha-beta-gamma vs roll-pitch-yaw 02:03:29 hooney has joined #testing 02:03:49 kawada_ has joined #testing 02:04:17 related thread: https://groups.google.com/forum/#!searchin/mobile-jsonwp-spec/stewart/mobile-jsonwp-spec/YtAIN6qYPH4/Y-b1v7QFDhcJ 02:06:52 hyeonseok has joined #testing 02:07:57 discussion about what points in maturity of other specs we start to add support in WebDriver for those specs 02:08:35 mark: not talking about shadow dom at all in the Webdriver spec would be better than what we have in the WebDriver spec now 02:09:14 wilhelm: we should move all unstable stuff to a separate draft 02:09:37 simonstewart: happy to punt shadow dom and device orientation 02:09:48 fyi http://caniuse.com/#search=orientation 02:10:06 +1 for moving 02:10:23 RRSAgent, make minutes 02:10:23 I have made the request to generate http://www.w3.org/2013/11/11-testing-minutes.html MikeSmith 02:10:36 Topic: ScrollToElement API 02:10:52 AutomatedTester: this came from some of the open-source projects 02:11:19 Who requested this again? 02:11:40 simonstewart: PageDown 02:12:03 Alexei from Selenium open-source porject brought this up 02:12:41 mark: so this API would look like, 1. find an element, then 2. scroll to it 02:13:47 what does "scroll to it" mean? scroll it to the top of the view port? the center of the view port? the bottom of the view port? something else? scroll the entire element? only the part that will fit? 02:13:54 tequila has joined #testing 02:14:02 rhauck1 has joined #testing 02:14:34 ChenJi has joined #testing 02:14:51 simonstewart: normally we say, bring the center of the element into the viewport 02:14:56 q? 02:16:06 simonstewart: we've yet to need such an API in many years 02:18:15 -jimevans 02:19:09 masahiro1 has joined #testing 02:20:06 ijongcheol has joined #testing 02:20:39 the room was in favor of not having a scroll-to-element API 02:21:53 Topic: Window management 02:22:10 AutomatedTester: this came from Alexei 02:22:25 jy has joined #testing 02:22:48 ... currently the API requires that we pass through a window handle (unique key) but we don't use it 02:22:54 simonstewart: we do use it 02:23:33 AutomatedTester: take a look at the resize command 02:25:04 discussion indicates we might have a bug in the *Driver implementations 02:25:54 s/*Driver// 02:29:25 45 minutes break 02:29:38 we will back at 11:15 local time 02:29:49 simonstewart has joined #testing 02:29:56 cwdoh has joined #testing 02:30:03 ken has joined #testing 02:30:44 mizuman has joined #testing 02:33:13 plh3 has joined #testing 02:34:26 -Wutong 02:34:27 Team_(testing)01:44Z has ended 02:34:27 Attendees were Wutong, +1.813.728.aaaa, jimevans 02:38:55 Sam_Lin has joined #testing 02:39:23 rufusding has joined #testing 02:39:32 tequila_ has joined #testing 02:40:57 taocai has joined #testing 02:43:00 myakura_ has joined #testing 02:51:37 taocai has left #testing 02:53:16 cwdoh has joined #testing 02:53:40 tequila_ has left #testing 02:54:41 kimwoonyoung has joined #testing 03:00:11 ijongcheol has joined #testing 03:03:36 cwdoh has joined #testing 03:04:45 denis has joined #testing 03:05:49 Yuanyan_Cao has joined #testing 03:08:32 simonstewart has joined #testing 03:10:57 AutomatedTester has joined #testing 03:11:48 kimwoonyoung has joined #testing 03:14:36 Sam_Lin has joined #testing 03:14:43 Hey 03:16:12 ijongcheol has joined #testing 03:16:17 kimwoonyoung has joined #testing 03:16:35 ijongcheol has joined #testing 03:17:25 ShuotaoGao has joined #testing 03:18:35 mimami has joined #testing 03:19:03 yuanyan has joined #testing 03:19:06 masahiro has joined #testing 03:20:14 rhauck has joined #testing 03:20:30 kimwoonyoung has joined #testing 03:20:46 RRSAgent, draft minutes 03:20:46 I have made the request to generate http://www.w3.org/2013/11/11-testing-minutes.html wilhelm 03:21:02 RRSAgent, make log public 03:21:22 RRSAgent, publish minutes 03:21:22 I have made the request to generate http://www.w3.org/2013/11/11-testing-minutes.html wilhelm 03:21:24 cwdoh_ has joined #testing 03:22:08 edoyle has joined #testing 03:22:53 rhauck1 has joined #testing 03:23:21 ken has joined #testing 03:25:28 rufusding has joined #testing 03:26:00 Scribe: wilhelm 03:26:06 Chair: simonstewart 03:26:20 Topic: Define interactable vs visible semantics 03:26:37 kimwoonyoung has joined #testing 03:26:38 simonstewart: We discussed this at the previous F2F. That hurt quite a lot. 03:26:44 ... We should nail this down now. 03:26:47 Sam_ has joined #testing 03:27:18 AutomatedTester: One of the things that come up regularly has to with viewport sizes, where elements end up on top of each other. 03:27:20 yuanyan has joined #testing 03:27:37 ... It's possibly just a Gecko bug. If you fire an event, it sometimes fires through an element. 03:27:51 ... Technically, it's visible. 03:27:57 simonstewart: Like the clickjacking problem? 03:27:58 AutomatedTester: Yes. 03:28:02 minami has joined #testing 03:28:05 AutomatedTester: If you tap it, it would do nothing. 03:28:12 .... When automating, it sometimes works, sometimes doesn't. 03:28:18 taocai has joined #testing 03:28:36 mizuman has joined #testing 03:28:37 ... When something has overflowed, not in same part of the DOM tree, but overlapping. 03:28:48 ... There's problems like that. 03:29:22 masahiro has joined #testing 03:29:27 (Discussion on specific case where an element is visible not not interactable.) 03:30:01 simonstewart: If you could use tab ordering to get to the element, you'd be okay. 03:30:14 mizuman_ has joined #testing 03:30:29 simonstewart: A problem that a naive implementation of most of the inputs would just find the element and send events to it. 03:30:40 ... More advanced implementation: Find the coordinates, interact with those. 03:30:49 myakura has joined #testing 03:30:53 Marc: Newer versions if Firefox does this. 03:31:10 simonstewart: FFDriver with native events does this. OS-level equivalent events. 03:31:18 AutomatedTester: Marionette gets this slightly wrong. 03:31:25 simonstewart: IEdriver does the expected thing. 03:31:59 simonstewart: I think visibility, even if obsucred by z-ordering, you're technically visible. 03:32:10 simonstewart: "If you're partly obscured, are you visible?" 03:32:20 Marc: The primary use of this is to determine if you can click something. 03:32:30 simonstewart: We have clickjacking tests for this in the Selenium test suite. 03:32:44 darobin has joined #testing 03:32:52 a12u has joined #testing 03:33:05 MikeSmith: I have jimevans on skype 03:33:09 simonstewart: It might be reasonable to go: we will do our best to interact with this element, but overlapping elements may intercept the element. 03:33:12 all set thanks 03:33:29 zqzhang has joined #testing 03:33:42 Marc: You want it to click the element you want it to click, or say it can't. 03:33:45 ... And error is fine. 03:33:57 Marc: Should we also have a predicate to check if ... 03:34:07 simonstewart: How often do you see problems with this? 03:34:17 Marc: If we take our FF tests and run them in Chrome. 03:34:22 ... Tests start failing. 03:34:34 Marc: Some of this is bad tests. 03:34:49 zqzhang has joined #testing 03:35:00 kennyluck has joined #testing 03:35:12 Marc: They wait until something is displayed, so you should be able to click something. 03:35:15 ... Not sufficient. 03:35:32 ... isInteractable? makes more sense. 03:35:37 AutomatedTester: This matches our use case. 03:35:49 AutomatedTester: (Describes this wrt. animations.) 03:36:12 simonstewart: The problem you're talking about sounds like the animation hasn't finished yet when you ask isVisible?. 03:36:35 simonstewart: THe problem is waiting for the animation to finish. Separate use case. 03:36:45 AutomatedTester: isDisplayed will say true for both cases. 03:37:09 AutomatedTester: Top one is visible and interactable. Bottom is visible, but not interactable. 03:37:20 simonstewart: ChromeDriver imeplementation makes sense. 03:38:01 Marc: You'll have to wait for element on top to be gone. 03:38:19 ... You don't care about that element. You care about the element underneath. 03:38:29 When can I click on A? 03:38:51 ... Your test should not depend on anything else than the element you care about. 03:39:07 simonstewart: Why doesn't your test author know about the covering lightbox? 03:39:31 rhauck has joined #testing 03:39:50 simonstewart: You want your tests breaking with a new, unknown lightbox. 03:39:59 rhauck has joined #testing 03:40:34 simonstewart: Something that dismisses itself should be easily detectable 03:40:53 Marc: The next time something new is added, you must update all your tests. 03:41:05 simonstewart: If it never? 03:41:10 Marc: If it never happens, you time out. 03:41:26 masahiro has joined #testing 03:41:33 abarsto has joined #testing 03:41:33 simonstewart: I'm trying to think of a case where you shouldn't need to edit your test. 03:41:48 ... I see why a new API is convenient, but not compelling... 03:42:10 simonstewart: Use case of "I don't know if my animations are finished" 03:42:42 zcorpan has joined #testing 03:43:13 simonstewart: If an element is covered by something in z-order, is it visible? 03:43:17 ... I say yes. 03:43:30 simonstewart: What if only one pixel is visible? 03:43:45 Marc: How do you deal with opacity, etc? 03:44:05 AutomatedTester: A user trying to do this: "I can see that element, it it visible" 03:44:45 AutomatedTester: If find the element, bounding box... 03:44:51 simonstewart: You'd need that for ever element in that area. 03:44:55 ... Very expensive. 03:45:23 simonstewart: CSS shapes? 03:45:25 masahiro_ has joined #testing 03:45:33 AutomatedTester: I agree. 03:45:55 AutomatedTester: On mobile, people check if something is visible. 03:46:24 Marc: They're trying to test if something is hidden by another element. 03:46:28 ... We cant do that well. 03:46:41 Marc: Screenshot diffing may be better. 03:47:01 simonstewart: Seems like a super-specialized use case. 03:47:26 AutomatedTester: Transforms are super-lightweight. 03:47:49 simonstewart: (Describing why this is still expensive.) 03:47:57 AutomatedTester: But people do it. 03:48:12 rufusding has joined #testing 03:48:12 simonstewart: I don't believe this happens as often as you think. 03:48:18 AutomatedTester: Disagrees. 03:48:24 plh has joined #testing 03:48:27 simonstewart: Then we should define this in a later version of the spec. 03:48:40 yuanyan has joined #testing 03:49:13 (Discussion on element at point.) 03:49:46 AutomatedTester: Element at point does not support opacity. 03:49:51 simonstewart: And transforms, shapes. 03:50:02 simonstewart: Do we want to add this additional complexity? Already very expensive. 03:50:11 ... To support use case of an inefficient transform. 03:50:32 simonstewart: This will be used by getText. 03:50:42 zcorpan has joined #testing 03:50:54 AutomatedTester: If we do responsvive design. Something got really big, things start overlapping as it gets smaller... 03:51:15 ... What happens in this case? 03:51:25 simonstewart: If you can scroll to it, gettext would return the content. 03:51:44 simonstewart: Done at the block level. 03:51:51 ... We have never paid attention to viewport size. 03:52:26 simonstewart: If a bit of text was visible, we would return all the text. 03:52:47 simonstewart: We are verging on having to write an AI for the visibility tests... 03:52:52 :) 03:52:54 AutomatedTester: Agreed. 03:52:57 mizuman has joined #testing 03:53:15 AutomatedTester: Your average developer will go "I don't understand why this is visible/invisible" 03:53:28 AutomatedTester: "I can see this element, it must be visible" 03:53:44 Marc: "I can't see it, it should not be visible." 03:53:59 simonstewart: It's not visible, but displayed. 03:54:42 AutomatedTester: Did we change the spec? The wording is "visibility". 03:54:50 simonstewart: We should change the section name. 03:55:15 ACTION: Rename section 10.1: determining visibility > determining displayed 03:55:15 Sorry, but no Tracker is associated with this channel. 03:55:23 trackbot, bye 03:55:23 trackbot has left #testing 03:56:18 simonstewart: Conclusion: rename section, we don't try to count all the crazy things try to do. 03:56:33 AutomatedTester: Is there a better word? 03:56:59 AutomatedTester: So many sonynyms. 03:57:07 s/sonynyms/synonyms 03:57:25 simonstewart: isDIsplayed matches people's expectations. 03:57:30 Marc: Matches CSS. 03:57:56 simonstewart: There will always be edge cases. "Opacity of 1%..." 03:58:03 ... Not adding options for fuzzy boundaries. 03:58:30 simonstewart: We want to give this part of the away spec away to some other group. For example CSS. 03:58:47 simonstewart: It touches both DOM and CSS, so they don't want to take it. 03:58:55 ... Maybe we can punt it to Accessibility? 03:59:18 ACTION: Discuss isDisplayed with Accessibility groups 03:59:34 AutomatedTester: Other issues with disabled elements in an AT. 04:00:21 simonstewart: If you have overflow hidden, and it's pushed out of its container... 04:00:53