01:30:08 RRSAgent has joined #html5perftesting 01:30:09 logging to http://www.w3.org/2015/10/28-html5perftesting-irc 01:30:15 Zakim has joined #html5perftesting 01:31:25 dwim_ has joined #html5perftesting 01:31:52 zakim, this is HTML5 Testing TPAC Break Out 01:31:52 got it, xiaoqian 01:32:11 chair: Kepeng Li 01:32:31 script: xiaoqian 01:33:27 nice to see you, xiaoqian 01:34:13 s/nice to see you, xiaoqian// 01:34:35 zqzhang has joined #html5perftesting 01:35:30 gsnedders has joined #html5perftesting 01:36:11 jiangtao has joined #html5perftesting 01:36:39 myakura has joined #html5perftesting 01:37:37 zephyr has joined #html5perftesting 01:38:28 hax has joined #html5perftesting 01:38:38 present+ jiangtao 01:38:38 chair: chaals 01:39:27 present+ zqzhang 01:39:43 JohnJansen has joined #html5perftesting 01:39:59 present+ myakura 01:40:04 QingAn has joined #html5perftesting 01:40:18 judy-zhu has joined #html5perftesting 01:40:53 present+ JohnJansen 01:41:06 jgraham has joined #html5perftesting 01:41:12 present+ gsnedders 01:41:13 present+ jgraham 01:41:30 peterw-melco has joined #html5perftesting 01:41:31 present+ dwim 01:41:49 AndreyR has joined #html5perftesting 01:42:00 dbaron has joined #html5perftesting 01:42:17 Andrey Rybka bloomberg 01:42:52 zqzhang: Zhiqiang Zhang, Intel 01:42:55 ivan has joined #html5perftesting 01:42:59 present+ dwim_samsung 01:43:11 present+ zephyr 01:43:34 [Presentation by Kepeng_Alibaba] 01:43:36 mchampion has joined #html5perftesting 01:45:15 Takeharu has joined #html5perftesting 01:45:24 SAM has joined #html5perftesting 01:45:31 ko has joined #html5perftesting 01:46:06 Kepeng: current issues in our opinion: 1) no frozen and complete version for compatibility testing; 2) lack of HTML5 Perf evaluate 01:46:19 kokabe has joined #html5perftesting 01:46:42 ... proposals: 1) create a HTML5 Testing CG 01:47:16 ... to create a manifest file for those test for 5.0 REC features 01:48:07 ... benefits: 1) for vendors, some of them wanted to know if their platform is aligned with the 5.0REC 01:48:21 2) for Web developers 01:48:43 ... Timeline 01:48:54 s/2)/... 2) 01:49:39 chaals has joined #html5perftesting 01:49:42 james_moz: James from Moz, done a lot of work on w-p-t 01:49:52 ... concerned about this idea 01:50:02 present+ chaals 01:50:11 ... as browser vendors, no likely we will be interested 01:50:28 q+ 01:50:31 ... we would like to have the latest features as we run the whole suite 01:50:46 ... it's ok to if we haven't implemented it 01:51:33 ... f.ex. Service Workers. Though it's not ready in the browsers, but it's helpful to have it in the w-p-t 01:52:29 ... for web devs, when you write a webapps, not likely they will be interested if those feature are in the browsers or not 01:52:34 q+ 01:53:07 q+ 01:53:10 ... there's often a desire that the spec can tell you what you can use 01:53:35 pjeong_ has joined #html5perftesting 01:53:49 q+ 01:53:58 yingying has joined #html5perftesting 01:54:19 q+ 01:54:42 paul_ms: when we shipped HTML5, the WG went thought a list of features that were not implemented yet, we didn't create a compatibility test suite 01:55:06 chaals_yandex: agree with James, why a frozen set is important? 01:55:43 ... features, spec drafts, and implementation change very often 01:55:52 mchampion has joined #html5perftesting 01:55:58 ... that won't work along with the timeline 01:56:05 q? 01:56:56 q+ cynthia 01:57:00 ToddReifsteck has joined #html5perftesting 01:57:01 ... if we have automative tests, f.ex. accessibility test, it's not there yet 01:57:19 ... why do we need to freeze the test set? 01:57:34 q+ paulc 01:57:52 joanie has joined #html5perftesting 01:57:53 cyns has joined #html5perftesting 01:58:00 q+ 01:58:17 gsnedders: specs won't be always stable, which will cause changes to test cases 01:58:36 ... don't think a frozen set is useful 01:59:34 Judy_alibaba: it's a proposal raised by the Chinese companies, I'll like to hear their issues 01:59:47 ack 01:59:56 q? 02:00:25 chaals: a lot of Chinese members have resource and talent that we can use 02:01:13 james: a bit of push back for the frozen proposal, but really positive to see contribution to w-p-t 02:01:53 q? 02:01:58 Judy: [going through the issues again] 02:03:38 Kepeng: frozen might not be the right word, some people might want to know if the browser are aligned with the spec 02:04:07 chaals: there are a lot of other browsers than the 4 major ones 02:04:53 judy: for alibaba, we want to know if our application are aligned with HTML5 02:05:50 q? 02:06:17 zqzhang_intel: at Intel we have a cross-walk project, our pratice is based on Blink with some other new features 02:06:54 ... we have some requirement for frozen feature 02:06:55 I hear 2 things browser and app. testing if browser are you asking for something like this https://html5test.com? and http://www.webpagetest.org for apps? 02:07:13 ... some test cases are create for the REC 02:07:53 ... I don't see mass requirement from us 02:07:54 AndreyR: you really want to look at web-platform-tests and ignore html5test, because it doesn't mean much 02:08:07 Judy: qihoo is testing the frozen features 02:08:29 q? 02:09:19 "I want to know what it means for an application to conform with HTML5, and how to do this above and beyond what validator.nu does 02:09:27 s/does$/does"/ 02:09:37 That's what I said before, for the minutes. 02:09:40 q+ 02:09:45 @@_korean: We have applications like emails, compatibility is a big issue for us 02:10:15 ... performance matters too 02:10:29 jiangtao has joined #html5perftesting 02:10:35 ... we hope W3C can solve this 02:10:53 rssagent, draft minutes 02:10:58 ack dbaron 02:11:00 s/@@_korean/hyungwook_naver/ 02:11:01 dbaron_moz: I wanted to ask about the performance problem in the slide 02:11:40 rrsagent, draft minutes 02:11:40 I have made the request to generate http://www.w3.org/2015/10/28-html5perftesting-minutes.html jiangtao 02:11:48 ... I learn that as developers, performance problem is a problem for the devs themselves 02:12:10 .... are you trying test the perf of a specific application? 02:12:43 cynthia_ms: I have create a project on test @@ 02:13:27 s/as developers, performance problem is a problem for the devs themselves/as a browser developer, we see that performance problems are rarely what the developers think they are, and you have to profile to find out what is slow/ 02:13:38 paul_ms: When we were trying to push HTML5 to REC, W3C tried to raise money to create a testing project 02:14:11 ... a lot of those willing to contribute had requirements to test applications 02:14:15 s/test the perf of a specific application/test theoretical limits in terms of operations per second (which is very hard to do well) or test the perf of a specific application/ 02:14:22 q? 02:14:40 chunming has joined #html5perftesting 02:14:46 s/I have create a project on test @@/We are going to have accessibility API testing working with web driver, to help automate accessibility tests/ 02:14:47 ... I want to remind you that there are members want to have such test projects 02:14:52 q+ 02:15:50 mchampion_ms: the most important things here is to understand, that the US vendors know each other well 02:16:14 q+ 02:16:46 ... what I get from this slides that Chinese companies and Korean members have misunderstanding of standards 02:16:56 q= judy, kepeng, chaals, jgraham 02:17:17 02:17:18 queue= judy, kepeng, chaals, jgraham 02:18:06 ... is the CG trying to identify which part of web platform work fine to create high performance applications? 02:18:45 judy: I feel that there are different points we are talking about 02:19:48 ... maybe some Chinese members and Korean members missed the common sense of Web standards here 02:20:19 ... Chinese vendors have poor experience here 02:21:20 ... one of the goal of the CG is to collect the compatibility issuees 02:22:01 ... 2nd @@ 02:22:19 ... 3rd @@@ 02:23:38 kepeng: we are willing to offer resource on both thread 02:23:57 q? 02:24:07 ack chaals 02:24:18 ack kepeng 02:24:20 hax_dev: my experience as real developers 02:24:21 I feel like I still don't know *what* they want to test "application" conformance. 02:24:51 And I feel like it's hard to say anything about what direction to take unless I understand what they need. 02:26:05 ... sometimes spec feature changes will cause the applications to fail and the developers may not know how to follow the spec and figure out what's wrong 02:26:34 jgraham: what I heart in the room is what I heart from devs all over the world 02:26:51 s/heart/hear/ 02:27:03 ... it's hard to have all the information for the devs 02:27:32 ... caniuse.com will be helpful for such occations 02:28:25 q+ mcham 02:28:37 ... we have never been able to provide the devs whether certain features work 02:29:38 ... the scope maybe ask the devs to speak up those features of high priority to the vendors 02:29:42 q? 02:29:48 ack jgraham 02:30:18 q+ 02:30:21 I think it's also worthwhile to point out that there are browers that aren't really involved with the W3C who have reasonable marketshare in China, so they might not run web-platform-tests. 02:30:37 chaals: we do want to work out, how can offer better information than caniuse 02:30:56 ... 1) what features work? 02:31:45 ... 2) we want to collect info about stable features 02:33:03 ... 3) problem about resources 02:36:08 ... 4) perf testing is tricky, and it's not what W3C existing tool is set to do 02:36:09 q? 02:36:09 q? 02:37:21 mcham_ms: w-p-t is mostly developed for browser engine developers, which is not the same as the Chinese browser works 02:37:50 ... chaals is asking for sth more data-driven 02:37:52 q+ 02:38:07 q+ to ask where to continue 02:38:12 ack mc 02:38:14 ... CG can start today and don't have to be very specific 02:38:50 jgraham: we have for a long time to visiualize the test result for w-p-t 02:39:25 ... which are a large set of data that will cause performance problems 02:39:41 ... visualise the features that are stable is very hard 02:40:53 ... historically we have been very careful to make these datas available 02:41:49 ... visualisation is something we want to enable but we really can't to provide the ideal interaction 02:42:05 ... performance testing is really really hard 02:42:39 ... it may make the cases to be too complicated 02:43:39 Judy: why you said we need to be careful to expose data? 02:44:56 jgraham: rating may cause misleading, and vendors may worry about contributing for marketing reasons 02:45:18 judy: the goal of w3c is to attract more involvement 02:45:43 ... we welcome diverse people with different issues 02:46:37 ... this CG may help to offer a opportunity for these people to communicate 02:46:47 q+ 02:47:05 ... it doesn't harm to hear the different voice 02:47:25 q? 02:47:41 chaals: it'd be useful to have a good set of test 02:48:01 ... but also good to have a list of those missing 02:48:31 ... we live in a world that companies have marketing department 02:48:55 ... who is interested in rating the browsers, which may be harmful to the web 02:49:16 ... looking at what kind of tests are missing 02:49:28 ack jgraham judy-zhu chaals 02:49:41 ack jgraham 02:49:47 ... most of the stuffs in WPWG, what things are stable, what things people want to use 02:49:48 ack judy-zhu 02:49:57 ... all of these informations are important 02:49:58 ack chaals 02:49:58 chaals, you wanted to ask where to continue 02:50:01 q+ 02:50:29 ... if we know more about that, we will get more sense and it's very helpful 02:50:44 q+ paulc 02:50:44 ... can it be done in a existing place? 02:51:37 kepeng: in China, we have a community that done some initial work about this 02:51:41 q? 02:52:05 ack jg 02:52:22 james: the situation of testing have improved a lot in the past 5 yrs 02:52:51 ... we have a p-t-i mailing list, everybody is welcomed to provide information there 02:53:05 q+ 02:53:27 ... it sounds like a good idea to have CG to solve language barrier 02:53:30 s/p-t-i/public-test-infra/ 02:54:00 ack paulc 02:54:14 paul: perfer to focus the resource on the new feature of the future 02:54:32 q+ mcham 02:54:34 q+ 02:54:50 ... it's your decision to choose those RECs important to you 02:55:26 zakim, close the queue 02:55:26 ... but please also try to improve those specs which will be useful to your companies in the future 02:55:26 ok, chaals, the speaker queue is closed 02:55:59 ... please avoid duplicate works 02:56:20 judy: it's important to look forward 02:56:45 ack judy-zhu 02:56:47 ... the current proposal if to start a CG to collect information 02:57:22 ... we'd like to invite people to telconf and the mailing list 02:58:07 mcham_ms: creating a cg to reflect the need of a certain community is valuable 02:58:34 ... in the language and culture they will be comfortable with 02:59:02 ... the most important point is to allow people to express their need 02:59:11 ... +1 to create a CG 02:59:51 q? 03:00:07 ack mcham 03:00:13 ack chaals 03:00:31 chaals: at least make a community group, the first mission is to figure out the working mode 03:02:01 ... it's important to create test cases for the current work of WGs and TFs 03:03:36 ... it will be great if the management is able to encourage to contribute test cases 03:04:03 ... [thank you] 03:04:50 RRSAgent, make minutes 03:04:50 I have made the request to generate http://www.w3.org/2015/10/28-html5perftesting-minutes.html xiaoqian_w3c 03:05:10 RRSAgent, make log public 03:46:43 chaals1 has joined #html5perftesting 04:27:19 JohnJansen has joined #html5perftesting 04:35:30 xiaoqian has left #html5perftesting 05:22:58 Zakim has left #html5perftesting 05:25:58 joanie has left #html5perftesting 05:26:29 chaals has joined #html5perftesting 05:33:49 myakura has joined #html5perftesting 05:33:57 rrsagent, make logs public 05:34:06 rrsagent, publish minutes 05:34:06 I have made the request to generate http://www.w3.org/2015/10/28-html5perftesting-minutes.html myakura 05:46:03 chaals has joined #html5perftesting 05:47:37 myakura has left #html5perftesting 06:43:19 JohnJansen has joined #html5perftesting 07:10:54 chaals has joined #html5perftesting 07:23:21 gsnedders has left #html5perftesting 07:37:01 jgraham has left #html5perftesting