01:30:08 RRSAgent has joined #html5perftesting
01:30:09 logging to http://www.w3.org/2015/10/28-html5perftesting-irc
01:30:15 Zakim has joined #html5perftesting
01:31:25 dwim_ has joined #html5perftesting
01:31:52 zakim, this is HTML5 Testing TPAC Break Out
01:31:52 got it, xiaoqian
01:32:11 chair: Kepeng Li
01:32:31 script: xiaoqian
01:33:27 nice to see you, xiaoqian
01:34:13 s/nice to see you, xiaoqian//
01:34:35 zqzhang has joined #html5perftesting
01:35:30 gsnedders has joined #html5perftesting
01:36:11 jiangtao has joined #html5perftesting
01:36:39 myakura has joined #html5perftesting
01:37:37 zephyr has joined #html5perftesting
01:38:28 hax has joined #html5perftesting
01:38:38 present+ jiangtao
01:38:38 chair: chaals
01:39:27 present+ zqzhang
01:39:43 JohnJansen has joined #html5perftesting
01:39:59 present+ myakura
01:40:04 QingAn has joined #html5perftesting
01:40:18 judy-zhu has joined #html5perftesting
01:40:53 present+ JohnJansen
01:41:06 jgraham has joined #html5perftesting
01:41:12 present+ gsnedders
01:41:13 present+ jgraham
01:41:30 peterw-melco has joined #html5perftesting
01:41:31 present+ dwim
01:41:49 AndreyR has joined #html5perftesting
01:42:00 dbaron has joined #html5perftesting
01:42:17 Andrey Rybka bloomberg
01:42:52 zqzhang: Zhiqiang Zhang, Intel
01:42:55 ivan has joined #html5perftesting
01:42:59 present+ dwim_samsung
01:43:11 present+ zephyr
01:43:34 [Presentation by Kepeng_Alibaba]
01:43:36 mchampion has joined #html5perftesting
01:45:15 Takeharu has joined #html5perftesting
01:45:24 SAM has joined #html5perftesting
01:45:31 ko has joined #html5perftesting
01:46:06 Kepeng: current issues in our opinion: 1) no frozen and complete version for compatibility testing; 2) lack of HTML5 Perf evaluate
01:46:19 kokabe has joined #html5perftesting
01:46:42 ... proposals: 1) create a HTML5 Testing CG
01:47:16 ... to create a manifest file for those test for 5.0 REC features
01:48:07 ... benefits: 1) for vendors, some of them wanted to know if their platform is aligned with the 5.0REC
01:48:21 2) for Web developers
01:48:43 ... Timeline
01:48:54 s/2)/... 2)
01:49:39 chaals has joined #html5perftesting
01:49:42 james_moz: James from Moz, done a lot of work on w-p-t
01:49:52 ... concerned about this idea
01:50:02 present+ chaals
01:50:11 ... as browser vendors, no likely we will be interested
01:50:28 q+
01:50:31 ... we would like to have the latest features as we run the whole suite
01:50:46 ... it's ok to if we haven't implemented it
01:51:33 ... f.ex. Service Workers. Though it's not ready in the browsers, but it's helpful to have it in the w-p-t
01:52:29 ... for web devs, when you write a webapps, not likely they will be interested if those feature are in the browsers or not
01:52:34 q+
01:53:07 q+
01:53:10 ... there's often a desire that the spec can tell you what you can use
01:53:35 pjeong_ has joined #html5perftesting
01:53:49 q+
01:53:58 yingying has joined #html5perftesting
01:54:19 q+
01:54:42 paul_ms: when we shipped HTML5, the WG went thought a list of features that were not implemented yet, we didn't create a compatibility test suite
01:55:06 chaals_yandex: agree with James, why a frozen set is important?
01:55:43 ... features, spec drafts, and implementation change very often
01:55:52 mchampion has joined #html5perftesting
01:55:58 ... that won't work along with the timeline
01:56:05 q?
01:56:56 q+ cynthia
01:57:00 ToddReifsteck has joined #html5perftesting
01:57:01 ... if we have automative tests, f.ex. accessibility test, it's not there yet
01:57:19 ... why do we need to freeze the test set?
01:57:34 q+ paulc
01:57:52 joanie has joined #html5perftesting
01:57:53 cyns has joined #html5perftesting
01:58:00 q+
01:58:17 gsnedders: specs won't be always stable, which will cause changes to test cases
01:58:36 ... don't think a frozen set is useful
01:59:34 Judy_alibaba: it's a proposal raised by the Chinese companies, I'll like to hear their issues
01:59:47 ack
01:59:56 q?
02:00:25 chaals: a lot of Chinese members have resource and talent that we can use
02:01:13 james: a bit of push back for the frozen proposal, but really positive to see contribution to w-p-t
02:01:53 q?
02:01:58 Judy: [going through the issues again]
02:03:38 Kepeng: frozen might not be the right word, some people might want to know if the browser are aligned with the spec
02:04:07 chaals: there are a lot of other browsers than the 4 major ones
02:04:53 judy: for alibaba, we want to know if our application are aligned with HTML5
02:05:50 q?
02:06:17 zqzhang_intel: at Intel we have a cross-walk project, our pratice is based on Blink with some other new features
02:06:54 ... we have some requirement for frozen feature
02:06:55 I hear 2 things browser and app. testing if browser are you asking for something like this https://html5test.com? and http://www.webpagetest.org for apps?
02:07:13 ... some test cases are create for the REC
02:07:53 ... I don't see mass requirement from us
02:07:54 AndreyR: you really want to look at web-platform-tests and ignore html5test, because it doesn't mean much
02:08:07 Judy: qihoo is testing the frozen features
02:08:29 q?
02:09:19 "I want to know what it means for an application to conform with HTML5, and how to do this above and beyond what validator.nu does
02:09:27 s/does$/does"/
02:09:37 That's what I said before, for the minutes.
02:09:40 q+
02:09:45 @@_korean: We have applications like emails, compatibility is a big issue for us
02:10:15 ... performance matters too
02:10:29 jiangtao has joined #html5perftesting
02:10:35 ... we hope W3C can solve this
02:10:53 rssagent, draft minutes
02:10:58 ack dbaron
02:11:00 s/@@_korean/hyungwook_naver/
02:11:01 dbaron_moz: I wanted to ask about the performance problem in the slide
02:11:40 rrsagent, draft minutes
02:11:40 I have made the request to generate http://www.w3.org/2015/10/28-html5perftesting-minutes.html jiangtao
02:11:48 ... I learn that as developers, performance problem is a problem for the devs themselves
02:12:10 .... are you trying test the perf of a specific application?
02:12:43 cynthia_ms: I have create a project on test @@
02:13:27 s/as developers, performance problem is a problem for the devs themselves/as a browser developer, we see that performance problems are rarely what the developers think they are, and you have to profile to find out what is slow/
02:13:38 paul_ms: When we were trying to push HTML5 to REC, W3C tried to raise money to create a testing project
02:14:11 ... a lot of those willing to contribute had requirements to test applications
02:14:15 s/test the perf of a specific application/test theoretical limits in terms of operations per second (which is very hard to do well) or test the perf of a specific application/
02:14:22 q?
02:14:40 chunming has joined #html5perftesting
02:14:46 s/I have create a project on test @@/We are going to have accessibility API testing working with web driver, to help automate accessibility tests/
02:14:47 ... I want to remind you that there are members want to have such test projects
02:14:52 q+
02:15:50 mchampion_ms: the most important things here is to understand, that the US vendors know each other well
02:16:14 q+
02:16:46 ... what I get from this slides that Chinese companies and Korean members have misunderstanding of standards
02:16:56 q= judy, kepeng, chaals, jgraham
02:17:17
02:17:18 queue= judy, kepeng, chaals, jgraham
02:18:06 ... is the CG trying to identify which part of web platform work fine to create high performance applications?
02:18:45 judy: I feel that there are different points we are talking about
02:19:48 ... maybe some Chinese members and Korean members missed the common sense of Web standards here
02:20:19 ... Chinese vendors have poor experience here
02:21:20 ... one of the goal of the CG is to collect the compatibility issuees
02:22:01 ... 2nd @@
02:22:19 ... 3rd @@@
02:23:38 kepeng: we are willing to offer resource on both thread
02:23:57 q?
02:24:07 ack chaals
02:24:18 ack kepeng
02:24:20 hax_dev: my experience as real developers
02:24:21 I feel like I still don't know *what* they want to test "application" conformance.
02:24:51 And I feel like it's hard to say anything about what direction to take unless I understand what they need.
02:26:05 ... sometimes spec feature changes will cause the applications to fail and the developers may not know how to follow the spec and figure out what's wrong
02:26:34 jgraham: what I heart in the room is what I heart from devs all over the world
02:26:51 s/heart/hear/
02:27:03 ... it's hard to have all the information for the devs
02:27:32 ... caniuse.com will be helpful for such occations
02:28:25 q+ mcham
02:28:37 ... we have never been able to provide the devs whether certain features work
02:29:38 ... the scope maybe ask the devs to speak up those features of high priority to the vendors
02:29:42 q?
02:29:48 ack jgraham
02:30:18 q+
02:30:21 I think it's also worthwhile to point out that there are browers that aren't really involved with the W3C who have reasonable marketshare in China, so they might not run web-platform-tests.
02:30:37 chaals: we do want to work out, how can offer better information than caniuse
02:30:56 ... 1) what features work?
02:31:45 ... 2) we want to collect info about stable features
02:33:03 ... 3) problem about resources
02:36:08 ... 4) perf testing is tricky, and it's not what W3C existing tool is set to do
02:36:09 q?
02:36:09 q?
02:37:21 mcham_ms: w-p-t is mostly developed for browser engine developers, which is not the same as the Chinese browser works
02:37:50 ... chaals is asking for sth more data-driven
02:37:52 q+
02:38:07 q+ to ask where to continue
02:38:12 ack mc
02:38:14 ... CG can start today and don't have to be very specific
02:38:50 jgraham: we have for a long time to visiualize the test result for w-p-t
02:39:25 ... which are a large set of data that will cause performance problems
02:39:41 ... visualise the features that are stable is very hard
02:40:53 ... historically we have been very careful to make these datas available
02:41:49 ... visualisation is something we want to enable but we really can't to provide the ideal interaction
02:42:05 ... performance testing is really really hard
02:42:39 ... it may make the cases to be too complicated
02:43:39 Judy: why you said we need to be careful to expose data?
02:44:56 jgraham: rating may cause misleading, and vendors may worry about contributing for marketing reasons
02:45:18 judy: the goal of w3c is to attract more involvement
02:45:43 ... we welcome diverse people with different issues
02:46:37 ... this CG may help to offer a opportunity for these people to communicate
02:46:47 q+
02:47:05 ... it doesn't harm to hear the different voice
02:47:25 q?
02:47:41 chaals: it'd be useful to have a good set of test
02:48:01 ... but also good to have a list of those missing
02:48:31 ... we live in a world that companies have marketing department
02:48:55 ... who is interested in rating the browsers, which may be harmful to the web
02:49:16 ... looking at what kind of tests are missing
02:49:28 ack jgraham judy-zhu chaals
02:49:41 ack jgraham
02:49:47 ... most of the stuffs in WPWG, what things are stable, what things people want to use
02:49:48 ack judy-zhu
02:49:57 ... all of these informations are important
02:49:58 ack chaals
02:49:58 chaals, you wanted to ask where to continue
02:50:01 q+
02:50:29 ... if we know more about that, we will get more sense and it's very helpful
02:50:44 q+ paulc
02:50:44 ... can it be done in a existing place?
02:51:37 kepeng: in China, we have a community that done some initial work about this
02:51:41 q?
02:52:05 ack jg
02:52:22 james: the situation of testing have improved a lot in the past 5 yrs
02:52:51 ... we have a p-t-i mailing list, everybody is welcomed to provide information there
02:53:05 q+
02:53:27 ... it sounds like a good idea to have CG to solve language barrier
02:53:30 s/p-t-i/public-test-infra/
02:54:00 ack paulc
02:54:14 paul: perfer to focus the resource on the new feature of the future
02:54:32 q+ mcham
02:54:34 q+
02:54:50 ... it's your decision to choose those RECs important to you
02:55:26 zakim, close the queue
02:55:26 ... but please also try to improve those specs which will be useful to your companies in the future
02:55:26 ok, chaals, the speaker queue is closed
02:55:59 ... please avoid duplicate works
02:56:20 judy: it's important to look forward
02:56:45 ack judy-zhu
02:56:47 ... the current proposal if to start a CG to collect information
02:57:22 ... we'd like to invite people to telconf and the mailing list
02:58:07 mcham_ms: creating a cg to reflect the need of a certain community is valuable
02:58:34 ... in the language and culture they will be comfortable with
02:59:02 ... the most important point is to allow people to express their need
02:59:11 ... +1 to create a CG
02:59:51 q?
03:00:07 ack mcham
03:00:13 ack chaals
03:00:31 chaals: at least make a community group, the first mission is to figure out the working mode
03:02:01 ... it's important to create test cases for the current work of WGs and TFs
03:03:36 ... it will be great if the management is able to encourage to contribute test cases
03:04:03 ... [thank you]
03:04:50 RRSAgent, make minutes
03:04:50 I have made the request to generate http://www.w3.org/2015/10/28-html5perftesting-minutes.html xiaoqian_w3c
03:05:10 RRSAgent, make log public
03:46:43 chaals1 has joined #html5perftesting
04:27:19 JohnJansen has joined #html5perftesting
04:35:30 xiaoqian has left #html5perftesting
05:22:58 Zakim has left #html5perftesting
05:25:58 joanie has left #html5perftesting
05:26:29 chaals has joined #html5perftesting
05:33:49 myakura has joined #html5perftesting
05:33:57 rrsagent, make logs public
05:34:06 rrsagent, publish minutes
05:34:06 I have made the request to generate http://www.w3.org/2015/10/28-html5perftesting-minutes.html myakura
05:46:03 chaals has joined #html5perftesting
05:47:37 myakura has left #html5perftesting
06:43:19 JohnJansen has joined #html5perftesting
07:10:54 chaals has joined #html5perftesting
07:23:21 gsnedders has left #html5perftesting
07:37:01 jgraham has left #html5perftesting