28 Oct 2015

See also: IRC log


jiangtao, zqzhang, myakura, JohnJansen, gsnedders, jgraham, dwim, dwim_samsung, zephyr, chaals
Kepeng Li


<xiaoqian> script: xiaoqian

<xiaoqian> chair: chaals

<AndreyR> Andrey Rybka bloomberg

<zqzhang> zqzhang: Zhiqiang Zhang, Intel

[Presentation by Kepeng_Alibaba]

Kepeng: current issues in our opinion: 1) no frozen and complete version for compatibility testing; 2) lack of HTML5 Perf evaluate
... proposals: 1) create a HTML5 Testing CG
... to create a manifest file for those test for 5.0 REC features
... benefits: 1) for vendors, some of them wanted to know if their platform is aligned with the 5.0REC
... 2) for Web developers
... Timeline

james_moz: James from Moz, done a lot of work on w-p-t
... concerned about this idea
... as browser vendors, no likely we will be interested
... we would like to have the latest features as we run the whole suite
... it's ok to if we haven't implemented it
... f.ex. Service Workers. Though it's not ready in the browsers, but it's helpful to have it in the w-p-t
... for web devs, when you write a webapps, not likely they will be interested if those feature are in the browsers or not
... there's often a desire that the spec can tell you what you can use

paul_ms: when we shipped HTML5, the WG went thought a list of features that were not implemented yet, we didn't create a compatibility test suite

chaals_yandex: agree with James, why a frozen set is important?
... features, spec drafts, and implementation change very often
... that won't work along with the timeline
... if we have automative tests, f.ex. accessibility test, it's not there yet
... why do we need to freeze the test set?

gsnedders: specs won't be always stable, which will cause changes to test cases
... don't think a frozen set is useful

Judy_alibaba: it's a proposal raised by the Chinese companies, I'll like to hear their issues

<gsnedders> ack

chaals: a lot of Chinese members have resource and talent that we can use

james: a bit of push back for the frozen proposal, but really positive to see contribution to w-p-t

Judy: [going through the issues again]

Kepeng: frozen might not be the right word, some people might want to know if the browser are aligned with the spec

chaals: there are a lot of other browsers than the 4 major ones

judy: for alibaba, we want to know if our application are aligned with HTML5

zqzhang_intel: at Intel we have a cross-walk project, our pratice is based on Blink with some other new features
... we have some requirement for frozen feature

<AndreyR> I hear 2 things browser and app. testing if browser are you asking for something like this https://html5test.com? and http://www.webpagetest.org for apps?

zqzhang_intel: some test cases are create for the REC
... I don't see mass requirement from us

<gsnedders> AndreyR: you really want to look at web-platform-tests and ignore html5test, because it doesn't mean much

Judy: qihoo is testing the frozen features

<gsnedders> "I want to know what it means for an application to conform with HTML5, and how to do this above and beyond what validator.nu does

<gsnedders> s/does$/does"/

<gsnedders> That's what I said before, for the minutes.

hyungwook_naver: We have applications like emails, compatibility is a big issue for us
... performance matters too
... we hope W3C can solve this

<jiangtao> rssagent, draft minutes

dbaron_moz: I wanted to ask about the performance problem in the slide
... I learn that as a browser developer, we see that performance problems are rarely what the developers think they are, and you have to profile to find out what is slow
... are you trying test theoretical limits in terms of operations per second (which is very hard to do well) or test the perf of a specific application?

cynthia_ms: We are going to have accessibility API testing working with web driver, to help automate accessibility tests

paul_ms: When we were trying to push HTML5 to REC, W3C tried to raise money to create a testing project
... a lot of those willing to contribute had requirements to test applications
... I want to remind you that there are members want to have such test projects

mchampion_ms: the most important things here is to understand, that the US vendors know each other well
... what I get from this slides that Chinese companies and Korean members have misunderstanding of standards
... is the CG trying to identify which part of web platform work fine to create high performance applications?

judy: I feel that there are different points we are talking about
... maybe some Chinese members and Korean members missed the common sense of Web standards here
... Chinese vendors have poor experience here
... one of the goal of the CG is to collect the compatibility issuees
... 2nd @@
... 3rd @@@

kepeng: we are willing to offer resource on both thread

hax_dev: my experience as real developers

<gsnedders> I feel like I still don't know *what* they want to test "application" conformance.

<gsnedders> And I feel like it's hard to say anything about what direction to take unless I understand what they need.

hax_dev: sometimes spec feature changes will cause the applications to fail and the developers may not know how to follow the spec and figure out what's wrong

jgraham: what I heart in the room is what I hear from devs all over the world
... it's hard to have all the information for the devs
... caniuse.com will be helpful for such occations
... we have never been able to provide the devs whether certain features work
... the scope maybe ask the devs to speak up those features of high priority to the vendors

<gsnedders> I think it's also worthwhile to point out that there are browers that aren't really involved with the W3C who have reasonable marketshare in China, so they might not run web-platform-tests.

chaals: we do want to work out, how can offer better information than caniuse
... 1) what features work?
... 2) we want to collect info about stable features
... 3) problem about resources
... 4) perf testing is tricky, and it's not what W3C existing tool is set to do

mcham_ms: w-p-t is mostly developed for browser engine developers, which is not the same as the Chinese browser works
... chaals is asking for sth more data-driven
... CG can start today and don't have to be very specific

jgraham: we have for a long time to visiualize the test result for w-p-t
... which are a large set of data that will cause performance problems
... visualise the features that are stable is very hard
... historically we have been very careful to make these datas available
... visualisation is something we want to enable but we really can't to provide the ideal interaction
... performance testing is really really hard
... it may make the cases to be too complicated

Judy: why you said we need to be careful to expose data?

jgraham: rating may cause misleading, and vendors may worry about contributing for marketing reasons

judy: the goal of w3c is to attract more involvement
... we welcome diverse people with different issues
... this CG may help to offer a opportunity for these people to communicate
... it doesn't harm to hear the different voice

chaals: it'd be useful to have a good set of test
... but also good to have a list of those missing
... we live in a world that companies have marketing department
... who is interested in rating the browsers, which may be harmful to the web
... looking at what kind of tests are missing

<jgraham> ack jgraham judy-zhu chaals

chaals: most of the stuffs in WPWG, what things are stable, what things people want to use
... all of these informations are important

<Zakim> chaals, you wanted to ask where to continue

chaals: if we know more about that, we will get more sense and it's very helpful
... can it be done in a existing place?

kepeng: in China, we have a community that done some initial work about this

james: the situation of testing have improved a lot in the past 5 yrs
... we have a public-test-infra mailing list, everybody is welcomed to provide information there
... it sounds like a good idea to have CG to solve language barrier

paul: perfer to focus the resource on the new feature of the future
... it's your decision to choose those RECs important to you
... but please also try to improve those specs which will be useful to your companies in the future
... please avoid duplicate works

judy: it's important to look forward
... the current proposal if to start a CG to collect information
... we'd like to invite people to telconf and the mailing list

mcham_ms: creating a cg to reflect the need of a certain community is valuable
... in the language and culture they will be comfortable with
... the most important point is to allow people to express their need
... +1 to create a CG

chaals: at least make a community group, the first mission is to figure out the working mode
... it's important to create test cases for the current work of WGs and TFs
... it will be great if the management is able to encourage to contribute test cases
... [thank you]

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.140 (CVS log)
$Date: 2015/10/28 05:34:11 $

Scribe.perl diagnostic output

[Delete this section before finalizing the minutes.]
This is scribe.perl Revision: 1.140  of Date: 2014-11-06 18:16:30  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: RRSAgent_Text_Format (score 1.00)

Succeeded: s/nice to see you, xiaoqian//
Succeeded: s/2)/... 2)/
FAILED: s/does$/does"/
Succeeded: s/@@_korean/hyungwook_naver/
Succeeded: s/as developers, performance problem is a problem for the devs themselves/as a browser developer, we see that performance problems are rarely what the developers think they are, and you have to profile to find out what is slow/
Succeeded: s/test the perf of a specific application/test theoretical limits in terms of operations per second (which is very hard to do well) or test the perf of a specific application/
Succeeded: s/I have create a project on test @@/We are going to have accessibility API testing working with web driver, to help automate accessibility tests/
Succeeded: s/heart/hear/
Succeeded: s/p-t-i/public-test-infra/
No ScribeNick specified.  Guessing ScribeNick: xiaoqian_w3c
Inferring Scribes: xiaoqian_w3c

WARNING: No "Topic:" lines found.

Present: jiangtao zqzhang myakura JohnJansen gsnedders jgraham dwim dwim_samsung zephyr chaals

WARNING: No meeting title found!
You should specify the meeting title like this:
<dbooth> Meeting: Weekly Baking Club Meeting

Got date from IRC log name: 28 Oct 2015
Guessing minutes URL: http://www.w3.org/2015/10/28-html5perftesting-minutes.html
People with action items: 

WARNING: No "Topic: ..." lines found!  
Resulting HTML may have an empty (invalid) <ol>...</ol>.

Explanation: "Topic: ..." lines are used to indicate the start of 
new discussion topics or agenda items, such as:
<dbooth> Topic: Review of Amy's report

[End of scribe.perl diagnostic output]