See also: IRC log
<xiaoqian> script: xiaoqian
<xiaoqian> chair: chaals
<AndreyR> Andrey Rybka bloomberg
<zqzhang> zqzhang: Zhiqiang Zhang, Intel
[Presentation by Kepeng_Alibaba]
Kepeng: current issues in our
opinion: 1) no frozen and complete version for compatibility
testing; 2) lack of HTML5 Perf evaluate
... proposals: 1) create a HTML5 Testing CG
... to create a manifest file for those test for 5.0 REC
features
... benefits: 1) for vendors, some of them wanted to know if
their platform is aligned with the 5.0REC
... 2) for Web developers
... Timeline
james_moz: James from Moz, done a
lot of work on w-p-t
... concerned about this idea
... as browser vendors, no likely we will be interested
... we would like to have the latest features as we run the
whole suite
... it's ok to if we haven't implemented it
... f.ex. Service Workers. Though it's not ready in the
browsers, but it's helpful to have it in the w-p-t
... for web devs, when you write a webapps, not likely they
will be interested if those feature are in the browsers or
not
... there's often a desire that the spec can tell you what you
can use
paul_ms: when we shipped HTML5, the WG went thought a list of features that were not implemented yet, we didn't create a compatibility test suite
chaals_yandex: agree with James,
why a frozen set is important?
... features, spec drafts, and implementation change very
often
... that won't work along with the timeline
... if we have automative tests, f.ex. accessibility test, it's
not there yet
... why do we need to freeze the test set?
gsnedders: specs won't be always
stable, which will cause changes to test cases
... don't think a frozen set is useful
Judy_alibaba: it's a proposal raised by the Chinese companies, I'll like to hear their issues
<gsnedders> ack
chaals: a lot of Chinese members have resource and talent that we can use
james: a bit of push back for the frozen proposal, but really positive to see contribution to w-p-t
Judy: [going through the issues again]
Kepeng: frozen might not be the right word, some people might want to know if the browser are aligned with the spec
chaals: there are a lot of other browsers than the 4 major ones
judy: for alibaba, we want to know if our application are aligned with HTML5
zqzhang_intel: at Intel we have a
cross-walk project, our pratice is based on Blink with some
other new features
... we have some requirement for frozen feature
<AndreyR> I hear 2 things browser and app. testing if browser are you asking for something like this https://html5test.com? and http://www.webpagetest.org for apps?
zqzhang_intel: some test cases
are create for the REC
... I don't see mass requirement from us
<gsnedders> AndreyR: you really want to look at web-platform-tests and ignore html5test, because it doesn't mean much
Judy: qihoo is testing the frozen features
<gsnedders> "I want to know what it means for an application to conform with HTML5, and how to do this above and beyond what validator.nu does
<gsnedders> s/does$/does"/
<gsnedders> That's what I said before, for the minutes.
hyungwook_naver: We have
applications like emails, compatibility is a big issue for
us
... performance matters too
... we hope W3C can solve this
<jiangtao> rssagent, draft minutes
dbaron_moz: I wanted to ask about
the performance problem in the slide
... I learn that as a browser developer, we see that
performance problems are rarely what the developers think they
are, and you have to profile to find out what is slow
... are you trying test theoretical limits in terms of
operations per second (which is very hard to do well) or test
the perf of a specific application?
cynthia_ms: We are going to have accessibility API testing working with web driver, to help automate accessibility tests
paul_ms: When we were trying to
push HTML5 to REC, W3C tried to raise money to create a testing
project
... a lot of those willing to contribute had requirements to
test applications
... I want to remind you that there are members want to have
such test projects
mchampion_ms: the most important
things here is to understand, that the US vendors know each
other well
... what I get from this slides that Chinese companies and
Korean members have misunderstanding of standards
... is the CG trying to identify which part of web platform
work fine to create high performance applications?
judy: I feel that there are
different points we are talking about
... maybe some Chinese members and Korean members missed the
common sense of Web standards here
... Chinese vendors have poor experience here
... one of the goal of the CG is to collect the compatibility
issuees
... 2nd @@
... 3rd @@@
kepeng: we are willing to offer resource on both thread
hax_dev: my experience as real developers
<gsnedders> I feel like I still don't know *what* they want to test "application" conformance.
<gsnedders> And I feel like it's hard to say anything about what direction to take unless I understand what they need.
hax_dev: sometimes spec feature changes will cause the applications to fail and the developers may not know how to follow the spec and figure out what's wrong
jgraham: what I heart in the room
is what I hear from devs all over the world
... it's hard to have all the information for the devs
... caniuse.com will be helpful for such occations
... we have never been able to provide the devs whether certain
features work
... the scope maybe ask the devs to speak up those features of
high priority to the vendors
<gsnedders> I think it's also worthwhile to point out that there are browers that aren't really involved with the W3C who have reasonable marketshare in China, so they might not run web-platform-tests.
chaals: we do want to work out,
how can offer better information than caniuse
... 1) what features work?
... 2) we want to collect info about stable features
... 3) problem about resources
... 4) perf testing is tricky, and it's not what W3C existing
tool is set to do
mcham_ms: w-p-t is mostly
developed for browser engine developers, which is not the same
as the Chinese browser works
... chaals is asking for sth more data-driven
... CG can start today and don't have to be very specific
jgraham: we have for a long time
to visiualize the test result for w-p-t
... which are a large set of data that will cause performance
problems
... visualise the features that are stable is very hard
... historically we have been very careful to make these datas
available
... visualisation is something we want to enable but we really
can't to provide the ideal interaction
... performance testing is really really hard
... it may make the cases to be too complicated
Judy: why you said we need to be careful to expose data?
jgraham: rating may cause misleading, and vendors may worry about contributing for marketing reasons
judy: the goal of w3c is to
attract more involvement
... we welcome diverse people with different issues
... this CG may help to offer a opportunity for these people to
communicate
... it doesn't harm to hear the different voice
chaals: it'd be useful to have a
good set of test
... but also good to have a list of those missing
... we live in a world that companies have marketing
department
... who is interested in rating the browsers, which may be
harmful to the web
... looking at what kind of tests are missing
<jgraham> ack jgraham judy-zhu chaals
chaals: most of the stuffs in
WPWG, what things are stable, what things people want to
use
... all of these informations are important
<Zakim> chaals, you wanted to ask where to continue
chaals: if we know more about
that, we will get more sense and it's very helpful
... can it be done in a existing place?
kepeng: in China, we have a community that done some initial work about this
james: the situation of testing
have improved a lot in the past 5 yrs
... we have a public-test-infra mailing list, everybody is
welcomed to provide information there
... it sounds like a good idea to have CG to solve language
barrier
paul: perfer to focus the
resource on the new feature of the future
... it's your decision to choose those RECs important to
you
... but please also try to improve those specs which will be
useful to your companies in the future
... please avoid duplicate works
judy: it's important to look
forward
... the current proposal if to start a CG to collect
information
... we'd like to invite people to telconf and the mailing
list
mcham_ms: creating a cg to
reflect the need of a certain community is valuable
... in the language and culture they will be comfortable
with
... the most important point is to allow people to express
their need
... +1 to create a CG
chaals: at least make a community
group, the first mission is to figure out the working
mode
... it's important to create test cases for the current work of
WGs and TFs
... it will be great if the management is able to encourage to
contribute test cases
... [thank you]
This is scribe.perl Revision: 1.140 of Date: 2014-11-06 18:16:30 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: RRSAgent_Text_Format (score 1.00) Succeeded: s/nice to see you, xiaoqian// Succeeded: s/2)/... 2)/ FAILED: s/does$/does"/ Succeeded: s/@@_korean/hyungwook_naver/ Succeeded: s/as developers, performance problem is a problem for the devs themselves/as a browser developer, we see that performance problems are rarely what the developers think they are, and you have to profile to find out what is slow/ Succeeded: s/test the perf of a specific application/test theoretical limits in terms of operations per second (which is very hard to do well) or test the perf of a specific application/ Succeeded: s/I have create a project on test @@/We are going to have accessibility API testing working with web driver, to help automate accessibility tests/ Succeeded: s/heart/hear/ Succeeded: s/p-t-i/public-test-infra/ No ScribeNick specified. Guessing ScribeNick: xiaoqian_w3c Inferring Scribes: xiaoqian_w3c WARNING: No "Topic:" lines found. Present: jiangtao zqzhang myakura JohnJansen gsnedders jgraham dwim dwim_samsung zephyr chaals WARNING: No meeting title found! You should specify the meeting title like this: <dbooth> Meeting: Weekly Baking Club Meeting Got date from IRC log name: 28 Oct 2015 Guessing minutes URL: http://www.w3.org/2015/10/28-html5perftesting-minutes.html People with action items: WARNING: No "Topic: ..." lines found! Resulting HTML may have an empty (invalid) <ol>...</ol>. Explanation: "Topic: ..." lines are used to indicate the start of new discussion topics or agenda items, such as: <dbooth> Topic: Review of Amy's report[End of scribe.perl diagnostic output]