15:24:49 <RRSAgent> RRSAgent has joined #CSS
15:24:49 <RRSAgent> logging to http://www.w3.org/2010/09/22-CSS-irc
15:24:55 <glazou> Zakim, this will be Style
15:24:55 <Zakim> ok, glazou; I see Style_CSS FP()12:00PM scheduled to start in 36 minutes
15:25:01 <glazou> RRSAgent, make logs public
15:35:03 <myakura> myakura has joined #css
15:55:08 <sylvaing> sylvaing has joined #css
15:55:17 <Zakim> Style_CSS FP()12:00PM has now started
15:55:23 <Zakim> +[plinss]
15:56:51 <dsinger> dsinger has joined #css
15:57:40 <oyvind> oyvind has joined #css
15:57:43 <Zakim> +dsinger
15:58:00 <dsinger> zakim, mute dsinger
15:58:00 <Zakim> dsinger should now be muted
15:58:17 <Zakim> +[Microsoft]
15:58:37 <JohnJansen> JohnJansen has joined #css
15:59:16 <Zakim> +sylvaing
16:00:08 <dbaron> dbaron has joined #css
16:00:32 <arronei> arronei has joined #CSS
16:01:16 <glazou> Zakim, code?
16:01:16 <Zakim> the conference code is 78953 (tel:+1.617.761.6200 tel:+33.4.26.46.79.03 tel:+44.203.318.0479), glazou
16:01:25 <Zakim> +[Microsoft.a]
16:01:29 <arronei> zakim, microsoft.a is me
16:01:29 <Zakim> +arronei; got it
16:02:08 <sylvaing> zakim, who's on the phone ?
16:02:08 <Zakim> On the phone I see [plinss], dsinger (muted), [Microsoft], sylvaing, arronei
16:02:21 <dethbakin> dethbakin has joined #css
16:02:24 <Zakim> +glazou
16:02:26 <sylvaing> zakim, [Microsoft] is arronei, johnjansen
16:02:26 <Zakim> I don't understand '[Microsoft] is arronei, johnjansen', sylvaing
16:02:34 <sylvaing> zakim, [Microsoft] is johnjansen
16:02:34 <Zakim> +johnjansen; got it
16:03:11 <dbaron> Zakim just rings rather than asking for a passcode
16:03:25 <bradk> bradk has joined #css
16:03:36 <dsinger> zakim, who is here?
16:03:36 <Zakim> On the phone I see [plinss], dsinger (muted), johnjansen, sylvaing, arronei, glazou
16:03:39 <Zakim> On IRC I see bradk, dethbakin, arronei, dbaron, JohnJansen, oyvind, dsinger, sylvaing, myakura, RRSAgent, Zakim, glazou, szilles, Hixie, Martijnc, lhnz, krijnh, kennyluck, karl,
16:03:42 <Zakim> ... tabatkins, plinss_, fantasai, Bert, plinss, gsnedders, jgraham, trackbot
16:03:44 <Zakim> +bradk
16:04:05 <sylvaing> Zakim, are you submitting an implementation report ?
16:04:07 <Zakim> I don't understand your question, sylvaing.
16:04:31 <dsinger> zakim, what do you understand?
16:04:31 <Zakim> I don't understand your question, dsinger.
16:04:32 <Zakim> +SteveZ
16:04:35 <sylvaing> Zakim, you're not the only one
16:04:35 <Zakim> I don't understand 'you're not the only one', sylvaing
16:04:52 <smfr> smfr has joined #css
16:05:05 <dbaron> ok, I'm giving up on Zakim for today
16:05:20 <Zakim> +smfr
16:05:30 <dbaron> five minutes spent trying to dial in is enough
16:08:11 <bradk> Zakim, much
16:08:11 <Zakim> I don't understand 'much', bradk
16:09:09 <alexmog> alexmog has joined #css
16:09:58 <smfr> Zakim: scribenick smfr
16:10:06 <smfr> no agenda additions
16:10:11 <glazou> Zakim, who is on the phone?
16:10:11 <Zakim> On the phone I see [plinss], dsinger (muted), johnjansen, sylvaing, arronei, glazou, bradk, SteveZ, smfr
16:10:15 <smfr> staus of implementation reports
16:10:39 <smfr> wanted to hear from mozilla and opera; no-one is on the call
16:10:46 <glazou> dbaron: can you answer through IRC?
16:10:57 <dbaron> what's the question?
16:11:00 <smfr> sylvaing: want to hear from apple
16:11:12 <dbaron> Over the weekend, before the template was up, I ran the tests for chapters 1-4
16:11:18 <smfr> smfr: we have not have resources to go through the test suite, unless it gets automated
16:11:24 <dbaron> but it turns out that isn't actually very useful for building the template, so I'll probably toss that work out
16:11:37 <smfr> how do I tell zakim that i'm scribe?
16:11:45 <sylvaing> scribenick:smfr
16:11:46 <glazou> ScribeNick: smfr
16:11:53 <dbaron> I'm hoping to run the noninteractive parts of the testsuite through the reftest harness to reduce the tests to unique images
16:12:04 <dbaron> which Opera says should be about ~3000 instead of ~9000 or something like that
16:12:21 <dbaron> I'm not exactly sure where that leaves me, though; it depends how much time I'll have.
16:12:59 <smfr> smfr: we could try to crowdsource running the tests
16:13:06 <smfr> sylvaing: so someone could fail every test
16:13:14 <smfr> sylvaing: this is a vendor's report; should be run by the vendor
16:14:01 <smfr> plinss: maybe we can trust the results if multiple people give the same answers
16:14:03 <dbaron> Also, I'm probably going to add an additional state to my implementation report
16:14:19 <dbaron> Since I'm unlikely to be able to figure out whether all the tests that fail are valid or not
16:14:41 <smfr> sylvaing: impl. report should explain if the report is not produced directly by the vendor
16:14:42 <dbaron> so I'll have three failing states instead of two: bug, fail, and invalid
16:15:08 <smfr> smfr: maybe we can use the crowdsourced results to focus our testing
16:15:43 <smfr> sylvaing: you still have to go through the results, so maybe you don't save that much
16:16:32 <smfr> smfr: does the harness let you query results for a given user agent?
16:17:05 <smfr> plinss: tabatkins is working on migrating the HP harness to the w3c server
16:17:26 <plinss_> s/to/from/
16:18:01 <smfr> sylvaing: apple probably won't make 10/15, nor google or mozilla
16:18:19 <smfr> sylvaing: google WILL make it (according to tabatkins )
16:18:36 <sylvaing> correction: microsoft+google indicated they'd make the date; apple,mozilla and opera not, it seems
16:19:28 <smfr> sylvaing: we wont' have two implementations for each testcase
16:20:13 <dbaron> BTW, shouldn't you be talking about 10/18 rather than 10/15 given when the test suite and implementation report template were available?
16:20:39 <smfr> yes, should be 10/18
16:20:41 <sylvaing> yes, 10/18; corrected
16:21:22 <bradk> What about others, such as Prince?
16:21:25 <smfr> plinss: don't have definite answer for mozilla and opera, iffy for apple
16:21:54 <smfr> CSS 2.1 CR Exit Criteria
16:22:19 <smfr> relaxed exit criteria to allow use of current betas
16:23:15 <smfr> smfr: do webkit nightly builds count? not public betas, but are downloadable
16:23:29 <smfr> bradk: opera have the same problem
16:24:13 <smfr> plinss: nightlies too unstable
16:24:58 <dsinger> dsinger has joined #css
16:25:06 <dsinger> zakim, unmute dsinger
16:25:06 <Zakim> dsinger should no longer be muted
16:25:26 <smfr> plinss: nightlies are ok so long as they have been available for a min. of a month
16:25:40 <dsinger> that's more it.  we want that the *feature* is stable, not that the *build* is old
16:26:09 <smfr> sylvaing: we have two vendors submitting reports by 10/18. can the other vendors estimate when they can submit?
16:26:11 <smfr> smfr: can't for apple
16:26:29 <bradk> Is it true that the feature is stable if it is not prefixed?
16:26:34 <dsinger> so, ask that the test passes in a build at least two weeks old (and that it hasn't broken in the meantime)
16:26:48 <smfr> plinss: do we want to relax exit criteria for 2.1?
16:26:54 <Zakim> +[Apple]
16:27:03 <Zakim> -dsinger
16:27:08 <dsinger> zakim, [apple] has dsinger
16:27:08 <Zakim> +dsinger; got it
16:27:51 <smfr> glazou: if we relax the build criteria, will it help?
16:27:59 <smfr> sylvaing: it's not the build, is the cost of running the tests
16:28:06 <smfr> s/is/it is
16:28:41 <smfr> glazou: we should be pragmatic. we should do whatever we need to make CSS 2.1 a Rec.
16:29:11 <smfr> sylvaing: there are other issues
16:30:45 <smfr> discussion at TPAC will ensure
16:30:55 <smfr> s/ensure/ensue
16:31:40 <smfr> sylvaing: maybe we can relax the rule for 2 passes on all tests
16:32:00 <smfr> plinss: do we have to have complete report from each vendor, or just enough to show 2 impls passing each test
16:32:59 <smfr> smfr: what happens if tests fail in all impls?
16:33:13 <smfr> arronei: there are 126 failures in all browsers
16:33:40 <dbaron> Does that 126 number include corrections from when that list was discussed on public-css-testsuite?
16:33:57 <dbaron> (i.e., errors in some of the tests were pointed out)
16:34:20 <smfr> sylvaing: how many tests fail in one or both of IE and chrome (the 2 impl. reports) we have
16:34:37 <smfr> glazou: does arronei have reports for all browsers
16:35:20 <smfr> sylvaing: yes, but MS will not submit reports for other vendors
16:35:34 <smfr> but MS willing to help share to cross-check the results
16:38:50 <smfr> test
16:39:15 <smfr> smfr talked about lack of automation
16:39:41 <smfr> dsinger: what's the real issue
16:39:54 <smfr> glazou: it's a problem for tests that don't pass in 2 browsers
16:40:26 <smfr> sylvaing: let's say MS and chrome submit, and other vendors focus on failing tests
16:40:40 <smfr> sylvaing: can we still go with that?
16:41:15 <smfr> glazou: we can take it to the letter. "we need 2 implementations for each feature"
16:41:45 <smfr> dsinger: we've met the spirit of the law, if not necessarily the letter
16:42:01 <sylvaing> (smfr, I think it's the reverse actually..)
16:42:10 <dsinger> or rather, we met the letter of the law, but more importantly, we also can say we met the spirit
16:42:20 <smfr> glazou: instead of submitting 4 columns of pass/fail, we list 2 browsers for each
16:42:25 <JohnJansen> it looks like there are about 1800 tests that neither IE nor Chrome pass
16:42:53 <JohnJansen> or rather about 7600 tests that we both pass
16:42:56 <smfr> szilles: it's up to the browser vendors to control how they look
16:43:24 <smfr> JohnJansen: 1800 tests that either IE or chrome fail
16:43:34 <smfr> arronei: it's only 126
16:44:29 <smfr> we have 1800 lacking two passes (from IE or chrome)
16:44:35 <smfr> so other vendors should focus on those tests
16:45:04 <smfr> sylvaing: if we had mozilla, what would the number be?
16:45:47 <smfr> dsinger: how many fail because the test is wrong?
16:46:27 <smfr> smfr / arronei: those tests are gradually being addressed
16:46:36 <smfr> dsinger: what about two tests that can't both be passed?
16:46:43 <smfr> arronei: haven't come across any of those
16:47:07 <dbaron> I've come across two that can't both be passed
16:47:09 <smfr> smfr: is the feature == test assumption for exit realistic?
16:47:11 <dbaron> but it was due to an error in one of them
16:47:22 <dbaron> feature == test was never the assumption
16:48:33 <smfr> smfr: a "feature" is covered by a set of tests, maybe we shouldn't require passes of all tests for a given feature
16:48:40 <smfr> plinss: we've done that before
16:48:56 <smfr> sylvaing: question for opera. do they need to submit data for 3 platforms
16:49:10 <smfr> plinss: it's not necessary, may be helpful if only one platform passes
16:49:44 <smfr> plinss: cannot count different platforms as different implementations
16:50:33 <smfr> sylvaing: re: feature vs. test: since we have 20% of tests failing, it doesn't matter much
16:51:10 <smfr> plinss: to conclude
16:51:20 <smfr> plinss: partial reports from some vendors are ok
16:51:28 <sylvaing> any objections ?
16:51:30 <smfr> no objections
16:52:11 <smfr> plinss: can we get a list of the 1800 tests that we need reports for
16:52:36 <smfr> JohnJansen: MS can submit its report on 9/29
16:53:19 <smfr> we don't know when chrome will submit
16:53:20 <smfr> plinss: can MS publish an informal list of where other vendors need to focus?
16:53:52 <smfr> JohnJansen: have to check
16:54:38 <smfr> JohnJansen: if we have mozilla, that 1800 number goes way down
16:55:04 <smfr> glazou: 7600 tests pass in both browsers, in both XHTML1 and HTML4, or just one?
16:55:05 <smfr> sylvaing: both
16:55:25 <smfr> s/ sylvaing / arronei 
16:55:30 <smfr> MS is testing IE9 beta
16:55:43 <JohnJansen> s/ both / IE9 beta and Chrome
16:55:54 <smfr> plinss: can we get resolution on exit criteria? IE9 beta is not good enough
16:56:05 <smfr> exit criteria currently state "shipping builds"
16:56:20 <smfr> JohnJansen: by the time of publishing, it will have been out for 30 days
16:56:27 <dbaron> did we previously resolve to change the 2.1 exit criteria to match the ones we've recently been using for css3 modules?
16:57:17 <smfr> RESOLUTION: will change exit criteria to 2 publicly available builds (including nightlies and betas), as long as they have been available to the general public for 1 month
16:58:08 <smfr> plinss: should not include experimental builds, or builds made to just pass a test
16:58:43 <smfr> plinss: intent should be that the feature should be present in nightlies for a month
16:58:51 <dbaron> builds along a development line intended for a release?
16:59:00 <dsinger> basically, we have to defend the results with a straight face.  that's the bottom line.
16:59:52 <smfr> plinss: this is adopting for 2.1 what we have been doing for CSS 3
17:00:24 <smfr> RESOLVED: adopt current exit criteria for CSS 2.1
17:01:08 <smfr> sylvaing: have an action to talk to tabatkins to see how chrome is doing, and share testcases that don't pass in both
17:01:17 <smfr> sylvaing: any actions on other vendors?
17:01:46 <smfr> action on other vendors: get implementation reports done; partial reports are acceptable
17:01:46 <trackbot> Sorry, couldn't find user - on
17:02:01 <smfr> plinss: no intention to slip the dates
17:02:21 <smfr> glazou: arronei, did you run the tests manually at least once?
17:02:31 <smfr> glazou: how many per day?
17:02:40 <smfr> arronei: 600 tests in an hour, manually
17:06:33 <smfr> smfr: i would like some kind of basic harness to come with the test suite
17:06:43 <Zakim> -SteveZ
17:06:44 <Zakim> -arronei
17:06:46 <Zakim> -[Apple]
17:06:46 <Zakim> -johnjansen
17:06:47 <Zakim> -sylvaing
17:06:54 <Zakim> -[plinss]
17:07:01 <Zakim> -bradk
17:07:02 <smfr> plinss: do I have to do anything with the minutes?
17:07:08 <Zakim> -smfr
17:07:09 <dbaron> dbaron has left #css
17:09:57 <plinss_> smfr: there's a script to format them nicely and they get posted to www-style, fantasai can help...
17:10:01 <Zakim> -glazou
17:10:02 <Zakim> Style_CSS FP()12:00PM has ended
17:10:04 <Zakim> Attendees were [plinss], dsinger, sylvaing, arronei, glazou, johnjansen, bradk, SteveZ, smfr
17:27:59 <oyvind> oyvind has left #css
17:28:02 <myakura> myakura has joined #css
17:50:06 <myakura> myakura has joined #css
17:53:51 <dbaron> dbaron has joined #css
18:06:27 <alexmog> alexmog has joined #css
18:19:46 <smfr> smfr has left #css
18:39:07 <Zakim> Zakim has left #CSS
18:56:02 <nimbupani> nimbupani has joined #css
19:28:25 <anne> anne has joined #css
20:23:43 <kennyluck> kennyluck has joined #CSS
21:41:28 <hyatt> hyatt has joined #css