IRC log of CSS on 2010-09-22

Timestamps are in UTC.

15:24:49 [RRSAgent]
RRSAgent has joined #CSS
15:24:49 [RRSAgent]
logging to http://www.w3.org/2010/09/22-CSS-irc
15:24:55 [glazou]
Zakim, this will be Style
15:24:55 [Zakim]
ok, glazou; I see Style_CSS FP()12:00PM scheduled to start in 36 minutes
15:25:01 [glazou]
RRSAgent, make logs public
15:35:03 [myakura]
myakura has joined #css
15:55:08 [sylvaing]
sylvaing has joined #css
15:55:17 [Zakim]
Style_CSS FP()12:00PM has now started
15:55:23 [Zakim]
+[plinss]
15:56:51 [dsinger]
dsinger has joined #css
15:57:40 [oyvind]
oyvind has joined #css
15:57:43 [Zakim]
+dsinger
15:58:00 [dsinger]
zakim, mute dsinger
15:58:00 [Zakim]
dsinger should now be muted
15:58:17 [Zakim]
+[Microsoft]
15:58:37 [JohnJansen]
JohnJansen has joined #css
15:59:16 [Zakim]
+sylvaing
16:00:08 [dbaron]
dbaron has joined #css
16:00:32 [arronei]
arronei has joined #CSS
16:01:16 [glazou]
Zakim, code?
16:01:16 [Zakim]
the conference code is 78953 (tel:+1.617.761.6200 tel:+33.4.26.46.79.03 tel:+44.203.318.0479), glazou
16:01:25 [Zakim]
+[Microsoft.a]
16:01:29 [arronei]
zakim, microsoft.a is me
16:01:29 [Zakim]
+arronei; got it
16:02:08 [sylvaing]
zakim, who's on the phone ?
16:02:08 [Zakim]
On the phone I see [plinss], dsinger (muted), [Microsoft], sylvaing, arronei
16:02:21 [dethbakin]
dethbakin has joined #css
16:02:24 [Zakim]
+glazou
16:02:26 [sylvaing]
zakim, [Microsoft] is arronei, johnjansen
16:02:26 [Zakim]
I don't understand '[Microsoft] is arronei, johnjansen', sylvaing
16:02:34 [sylvaing]
zakim, [Microsoft] is johnjansen
16:02:34 [Zakim]
+johnjansen; got it
16:03:11 [dbaron]
Zakim just rings rather than asking for a passcode
16:03:25 [bradk]
bradk has joined #css
16:03:36 [dsinger]
zakim, who is here?
16:03:36 [Zakim]
On the phone I see [plinss], dsinger (muted), johnjansen, sylvaing, arronei, glazou
16:03:39 [Zakim]
On IRC I see bradk, dethbakin, arronei, dbaron, JohnJansen, oyvind, dsinger, sylvaing, myakura, RRSAgent, Zakim, glazou, szilles, Hixie, Martijnc, lhnz, krijnh, kennyluck, karl,
16:03:42 [Zakim]
... tabatkins, plinss_, fantasai, Bert, plinss, gsnedders, jgraham, trackbot
16:03:44 [Zakim]
+bradk
16:04:05 [sylvaing]
Zakim, are you submitting an implementation report ?
16:04:07 [Zakim]
I don't understand your question, sylvaing.
16:04:31 [dsinger]
zakim, what do you understand?
16:04:31 [Zakim]
I don't understand your question, dsinger.
16:04:32 [Zakim]
+SteveZ
16:04:35 [sylvaing]
Zakim, you're not the only one
16:04:35 [Zakim]
I don't understand 'you're not the only one', sylvaing
16:04:52 [smfr]
smfr has joined #css
16:05:05 [dbaron]
ok, I'm giving up on Zakim for today
16:05:20 [Zakim]
+smfr
16:05:30 [dbaron]
five minutes spent trying to dial in is enough
16:08:11 [bradk]
Zakim, much
16:08:11 [Zakim]
I don't understand 'much', bradk
16:09:09 [alexmog]
alexmog has joined #css
16:09:58 [smfr]
Zakim: scribenick smfr
16:10:06 [smfr]
no agenda additions
16:10:11 [glazou]
Zakim, who is on the phone?
16:10:11 [Zakim]
On the phone I see [plinss], dsinger (muted), johnjansen, sylvaing, arronei, glazou, bradk, SteveZ, smfr
16:10:15 [smfr]
staus of implementation reports
16:10:39 [smfr]
wanted to hear from mozilla and opera; no-one is on the call
16:10:46 [glazou]
dbaron: can you answer through IRC?
16:10:57 [dbaron]
what's the question?
16:11:00 [smfr]
sylvaing: want to hear from apple
16:11:12 [dbaron]
Over the weekend, before the template was up, I ran the tests for chapters 1-4
16:11:18 [smfr]
smfr: we have not have resources to go through the test suite, unless it gets automated
16:11:24 [dbaron]
but it turns out that isn't actually very useful for building the template, so I'll probably toss that work out
16:11:37 [smfr]
how do I tell zakim that i'm scribe?
16:11:45 [sylvaing]
scribenick:smfr
16:11:46 [glazou]
ScribeNick: smfr
16:11:53 [dbaron]
I'm hoping to run the noninteractive parts of the testsuite through the reftest harness to reduce the tests to unique images
16:12:04 [dbaron]
which Opera says should be about ~3000 instead of ~9000 or something like that
16:12:21 [dbaron]
I'm not exactly sure where that leaves me, though; it depends how much time I'll have.
16:12:59 [smfr]
smfr: we could try to crowdsource running the tests
16:13:06 [smfr]
sylvaing: so someone could fail every test
16:13:14 [smfr]
sylvaing: this is a vendor's report; should be run by the vendor
16:14:01 [smfr]
plinss: maybe we can trust the results if multiple people give the same answers
16:14:03 [dbaron]
Also, I'm probably going to add an additional state to my implementation report
16:14:19 [dbaron]
Since I'm unlikely to be able to figure out whether all the tests that fail are valid or not
16:14:41 [smfr]
sylvaing: impl. report should explain if the report is not produced directly by the vendor
16:14:42 [dbaron]
so I'll have three failing states instead of two: bug, fail, and invalid
16:15:08 [smfr]
smfr: maybe we can use the crowdsourced results to focus our testing
16:15:43 [smfr]
sylvaing: you still have to go through the results, so maybe you don't save that much
16:16:32 [smfr]
smfr: does the harness let you query results for a given user agent?
16:17:05 [smfr]
plinss: tabatkins is working on migrating the HP harness to the w3c server
16:17:26 [plinss_]
s/to/from/
16:18:01 [smfr]
sylvaing: apple probably won't make 10/15, nor google or mozilla
16:18:19 [smfr]
sylvaing: google WILL make it (according to tabatkins )
16:18:36 [sylvaing]
correction: microsoft+google indicated they'd make the date; apple,mozilla and opera not, it seems
16:19:28 [smfr]
sylvaing: we wont' have two implementations for each testcase
16:20:13 [dbaron]
BTW, shouldn't you be talking about 10/18 rather than 10/15 given when the test suite and implementation report template were available?
16:20:39 [smfr]
yes, should be 10/18
16:20:41 [sylvaing]
yes, 10/18; corrected
16:21:22 [bradk]
What about others, such as Prince?
16:21:25 [smfr]
plinss: don't have definite answer for mozilla and opera, iffy for apple
16:21:54 [smfr]
CSS 2.1 CR Exit Criteria
16:22:19 [smfr]
relaxed exit criteria to allow use of current betas
16:23:15 [smfr]
smfr: do webkit nightly builds count? not public betas, but are downloadable
16:23:29 [smfr]
bradk: opera have the same problem
16:24:13 [smfr]
plinss: nightlies too unstable
16:24:58 [dsinger]
dsinger has joined #css
16:25:06 [dsinger]
zakim, unmute dsinger
16:25:06 [Zakim]
dsinger should no longer be muted
16:25:26 [smfr]
plinss: nightlies are ok so long as they have been available for a min. of a month
16:25:40 [dsinger]
that's more it. we want that the *feature* is stable, not that the *build* is old
16:26:09 [smfr]
sylvaing: we have two vendors submitting reports by 10/18. can the other vendors estimate when they can submit?
16:26:11 [smfr]
smfr: can't for apple
16:26:29 [bradk]
Is it true that the feature is stable if it is not prefixed?
16:26:34 [dsinger]
so, ask that the test passes in a build at least two weeks old (and that it hasn't broken in the meantime)
16:26:48 [smfr]
plinss: do we want to relax exit criteria for 2.1?
16:26:54 [Zakim]
+[Apple]
16:27:03 [Zakim]
-dsinger
16:27:08 [dsinger]
zakim, [apple] has dsinger
16:27:08 [Zakim]
+dsinger; got it
16:27:51 [smfr]
glazou: if we relax the build criteria, will it help?
16:27:59 [smfr]
sylvaing: it's not the build, is the cost of running the tests
16:28:06 [smfr]
s/is/it is
16:28:41 [smfr]
glazou: we should be pragmatic. we should do whatever we need to make CSS 2.1 a Rec.
16:29:11 [smfr]
sylvaing: there are other issues
16:30:45 [smfr]
discussion at TPAC will ensure
16:30:55 [smfr]
s/ensure/ensue
16:31:40 [smfr]
sylvaing: maybe we can relax the rule for 2 passes on all tests
16:32:00 [smfr]
plinss: do we have to have complete report from each vendor, or just enough to show 2 impls passing each test
16:32:59 [smfr]
smfr: what happens if tests fail in all impls?
16:33:13 [smfr]
arronei: there are 126 failures in all browsers
16:33:40 [dbaron]
Does that 126 number include corrections from when that list was discussed on public-css-testsuite?
16:33:57 [dbaron]
(i.e., errors in some of the tests were pointed out)
16:34:20 [smfr]
sylvaing: how many tests fail in one or both of IE and chrome (the 2 impl. reports) we have
16:34:37 [smfr]
glazou: does arronei have reports for all browsers
16:35:20 [smfr]
sylvaing: yes, but MS will not submit reports for other vendors
16:35:34 [smfr]
but MS willing to help share to cross-check the results
16:38:50 [smfr]
test
16:39:15 [smfr]
smfr talked about lack of automation
16:39:41 [smfr]
dsinger: what's the real issue
16:39:54 [smfr]
glazou: it's a problem for tests that don't pass in 2 browsers
16:40:26 [smfr]
sylvaing: let's say MS and chrome submit, and other vendors focus on failing tests
16:40:40 [smfr]
sylvaing: can we still go with that?
16:41:15 [smfr]
glazou: we can take it to the letter. "we need 2 implementations for each feature"
16:41:45 [smfr]
dsinger: we've met the spirit of the law, if not necessarily the letter
16:42:01 [sylvaing]
(smfr, I think it's the reverse actually..)
16:42:10 [dsinger]
or rather, we met the letter of the law, but more importantly, we also can say we met the spirit
16:42:20 [smfr]
glazou: instead of submitting 4 columns of pass/fail, we list 2 browsers for each
16:42:25 [JohnJansen]
it looks like there are about 1800 tests that neither IE nor Chrome pass
16:42:53 [JohnJansen]
or rather about 7600 tests that we both pass
16:42:56 [smfr]
szilles: it's up to the browser vendors to control how they look
16:43:24 [smfr]
JohnJansen: 1800 tests that either IE or chrome fail
16:43:34 [smfr]
arronei: it's only 126
16:44:29 [smfr]
we have 1800 lacking two passes (from IE or chrome)
16:44:35 [smfr]
so other vendors should focus on those tests
16:45:04 [smfr]
sylvaing: if we had mozilla, what would the number be?
16:45:47 [smfr]
dsinger: how many fail because the test is wrong?
16:46:27 [smfr]
smfr / arronei: those tests are gradually being addressed
16:46:36 [smfr]
dsinger: what about two tests that can't both be passed?
16:46:43 [smfr]
arronei: haven't come across any of those
16:47:07 [dbaron]
I've come across two that can't both be passed
16:47:09 [smfr]
smfr: is the feature == test assumption for exit realistic?
16:47:11 [dbaron]
but it was due to an error in one of them
16:47:22 [dbaron]
feature == test was never the assumption
16:48:33 [smfr]
smfr: a "feature" is covered by a set of tests, maybe we shouldn't require passes of all tests for a given feature
16:48:40 [smfr]
plinss: we've done that before
16:48:56 [smfr]
sylvaing: question for opera. do they need to submit data for 3 platforms
16:49:10 [smfr]
plinss: it's not necessary, may be helpful if only one platform passes
16:49:44 [smfr]
plinss: cannot count different platforms as different implementations
16:50:33 [smfr]
sylvaing: re: feature vs. test: since we have 20% of tests failing, it doesn't matter much
16:51:10 [smfr]
plinss: to conclude
16:51:20 [smfr]
plinss: partial reports from some vendors are ok
16:51:28 [sylvaing]
any objections ?
16:51:30 [smfr]
no objections
16:52:11 [smfr]
plinss: can we get a list of the 1800 tests that we need reports for
16:52:36 [smfr]
JohnJansen: MS can submit its report on 9/29
16:53:19 [smfr]
we don't know when chrome will submit
16:53:20 [smfr]
plinss: can MS publish an informal list of where other vendors need to focus?
16:53:52 [smfr]
JohnJansen: have to check
16:54:38 [smfr]
JohnJansen: if we have mozilla, that 1800 number goes way down
16:55:04 [smfr]
glazou: 7600 tests pass in both browsers, in both XHTML1 and HTML4, or just one?
16:55:05 [smfr]
sylvaing: both
16:55:25 [smfr]
s/ sylvaing / arronei
16:55:30 [smfr]
MS is testing IE9 beta
16:55:43 [JohnJansen]
s/ both / IE9 beta and Chrome
16:55:54 [smfr]
plinss: can we get resolution on exit criteria? IE9 beta is not good enough
16:56:05 [smfr]
exit criteria currently state "shipping builds"
16:56:20 [smfr]
JohnJansen: by the time of publishing, it will have been out for 30 days
16:56:27 [dbaron]
did we previously resolve to change the 2.1 exit criteria to match the ones we've recently been using for css3 modules?
16:57:17 [smfr]
RESOLUTION: will change exit criteria to 2 publicly available builds (including nightlies and betas), as long as they have been available to the general public for 1 month
16:58:08 [smfr]
plinss: should not include experimental builds, or builds made to just pass a test
16:58:43 [smfr]
plinss: intent should be that the feature should be present in nightlies for a month
16:58:51 [dbaron]
builds along a development line intended for a release?
16:59:00 [dsinger]
basically, we have to defend the results with a straight face. that's the bottom line.
16:59:52 [smfr]
plinss: this is adopting for 2.1 what we have been doing for CSS 3
17:00:24 [smfr]
RESOLVED: adopt current exit criteria for CSS 2.1
17:01:08 [smfr]
sylvaing: have an action to talk to tabatkins to see how chrome is doing, and share testcases that don't pass in both
17:01:17 [smfr]
sylvaing: any actions on other vendors?
17:01:46 [smfr]
action on other vendors: get implementation reports done; partial reports are acceptable
17:01:46 [trackbot]
Sorry, couldn't find user - on
17:02:01 [smfr]
plinss: no intention to slip the dates
17:02:21 [smfr]
glazou: arronei, did you run the tests manually at least once?
17:02:31 [smfr]
glazou: how many per day?
17:02:40 [smfr]
arronei: 600 tests in an hour, manually
17:06:33 [smfr]
smfr: i would like some kind of basic harness to come with the test suite
17:06:43 [Zakim]
-SteveZ
17:06:44 [Zakim]
-arronei
17:06:46 [Zakim]
-[Apple]
17:06:46 [Zakim]
-johnjansen
17:06:47 [Zakim]
-sylvaing
17:06:54 [Zakim]
-[plinss]
17:07:01 [Zakim]
-bradk
17:07:02 [smfr]
plinss: do I have to do anything with the minutes?
17:07:08 [Zakim]
-smfr
17:07:09 [dbaron]
dbaron has left #css
17:09:57 [plinss_]
smfr: there's a script to format them nicely and they get posted to www-style, fantasai can help...
17:10:01 [Zakim]
-glazou
17:10:02 [Zakim]
Style_CSS FP()12:00PM has ended
17:10:04 [Zakim]
Attendees were [plinss], dsinger, sylvaing, arronei, glazou, johnjansen, bradk, SteveZ, smfr
17:27:59 [oyvind]
oyvind has left #css
17:28:02 [myakura]
myakura has joined #css
17:50:06 [myakura]
myakura has joined #css
17:53:51 [dbaron]
dbaron has joined #css
18:06:27 [alexmog]
alexmog has joined #css
18:19:46 [smfr]
smfr has left #css
18:39:07 [Zakim]
Zakim has left #CSS
18:56:02 [nimbupani]
nimbupani has joined #css
19:28:25 [anne]
anne has joined #css
20:23:43 [kennyluck]
kennyluck has joined #CSS
21:41:28 [hyatt]
hyatt has joined #css