Re: Writing Modes PR & Testsuite

In principle, the tests fail in all UAs, despite what the test harness
currently lists, and as such we should be explaining failures that
amount to almost half the testsuite to the Director when we want to
move to PR. That is, IMO, ridiculous. Looking at this some more, it
turns out that many of these aren't down to anti-aliasing but many are
down to image scaling. We now have a bug at
<https://github.com/w3c/csswg-test/issues/1174> for the mass-failures.
I'll try to get a decent summary of what's needed to fix everything
later today or tomorrow.

/g

On Thu, Jan 12, 2017 at 3:31 PM, Koji Ishii <kojiishi@gmail.com> wrote:
> Thank you Geoffrey for running the tests and filing a test bug. Anti-alias
> issues are not easy to deal with, as far as I understand, Gérard and Xidorn
> are working on it in https://bugzilla.mozilla.org/show_bug.cgi?id=1258916.
> In general, Gérard is following the CSS test guidelines to avoid anti-alias
> issues, but the guidelines don't seem to be perfect. I agree that we should
> continue working on improving tests, and multiple vendors working together
> is important for issues like this, so your inputs are highly appreciated.
>
> I don't think, however, anti-alias issues can affect the spec status, tests
> can and should be improved regardless of the spec status. Hope W3C process
> experts can commet if I'm wrong.
>
> /koji
>
> 2017-01-12 14:48 GMT-08:00 Geoffrey Sneddon <me@gsnedders.com>:
>>
>> Yesterday, we resolved to publish either a CR or, preferably if the
>> Process allows it, a PR of Writing Modes Level 3.
>>
>> However, I'm unconvinced we have a testsuite sufficient to leave CR
>> (and proceed to PR) with; we seem to have hundreds of reftests which
>> fail due to anti-aliasing differences.
>>
>> According to the test harness, Firefox passes 89.02% of the testsuite
>> (which it makes out to be 1120 tests); running locally all automated
>> tests, I get 1045 tests (910 parents, 135 subtests; as far as I'm
>> aware the harness has no notion of harnesses), of which 457 pass and
>> 588 fail: this implies that 89.02% of the entire testsuite cannot
>> pass.
>>
>> Looking into many of the failures, it quickly became apparent that
>> hundreds of these failures is down to anti-aliasing differences in
>> reftests; I've filed a bug for this at
>> <https://github.com/w3c/csswg-test/issues/1174>.
>>
>> As such, I at least don't view the testsuite as ready to publish a PR
>> and would raise a formal objection if we resolved to do so.
>>
>> /Geoffrey.
>>
>

Received on Friday, 13 January 2017 02:03:35 UTC