Fwd: Chrome HTML4 implementation report (nearly) finished

Sending to the public list, because there's no particular reason for
our implementation report to be private.

~TJ


---------- Forwarded message ----------
From: Tab Atkins Jr. <jackalmage@gmail.com>
Date: Mon, Oct 4, 2010 at 2:39 PM
Subject: Chrome HTML4 implementation report (nearly) finished
To: w3c-css-wg <w3c-css-wg@w3.org>


File is attached.

Some files were skipped, almost all due to 404s in the testsuite or
the inability to easily do paged tests (Chrome doesn't have a print
preview).  I'll list all of these later in the email.

There were also a handful of tests I marked invalid.

Doing this took about 10 hours, which I spread across 3 days for my
sanity. (thanks for the harness, Simon!).  I'm somewhat loathe to
repeat the entire process for the XHTML1 suite.  I'd *really* like to
do a chunk of reftest automation first - apparently, we could create
only 100 refs and wipe out a third of the suite.

Also, one thing that took me a while was finding a font that would
render Deseret.  I finally found an excellent site for this:
http://users.teilar.gr/~g1951d/ .  I could render all the other
scripts using the system default fonts.

Invalid Tests
=============

border-collapse-applies-to-001.htm
border-collapse-applies-to-005.htm
border-collapse-applies-to-008.htm
border-collapse-applies-to-009.htm
border-collapse-applies-to-010.htm
border-collapse-applies-to-011.htm
border-collapse-applies-to-014.htm
All of these tests say "below" when they mean "to the side of".
Chrome passes all of them if the pass condition is changed
appropriately.

font-weight-normal-001.htm
font-weight-lighter-001.htm
font-weight-bolder-001.htm
These are reftests, but the tests are written in such a way that it's
impossible to pass - some of the columns in the test contain "A"s,
while the similar columns in the ref contain weight numbers.  (The
contents of the cells aren't being tested, just the
boldness/lightness.)

html4/line-height-111.htm       invalid
The pass condition only mentions two boxes, but there are 3 boxes in
the actual test.

html4/before-after-images-001.htm       invalid
The ref is badly constructed - it assumes a particular rendering of
missing images (particularly, FF's rendering).  Chrome's rendering of
the test is perfectly valid.  The ref should be changed to actually
include an <img> pointing to a missing image, so that it will match
the test no matter how the browser renders a missing image.

html4/bidi-alt-001.htm  invalid
This test was also designed with FF's alt-text behavior in mind.
Chrome's rendering is perfectly fine by CSS2.1.

html4/rtl-linebreak.htm invalid
Chrome's rendering of the test matches the ref, but the stated pass
condition doesn't match either of them.  In Chrome, the "Two" only has
a border on top and bottom in both test and ref.

html4/inlines-003.htm   invalid
I have no clue what the pass condition is supposed to be for this.
It's definitely not a "more obvious version of the X grid test", as it
claims.

html4/before-after-dynamic-restyle-001.htm      invalid
The test matches the ref, but both contain quite a bit of red.  My
natural instinct is to mark it failing, despite the match between the
two.  This should be changed to show no red when it passes.  (If it's
supposed to show no red on a pass, then the ref is invalid.)

html4/font-size-123.htm invalid
html4/quotes-035.htm    invalid
html4/quotes-036.htm    invalid
These tests have no pass condition.  I have no idea how to tell if
they've passed or failed.


Skipped Tests
=============

html4/absolute-replaced-height-006.htm  skipped (404)
html4/absolute-replaced-height-013.htm  skipped (404)
html4/absolute-replaced-height-020.htm  skipped (404)
html4/absolute-replaced-height-027.htm  skipped (404)
html4/absolute-replaced-height-034.htm  skipped (404)
html4/absolute-replaced-width-002.htm   skipped (404)
html4/absolute-replaced-width-003.htm   skipped (404)
html4/absolute-replaced-width-004.htm   skipped (404)
html4/absolute-replaced-width-009.htm   skipped (404)
html4/absolute-replaced-width-010.htm   skipped (404)
html4/absolute-replaced-width-011.htm   skipped (404)
html4/absolute-replaced-width-016.htm   skipped (404)
html4/absolute-replaced-width-017.htm   skipped (404)
html4/absolute-replaced-width-018.htm   skipped (404)
html4/absolute-replaced-width-023.htm   skipped (404)
html4/absolute-replaced-width-024.htm   skipped (404)
html4/absolute-replaced-width-025.htm   skipped (404)
html4/absolute-replaced-width-030.htm   skipped (404)
html4/absolute-replaced-width-031.htm   skipped (404)
html4/absolute-replaced-width-032.htm   skipped (404)
html4/absolute-replaced-width-037.htm   skipped (404)
html4/absolute-replaced-width-038.htm   skipped (404)
html4/absolute-replaced-width-039.htm   skipped (404)
html4/absolute-replaced-width-051.htm   skipped (404)
html4/absolute-replaced-width-052.htm   skipped (404)
html4/absolute-replaced-width-053.htm   skipped (404)
html4/absolute-replaced-width-065.htm   skipped (404)
html4/absolute-replaced-width-066.htm   skipped (404)
html4/absolute-replaced-width-067.htm   skipped (404)
html4/at-charset-012.htm        skipped (404)
html4/at-charset-013.htm        skipped (404)
html4/attribute-value-selector-008.htm  skipped (404)
html4/attribute-value-selector-009.htm  skipped (404)
html4/block-replaced-height-006.htm     skipped (404)
html4/block-replaced-width-002.htm      skipped (404)
html4/block-replaced-width-003.htm      skipped (404)
html4/block-replaced-width-004.htm      skipped (404)
html4/content-attr-case-002.htm skipped (404)
html4/float-replaced-height-006.htm     skipped (404)
html4/float-replaced-width-007.htm      skipped (404)
html4/float-replaced-width-008.htm      skipped (404)
html4/float-replaced-width-009.htm      skipped (404)
html4/frameset-border-spacing-001.htm   skipped (404)
html4/inline-block-replaced-height-006.htm      skipped (404)
html4/inline-block-replaced-width-002.htm       skipped (404)
html4/inline-block-replaced-width-003.htm       skipped (404)
html4/inline-block-replaced-width-004.htm       skipped (404)
html4/inline-replaced-height-006.htm    skipped (404)
html4/inline-replaced-width-002.htm     skipped (404)
html4/inline-replaced-width-003.htm     skipped (404)
html4/inline-replaced-width-004.htm     skipped (404)
html4/lang-pseudoclass-002.htm  skipped (404)
All of these had either a missing file or a missing ref.

html4/abspos-paged-001.htm      skipped (no print preview)
html4/abspos-paged-002.htm      skipped (no print preview)
html4/allowed-page-breaks-001a.htm      skipped (no print preview)
html4/allowed-page-breaks-001b.htm      skipped (no print preview)
html4/allowed-page-breaks-001c.htm      skipped (no print preview)
html4/allowed-page-breaks-002.htm       skipped (no print preview)
html4/allowed-page-breaks-003.htm       skipped (no print preview)
html4/allowed-page-breaks-004.htm       skipped (no print preview)
html4/allowed-page-breaks-005.htm       skipped (no print preview)
html4/allowed-page-breaks-006.htm       skipped (no print preview)
html4/allowed-page-breaks-007a.htm      skipped (no print preview)
html4/allowed-page-breaks-007b.htm      skipped (no print preview)
html4/at-import-008.htm skipped (no print preview)
html4/at-page-rule-001.htm      skipped (no print preview)
html4/at-page-rule-002-a.htm    skipped (no print preview)
html4/at-page-rule-002-c.htm    skipped (no print preview)
html4/background-attachment-010.htm     skipped (no print preview)
html4/containing-block-024.htm  skipped (no print preview)
html4/containing-block-025.htm  skipped (no print preview)
html4/first-page-selectors-001.htm      skipped (no print preview)
html4/first-page-selectors-002.htm      skipped (no print preview)
html4/first-page-selectors-003.htm      skipped (no print preview)
html4/first-page-selectors-004.htm      skipped (no print preview)
html4/forced-page-breaks-000.htm        skipped (no print preview)
html4/forced-page-breaks-001.htm        skipped (no print preview)
html4/forced-page-breaks-002.htm        skipped (no print preview)
html4/orphans-001.htm   skipped (no print preview)
html4/orphans-002.htm   skipped (no print preview)
html4/orphans-003.htm   skipped (no print preview)
html4/orphans-004a.htm  skipped (no print preview)
html4/orphans-004b.htm  skipped (no print preview)
html4/orphans-004c.htm  skipped (no print preview)
html4/overflow-print-001.htm    skipped (no print preview)
html4/page-box-000.htm  skipped (no print preview)
html4/page-break-after-000.htm  skipped (no print preview)
html4/page-break-after-001.htm  skipped (no print preview)
html4/page-break-after-002.htm  skipped (no print preview)
html4/page-break-after-003.htm  skipped (no print preview)
html4/page-break-after-004.htm  skipped (no print preview)
html4/page-break-after-005.htm  skipped (no print preview)
html4/page-break-after-006.htm  skipped (no print preview)
html4/page-break-after-007.htm  skipped (no print preview)
html4/page-break-after-008.htm  skipped (no print preview)
html4/page-break-after-009.htm  skipped (no print preview)
html4/page-break-after-010.htm  skipped (no print preview)
html4/page-break-before-000.htm skipped (no print preview)
html4/page-break-before-001.htm skipped (no print preview)
html4/page-break-before-002.htm skipped (no print preview)
html4/page-break-before-003.htm skipped (no print preview)
html4/page-break-before-004.htm skipped (no print preview)
html4/page-break-before-005.htm skipped (no print preview)
html4/page-break-before-006.htm skipped (no print preview)
html4/page-break-before-007-b.htm       skipped (no print preview)
html4/page-break-before-007.htm skipped (no print preview)
html4/page-break-before-008.htm skipped (no print preview)
html4/page-break-before-009.htm skipped (no print preview)
html4/page-break-before-010.htm skipped (no print preview)
html4/page-break-before-011.htm skipped (no print preview)
html4/page-break-before-020.htm skipped (no print preview)
html4/page-break-inside-000.htm skipped (no print preview)
html4/page-break-inside-001.htm skipped (no print preview)
html4/page-break-inside-002.htm skipped (no print preview)
html4/page-break-inside-003.htm skipped (no print preview)
html4/page-break-inside-004.htm skipped (no print preview)
html4/page-break-inside-005.htm skipped (no print preview)
html4/page-break-inside-006.htm skipped (no print preview)
html4/page-break-margins-000.htm        skipped (no print preview)
html4/page-break-margins-001.htm        skipped (no print preview)
html4/page-break-margins-002.htm        skipped (no print preview)
html4/page-break-margins-003.htm        skipped (no print preview)
html4/page-break-margins-004.htm        skipped (no print preview)
html4/page-breaks-100.htm       skipped (no print preview)
html4/page-breaks-101.htm       skipped (no print preview)
html4/page-container-000.htm    skipped (no print preview)
html4/page-container-001.htm    skipped (no print preview)
html4/page-container-002.htm    skipped (no print preview)
html4/page-container-003.htm    skipped (no print preview)
html4/page-container-004.htm    skipped (no print preview)
html4/page-container-005.htm    skipped (no print preview)
html4/page-container-006.htm    skipped (no print preview)
html4/page-container-007.htm    skipped (no print preview)
html4/page-container-008.htm    skipped (no print preview)
html4/page-container-009.htm    skipped (no print preview)
html4/page-container-010.htm    skipped (no print preview)
html4/page-container-011.htm    skipped (no print preview)
html4/page-grammar-002.htm      skipped (no print preview)
html4/page-margin-000.htm       skipped (no print preview)
html4/page-margin-001.htm       skipped (no print preview)
html4/page-margin-002.htm       skipped (no print preview)
html4/page-props-100-a.htm      skipped (no print preview)
html4/page-props-100-b.htm      skipped (no print preview)
html4/page-props-101.htm        skipped (no print preview)
html4/page-props-102.htm        skipped (no print preview)
html4/page-props-103.htm        skipped (no print preview)
html4/page-selectors-001.htm    skipped (no print preview)
html4/page-selectors-002.htm    skipped (no print preview)
html4/page-selectors-003.htm    skipped (no print preview)
html4/page-selectors-004.htm    skipped (no print preview)
html4/page-selectors-006.htm    skipped (no print preview)
html4/position-fixed-004.htm    skipped (no print preview)
html4/position-fixed-005.htm    skipped (no print preview)
html4/table-footer-group-004.htm        skipped (no print preview)
html4/table-header-group-004.htm        skipped (no print preview)
html4/widows-001.htm    skipped (no print preview)
html4/widows-002.htm    skipped (no print preview)
html4/widows-003.htm    skipped (no print preview)
html4/widows-004a.htm   skipped (no print preview)
html4/widows-004b.htm   skipped (no print preview)
html4/widows-004c.htm   skipped (no print preview)
I skipped all of these because Chrome has no print preview, so I would
have had to print to paper or a file to do them.  I'll get around to
that, I just didn't want to bother with it while running the rest of
the tests.

html4/margin-collapse-160.htm   skipped (can't switch to projection or
print mode)
I can't switch to projection mode, and I didn't want to interrupt my
flow to print it to a file.  I'll get to this one later.

html4/cascade-001.htm   skipped (no user stylesheets)
html4/cascade-002.htm   skipped (no user stylesheets)
html4/cascade-003.htm   skipped (no user stylesheets)
html4/cascade-004.htm   skipped (no user stylesheets)
html4/cascade-006.htm   skipped (no user stylesheets)
html4/cascade-010.htm   skipped (no user stylesheets)
html4/cascade-011.htm   skipped (no user stylesheets)
html4/html-precedence-003.htm   skipped (no user stylesheets)
html4/user-stylesheet-001.htm   skipped (no user stylesheets)
html4/user-stylesheet-002.htm   skipped (no user stylesheets)
html4/user-stylesheet-003.htm   skipped (no user stylesheets)
html4/user-stylesheet-004.htm   skipped (no user stylesheets)
html4/user-stylesheet-005.htm   skipped (no user stylesheets)
html4/user-stylesheet-006.htm   skipped (no user stylesheets)
html4/user-stylesheet-007.htm   skipped (no user stylesheets)
html4/user-stylesheet-008.htm   skipped (no user stylesheets)
html4/user-stylesheet-009.htm   skipped (no user stylesheets)
html4/user-stylesheet-010.htm   skipped (no user stylesheets)
html4/user-stylesheet-011.htm   skipped (no user stylesheets)
html4/user-stylesheet-012.htm   skipped (no user stylesheets)
html4/user-stylesheet-013.htm   skipped (no user stylesheets)
html4/user-stylesheet-014.htm   skipped (no user stylesheets)
html4/user-stylesheet-015.htm   skipped (no user stylesheets)
html4/user-stylesheet-016.htm   skipped (no user stylesheets)
html4/user-stylesheet-017.htm   skipped (no user stylesheets)
html4/user-stylesheet-018.htm   skipped (no user stylesheets)
I skipped these because Chrome doesn't yet have user stylesheets, so
these are untestable.

html4/lang-selector-005.htm     skipped (can't set)
I can't set the language of a page in Chrome.

~TJ