TestPlan-Format

From Fonts WG
Revision as of 19:04, 5 February 2011 by Tleming (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Test Plan - Format

Plan for testing all the assertions which apply to the WOFF file format itself. The majority of these will be tested by a validation tool.


The work on the test suite that has been completed can bee seen in CVS or by visiting this page.


Text color highlights on this page are as follows: red text indicates questions that need to be addressed by the Working Group, gray text indicates portions of the test suite that are not yet implemented and blue text indicates initial ideas about how the tests can be presented to User Agents.

File Structure Tests

Header

signature (magicnumber)

Test Case Description
Header Signature Invalid Value The signature field contains "XXXX" instead of "wOFF".

flavor

Test Case Description
Header Flavor Incorrectly Set to 0x00010000 The header flavor is set to 0x00010000 but the table data contains CFF data, not TTF data.
Header Flavor Incorrectly Set to OTTO The header flavor is set to OTTO but the table data contains TTF data, not CFF data.

length

This should be a testable assertion in the WOFF Header section of the spec. It is labeled as a MUST in the Summary of Conformance Requirements.

Test Case Description
Header Length Too Short The length field contains a value that is four bytes shorter than the actual data.
Header Length Too Long The length field contains a value that is four bytes longer than the actual data.

numTables

Test Case Description
Header Number of Tables Set to Zero The header contains 0 in the numTables field. A table directory and table data are present.

totalSfntSize (longword)

Test Case Description
Header totalSfntSize Not a Multiple of 4 The totalSfntSize field contains a value that is missing padding bytes between two tables.
Header Total SFNT Size Too Long The totalSfntSize field contains a value that is is four bytes too long.
Header Total SFNT Size Too Short The totalSfntSize field contains a value that is is four bytes too short.

reserved (reserved)

Test Case Description
Header Number of Tables Set to Zero The header contains 0 in the numTables field. A table directory and table data are present.

Data Blocks

Extraneous Data (noextraneous)

Test Case Description
Extraneous Data Between Directory and Table Data There are four null bytes between the table directory and the table data.
Extraneous Data Between Tables There are four null bytes between each of the table data blocks.
Extraneous Data After Table Data There are four null bytes after the table data block and there is no metadata or private data.
Extraneous Data Between Table Data and Metadata There are four null bytes between the table data and the metadata.
Extraneous Data Between Table Data and Private Data There are four null bytes between the table data and the private data.
Extraneous Data Between Metdata and Private Data There are four null bytes between the metadata and the private data.
Extraneous Data After Metadata There are four null bytes after the metadata and there is no private data.
Extraneous Data After Private Data There are four null bytes after the private data.

Overlaps

This is the User Agent overlap-reject testable assertion. This should be tagged as a file format testable assertion as well.

Test Case Description
Table Data Blocks Overlap The second table's offset is four bytes before the end of the first table's data.
Metadata Overlaps Table Data The metadata offset is four bytes before the end of the table data.
Private Data Overlaps Table Data The private data offset is four bytes before the end of the table data.
Private Data Overlaps Metadata The private data offset is four bytes before the end of the metadata.

Another approach to making the files invalid would be to increase the length in each data block by four bytes. The effect would be the same as the current cases, but maybe these kinds of bad files should also be part of the test suite for completeness. It may also be a good idea to create test cases that have offsets that are before the end of the header or the end of the table directory.

Metadata Not Present (zerometaprivate)

Test Case Description
Metadata Length Not Set to Zero The metadata length is set to one but the offset is zero. This test case also fails for other reasons: the offset/length creates an overlap with the header block, the metadata won't decompress.
Metadata Offset Not Set to Zero The metadata length is set to zero but the offset is set to the end of the file.

Private Data Not Present (zerometaprivate)

Test Case Description
Private Data Length Not Set to Zero The private data length is set to one but the offset is zero. This test case also fails for another reason: the offset/length creates an overlap with the header block.
Private Data Offset Not Set to Zero The private data length is set to zero but the offset is set to the end of the file.

Ordering (afterdirectory, metadata-afterfonttable and private-last)

Test Case Description Table Data After Metadata The table data block is stored after the metadata block. This is the afterdirectory test case.
Table Data After Private Data The table data block is stored after the private data block. This is the afterdirectory test case.
Metadata After Private Data The metadata block is stored after the private data block. This is the metadata-afterfonttable test case.
Private Data Before Metadata The private data block is stored before the metadata block. This is the private-last test case.

metadata-noprivatepad

Test Case Description
Metadata Has Unnecessary Padding The metadata block is padded to a four-byte boundary but there is no private data.ding 1

Table Directory

4-Byte Boundary (tablesize-longword)

Test Case Description
Font Table Data Not On 4-Byte Boundary Two vendor-space tables are inserted into the table directory and data: AAAA is three bytes long. AAAB is five bytes long. AAAA is not padded so that the AAAB table does not begin on a four-byte boundary. The tables that follow are all aligned correctly.
Final Font Table Data Not Padded The final table in the table data block is not padded to a 4-byte boundary.
Font Table Data Padded With Non-Null Table data is padded with \01 instead of \00.

Overlaps

This is the User Agent diroverlap-reject testable assertion. Should it be tagged as a file format testable assertion as well?

Test Case Description
Font Table Data Offset Past End of File The offset to the data block for the final table data is four bytes beyond the end of the file.
Font Table Data Offset+Length Past End of File The defined length for the final table causes the data block to be four bytes beyond the end of the file.
Font Table Data Overlaps Metadata The final table starts four bytes after the start of the metadata. This will fail for another reason: the calculated length (header length + directory length + entry lengths + metadata length) will not match the stored length in the header.
Font Table Data Overlaps Private Data The final table starts four bytes after the start of the private data. This will fail for another reason: the calculated length (header length + directory length + entry lengths + private data length) will not match the stored length in the header.
Two Table Data Blocks Overlap The final table starts four bytes before the end of the previous table. This will fail for another reason: the calculated length (header length + directory length + entry lengths) will not match the stored length in the header.

compLength (compressedlarger)

Test Case Test Case Notes
Font Table Data Compressed Length Greater Than Original Length At least one table's compLength is larger than the origLength.

origLength (origLength)

Test Case Description
Original Length Less Than Decompressed Length One table has table that when decompressed has a length that is four bytes longer than the value listed in origLength.
Original Length Greater Than Decompressed Length One table has table that when decompressed has a length that is four bytes shorter than the value listed in origLength.

origChecksum

This is the authoring tool checksumvalidate testable assertion. A validator does have enough information to verify checksums. As stated several times, validating the checksums must not be a requirement of user agents. But it may be a good idea to make it a requirement of the file format as well as the authoring tool.

Should the head checkSumAdjustment field also be verified?

Test Case Description
Font Table Directory Contains Invalid Original CheckSum The checksum for the CFF table is set to 0.
Font head Table CheckSum Incorrect Adjustment The head table checksum adjustment is set to 0.

Ascending Order (ascending)

Test Case Description
Font Table Directory Not In Ascending Order The table directory is stored in descending order.

Order (sameorder)

This one is going to be tricky one for validators to check. The validator will need both the WOFF and the original SFNT. Considering that in many cases the web author will not have the original SFNT they won't be able to perform validation on the WOFF. Perhaps this testable assertion could be applied to authoring tools instead of the file format.

Table Data

Compression

Test Case Description
Font Table Data Not Compressed None of the tables are stored in compressed form.
Font Table Data Is Compressed When Possible All of the tables that will be smaller when compressed are stored in their compressed state.
Not All Font Table Data Is Compressed When Possible Only one of the tables that would be smaller when compressed is stored in the compressed state.
Font Table Data Is Compressed At Different Levels The font data tables are compressed using at least two different levels.

Compression Format (mustzlib)

Test Case Description
Font Table Data Invalid Compressed Data One compressed table has had its compressed data replaced with \01 making it incompatible with zlib.

Metadata

metaOrigLength

Test Case Description
Decompressed Metadata Length Greater Than metaOrigLength The metadata decompressed to a length that is 1 byte greater than the length defined in metaOrigLength
Decompressed Metadata Length Less Than metaOrigLength The metadata decompressed to a length that is 1 byte smaller than the length defined in metaOrigLength

Metadata

Invalid (invalid-mustignore)

Test Case Description
Metadata Invalid Compression The metadata is stored in an uncompressed state and therefore does not have the proper compression format. (metadata-alwayscompress)
Unescaped < in Content The text element in the description element contains an unescaped <. (metadata-wellformed)
Unescaped & in Content The text element in the description element contains an unescaped &. (metadata-wellformed)
Mismatched Element Tags One element begins with <description> but ends with </mismatch>. (metadata-wellformed)
Unclosed Element Tag The text element element in the description element is not closed. (metadata-wellformed)
Case Mismatch in Element Tags The <description> element is closed with <DESCRIPTION>. (metadata-wellformed)
More Than One Root Element The metadata root element occurs twice. (metadata-wellformed)
Unknown Encoding The xml encoding is set to 'VSCACS-GFV-X-CQ34QTAB2Q-IS-NOT-A-VALID-ENCODING'. (metadata-wellformed)
UTF-8 Encoding The xml encoding is set to UTF-8. (metadata-encoding)
UTF-16 Encoding The xml encoding is set to UTF-16. (metadata-encoding)
Invalid Encoding The xml encoding is set to ISO-8859-1. (metadata-encoding)

Schema Validity

metadata Element

Test Case Description
Valid metadata Element The metadata element matches the schema.
No version Attribute in metadata Element The metadata element does not contain the required version attribute. This should be linked to a testable assertion in the spec.
Invalid version Attribute Value in metadata Element The metadata element version attribute is set to 'ABC'. Is 1.0 a requirement?
Unknown Attrbute in metadata Element The metadata element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element metadata Element The metadata element contains an unknown child element. This should be linked to a testable assertion in the spec.

uniqueid Element

Title Description
Valid uniqueid Element The uniqueid element matches the schema.
No uniqueid Element The uniqueid element doesn't exist. This should be linked to a testable assertion in the spec.
More Than One uniqueid Element The uniqueid element occurs twice. This should be linked to a testable assertion in the spec.
No id Attribute in uniqueid Element The uniqueid element does not contain the required id attribute. This should be linked to a testable assertion in the spec.
Unknown Attribute in uniqueid Element The uniqueid element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Child Element in uniqueid Element The uniqueid element contains a child element. This should be linked to a testable assertion in the spec.
Content in uniqueid Element The uniqueid element contains content. This should be linked to a testable assertion in the spec.

vendor Element

Title Description
Valid vendor Element The vendor element matches the schema.
Valid vendor Element Without url Attribute The vendor element does not contain a url attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
No vendor Element The vendor element doesn't exist. This should be linked to a testable assertion in the spec.
More Than One vendor Element The vendor element occurs twice. This should be linked to a testable assertion in the spec.
No name Attribute in vendor Element The vendor element does not contain the required name attribute. This should be linked to a testable assertion in the spec.
Unknown Attribute in vendor Element The vendor element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Child Element in vendor Element The vendor element contains a child element. This should be linked to a testable assertion in the spec.
Content in vendor Element The vendor element contains content. This should be linked to a testable assertion in the spec.

credits Element

Title Description
Valid credits Element With Single credit Element The credits element matches the schema and it contains one credit child element. This should be linked to a testable assertion in the spec.
Valid credits Element With Two credit Elements The credits element matches the schema and it contains two credit child elements. This should be linked to a testable assertion in the spec.
More Than One credits Element The credits element occurs more than once. This should be linked to a testable assertion in the spec.
No credit Element in credits Element The credits element does not contain a credit child element. This should be linked to a testable assertion in the spec.
Unknown Attribute in credits Element The credits element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in credits Element The credits element contains an unknown child element. This should be linked to a testable assertion in the spec.
Content in credits Element The credits element contains an content. This should be linked to a testable assertion in the spec.

credit Element

Title Description
Valid credit Element The credit element matches the schema.
Valid credit Element Without url Attribute The credit element does not contain a url attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
Valid credit Element Without role Attribute The credit element does not contain a role attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
No name attribute in credit Element The credit element does not contain a name attribute. This should be linked to a testable assertion in the spec.
Unknown attribute in credit Element The credit element contains and unknown attribute. This should be linked to a testable assertion in the spec.
Child Element in credit Element The credit element contains a child element. This should be linked to a testable assertion in the spec.
Content in credit Element The credit element contains content. This should be linked to a testable assertion in the spec.

description Element

Title Description
Valid description Element The description element matches the schema.
Valid description Element Without url Attribute The description element does not contain a url attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
Valid description Element With One No Language Tagged text Element The description element matches the schema. It contains one text element that does not have a language tag. This should be linked to a testable assertion in the spec.
Valid description Element With One Language Tagged text Element The description element matches the schema. It contains one text element that has a language tag. This should be linked to a testable assertion in the spec.
Valid description Element With Mixed text Element Language Tags 1 The description element matches the schema. One text element does not have a language tag. One text element has a language tag. This should be linked to a testable assertion in the spec.
Valid description Element With Mixed text Element Language Tags 2 The description element matches the schema. Two text elements have a language tags. This should be linked to a testable assertion in the spec.
More Than One description Element The description element occurs more than once. This should be linked to a testable assertion in the spec.
No text Element in description Element The description element does not contain a text child element. This should be linked to a testable assertion in the spec.
Unknown Attribute in description Element The description element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in description Element The description element contains an unknown child element. This should be linked to a testable assertion in the spec.
Content in description Element The description element contains content. This should be linked to a testable assertion in the spec.
Unknown Attribute in description Element text Element The description element contains a text element with an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in description Element text Element The description element contains a text element with an unknown child element. This should be linked to a testable assertion in the spec.

license Element

Title Description
Valid license Element The license element matches the schema.
Valid license Element The license element does not have a url attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
Valid license Element The license element does not have an id attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
Valid license Element With One No Language Tagged text Element The license element matches the schema. It contains one text element that does not have a language tag. This should be linked to a testable assertion in the spec.
Valid license Element With One Language Tagged text Element The license element matches the schema. It contains one text element that has a language tag. This should be linked to a testable assertion in the spec.
Valid license Element With Mixed text Element Language Tags 1 The license element matches the schema. One text element does not have a language tag. One text element has a language tag. This should be linked to a testable assertion in the spec.
Valid license Element With Mixed text Element Language Tags 2 The license element matches the schema. Two text elements have a language tags. This should be linked to a testable assertion in the spec.
More Than One license Element The license element occurs more than once. This should be linked to a testable assertion in the spec.
No text Element in license Element The license element does not contain a text child element. This should be linked to a testable assertion in the spec.
Unknown Attribute in license Element The license element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in license Element The license element contains an unknown child element. This should be linked to a testable assertion in the spec.
Content in license Element The license element contains content. This should be linked to a testable assertion in the spec.
Unknown Attribute in license Element text Element The license element contains a text element with an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in license Element text Element The license element contains a text element with an unknown child element. This should be linked to a testable assertion in the spec.

copyright Element

Title Description
Valid copyright Element With One No Language Tagged text Element The copyright element matches the schema. It contains one text element that does not have a language tag. This should be linked to a testable assertion in the spec.
Valid copyright Element With One Language Tagged text Element The copyright element matches the schema. It contains one text element that has a language tag. This should be linked to a testable assertion in the spec.
Valid copyright Element With Mixed text Element Language Tags 1 The copyright element matches the schema. One text element does not have a language tag. One text element has a language tag. This should be linked to a testable assertion in the spec.
Valid copyright Element With Mixed text Element Language Tags 2 The copyright element matches the schema. Two text elements have a language tags. This should be linked to a testable assertion in the spec.
More Than One copyright Element The copyright element occurs more than once. This should be linked to a testable assertion in the spec.
No text Element in copyright Element The copyright element does not contain a text child element. This should be linked to a testable assertion in the spec.
Unknown Attribute in copyright Element The copyright element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in copyright Element The copyright element contains an unknown child element. This should be linked to a testable assertion in the spec.
Content in copyright Element The copyright element contains content. This should be linked to a testable assertion in the spec.
Unknown Attribute in copyright Element text Element The copyright element contains a text element with an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in copyright Element text Element The copyright element contains a text element with an unknown child element. This should be linked to a testable assertion in the spec.

trademark Element

Title Description
Valid trademark Element With One No Language Tagged text Element The trademark element matches the schema. It contains one text element that does not have a language tag.
Valid trademark Element With One Language Tagged text Element The trademark element matches the schema. It contains one text element that has a language tag. This should be linked to a testable assertion in the spec.
Valid trademark Element With Mixed text Element Language Tags 1 The trademark element matches the schema. One text element does not have a language tag. One text element has a language tag. This should be linked to a testable assertion in the spec.
Valid trademark Element With Mixed text Element Language Tags 2 The trademark element matches the schema. Two text elements have a language tags. This should be linked to a testable assertion in the spec.
More Than One trademark Element The trademark element occurs more than once. This should be linked to a testable assertion in the spec.
No text Element in trademark Element The trademark element does not contain a text child element. This should be linked to a testable assertion in the spec.
Unknown Attribute in trademark Element The trademark element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in trademark Element The trademark element contains an unknown child element. This should be linked to a testable assertion in the spec.
Content in trademark Element The trademark element contains content. This should be linked to a testable assertion in the spec.
Unknown Attribute in trademark Element text Element The trademark element contains a text element with an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in trademark Element text Element The trademark element contains a text element with an unknown child element. This should be linked to a testable assertion in the spec.

licensee Element

Title Description
Valid licensee Element The uniqueid element matches the schema.
More Than One licensee Element The uniqueid element occurs more than once. This should be linked to a testable assertion in the spec.
No name Attribute in licensee Element The uniqueid element does not contain the required name attribute. This should be linked to a testable assertion in the spec.
Unknown Attribute in licensee Element The uniqueid element occures more than once. This should be linked to a testable assertion in the spec.
Child Element in licensee Element The uniqueid element contains a child element. This should be linked to a testable assertion in the spec.
Content in licensee Element The uniqueid element contains content. This should be linked to a testable assertion in the spec.

extension element

Title Description
Valid extension Element The extension element matches the schema. This should be linked to a testable assertion in the spec.
Two Valid extension Elements Two extension elements match the schema. This should be linked to a testable assertion in the spec.
Valid extension Element Without id Attribute The extension element does not have an id attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
Valid extension Element Without name Element The extension element does not have a name child element but it still matches the schema. This should be linked to a testable assertion in the spec.
Valid extension Element With Two name Elements 1 The extension element contains one name element without a lang attribute and another with a lang attribute. This should be linked to a testable assertion in the spec.
Valid extension Element With Two name Elements 2 The extension element contains two name elements with lang attributes. This should be linked to a testable assertion in the spec.
Valid extension Element With Two item Elements The extension element contains two item child elements. This should be linked to a testable assertion in the spec.
No item Element in extension Element The extension element does not contain an item child element. This should be linked to a testable assertion in the spec.
Unknown Attribute in extension Element The extension element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in extension Element The extension element contains an unknown child element. This should be linked to a testable assertion in the spec.
Content in extension Element The extension element contains content. This should be linked to a testable assertion in the spec.
Valid item Element in extension Element The item element in the extension element matches the schema. This should be linked to a testable assertion in the spec.
Valid item Element With Multiple Languages in extension Element The item element in the extension element contains a variety of languages. This should be linked to a testable assertion in the spec.
Valid item Element Without id Attribute in extension Element The item element in the extension element does not contain an id attribute but it still matches the schema. This should be linked to a testable assertion in the spec.
Valid item Element With Two name Elements in extension Element 1 The item element in the extension element contains one name child element with no lang attribute and one with a lang attribute. This should be linked to a testable assertion in the spec.
Valid item Element With Two name Elements in extension Element 2 The item element in the extension element contains two name child elements with lang attributes. This should be linked to a testable assertion in the spec.
Valid item Element With Two value Elements in extension Element 1 The item element in the extension element contains one value child element with no lang attribute and one with a lang attribute. This should be linked to a testable assertion in the spec.
Valid item Element With Two value Elements in extension Element 2 The item element in the extension element contains two value child elements with lang attributes. This should be linked to a testable assertion in the spec.
No name Element in item Element in extension Element The item element in the extension element does not contain a name child element. This should be linked to a testable assertion in the spec.
No value Element in item Element in extension Element The item element in the extension element does not contain a value child element. This should be linked to a testable assertion in the spec.
Unknown Attribute in item Element in extension Element The item element in the extension element contains an unknown attribute. This should be linked to a testable assertion in the spec.
Unknown Child Element in item Element in extension Element The item element in the extension element contains an unknown child element. This should be linked to a testable assertion in the spec.
Content in item Element in extension Element The item element in the extension element contains content. This should be linked to a testable assertion in the spec.
Valid name Element in item Element in extension Element The name element in the item element in the extension element matches the schema. This should be linked to a testable assertion in the spec.
Valid name Element With lang Attribute in item Element in extension Element The name element in the item element in the extension element contains a lang attribute and it matches the schema. This should be linked to a testable assertion in the spec.
Unkown Attribute in name Element in item Element in extension Element The name element in the item element in the extension element contains an unkown attribute. This should be linked to a testable assertion in the spec.
Child Element in name Element in item Element in extension Element The name element in the item element in the extension element contains a child element. This should be linked to a testable assertion in the spec.
Valid value Element in item Element in extension Element The value element in the item element in the extension element matches the schema. This should be linked to a testable assertion in the spec.
Valid value Element With lang Attribute in item Element in extension Element The value element in the item element in the extension element contains a lang attribute and it matches the schema. This should be linked to a testable assertion in the spec.
Unkown Attribute in value Element in item Element in extension Element The value element in the item element in the extension element contains an unkown attribute. This should be linked to a testable assertion in the spec.
Child Element in value Element in item Element in extension Element The value element in the item element in the extension element contains a child element. This should be linked to a testable assertion in the spec.

Private Data

4-Byte Alignment (private-padalign)

Test Case Description
Private Data Does Not Begin of 4-Byte Boundary The private data does not begin on a four byte boundary because the metadata is not padded. This will fail for another reason: the calculated length (header length + directory length + entry lengths + metadata length + private data length) will not match the stored length in the header.
Padding Between Metadata and Private Data is Non-Null Metadata is padded with \01 instead of \00.