Creating tests for the i18n test suite

To create tests for the i18n test suite you need to create or add to a data.php file. The data.php file contains the essential information needed for a set of related tests. The file /International/tests/tests-html-css-devt/generate.php converts that data into actual test pages in up to 6 formats (html4.01, html5, and xhtml 1.0 served as text/html, and xhtml 1.0, xhtml5, and xhtml 1.1 served as application/xhtml+xml). To produce an actual text file you call generate.php with parameters to indicate the location of the data.php file, the actual test to display, and the format in which to display it.

For example: /International/tests/tests-html-css-devt/generate.php?directory=text-direction&test=33&format=html5

A data.php file sits in a subdirectory of /International/tests/tests-html-css-devt/ and has a subdirectory of its own called support which contains graphics, stylesheets and other external files needed for the tests in the data.php file.

Creating or adding to a data.php file

Variables at the top of the file

Set the variables at the top of the file. Here is an example:

$firsttest = '11';
$filename = 'character-encoding';

$author="Richard Ishida";
$authorhref = "mailto:ishida@w3.org";

$localstyling = "/* the CSS below is not part of the test */\ntable td { border: 1px solid #CCC; }\nimg { margin: 10px; }\n";

  1. $firsttest indicates which is the first test when the tests are arranged in sequence. Commonly this will be 1, but it allows you to change the start point, if you wish, without wholesale rearrangement of the data.
  2. $filename is the name of the directory in which the data.php sits.
  3. $author is a comma-separated list of test authors. It will appear in every test generated from this data.php file. Same goes for $authorhref, which is a comma-separated list mailtos or URIs.
  4. $localstyling applies CSS styles that are not part of the test itself to every test generated from the data.php file.

Test syntax

Always use strict xhtml syntax for the code of your test. Things like /> will be changed to > by the generate file for html formats.

Test format

The majority of the data.php file is filled with assignments to the test array like those in the following example:

$test[2]=array(
'title'=>'decimal ncr',
'assert'=>'A decimal numeric character reference produces the intended character.',
'instructions'=>'<div style="float:left;"><img src="/International/tests/images/equal.png" alt="equal" /></div><p class="instructions">Test passes if you see the same character twice. (Font differences are irrelevant.)</p><p class="instructionNote" dir="ltr">This test is only valid if ....</p>',
'test'=>'<div class="test">&#225; <br /><img src="escapes/support/aacutelc.png" alt="reference image" /></div>',
'author'=>'Richard Ishida<mailto:ishida@w3.org',
'related'=>'/International/tests/tests-html-css/list-encoding-html-css',
'formats'=>"h4,h5,xh,xx,x5,x11",
'next'=>"3"
);

Data for a specific test always starts with $test[...]=array(. The declaration of the array (ie. $test = array();) appears in the file that includes the data.php file.)

That line is followed by a number of assignments. There should always be the following assignments: title, assert, instructions, test, formats, related, next.

  1. $title is a short unique title for the test. It should be plain text.
  2. $assert states what you expect the test to demonstrate if it passes. It should usually be plain text, but can include simple inline markup. The assertion doesn't describe what you should look for, it describes the expected behaviour that the test is testing. If you are creating a test for something that is not in a specification, you should begin the assertion with '[Exploratory test]'.
  3. $instructions say what to look for to ascertain whether the test supports the assertion. Usually the instructions will start with "Test passes if...". Use block markup around the instructions. Usually this will be <p>, but it should include list markup if there is more than one thing to check for.
    After the basic instruction text, the instructions can contain small notes related to things such as dependencies. For example: <p class="instructionNote" dir="ltr">This test is only valid if ....</p>.
    Where a test involves checking that two things look the same or look different, you can include a graphic in the instructions. Add this code before the paragraph containing the main instruction set when the test requires you to check that two things look the same: <div style="float:left;"><img src="http://www.w3.org/International/tests/images/equal.png" alt="equal" /></div>. When the test passes if two things do NOT look the same, point to the graphic called notequal.png in the same location.
    Here is an example that includes all of the above:
    'instructions'=>'<div style="float:left;"><img src="/International/tests/images/equal.png" alt="equal" /></div><p class="instructions">Test passes if you see the same character twice. (Font differences are irrelevant.)</p><p class="instructionNote" dir="ltr">This test is only valid if ....</p>',
  4. $test contains the markup needed for the test. Usually the markup should be enclosed by <div class="test">...</div>. In the current test style this puts an orange box around the test markup.
  5. $author is a list of comma separated name+'<'+mailto|url sequences that identify the author(s) of the test (for example, 'Richard Ishida<mailto:ishida@w3.org,Aharon Lanin<mailto:aharon@google.org').
  6. $related is a URI that points to a page containing links to this test in other formats and to other tests related to the same general topic (including, usually, all the other tests in this data.php file).
  7. $formats is a list of relevant formats for this test. It should include one of more of the following comma-separated labels: h4, h5, xhtml, xhtmlx, xhtml5, xhtml11.
    These mean the following:
    h4 (html4.01), h5 (html5), xh (xhtml 1.0 served as text/html), xx (xhtml 1.0 served as application/xhtml+xml), x5 (xhtml5), x11 (xhtml 1.1 served as application/xhtml+xml)
    If a test relates to behaviour that is only specified in a subset of these formats (eg. html5 and xhtml5), you would include only those labels here. If you want to ascertain whether the behaviour applies for other formats also (such as the charset attribute on the meta element), create a new test and label the assertion as exploratory and include format labels for whatever other formats you want to test.
  8. $next is the number of the next test in the sequence. If this is the last test, use 'next'=>"".

Optional extra variables

A number of additional variables can be used to override or extend the normal test code produced by the generate.php file. Note that most of these settings will apply the markup specified in all formats produced by generate.php. For example, if you are testing for recognition of meta element encoding declarations you will usually need to use $metaencodingdecls only with $formats set to h4, h5 or xhtml. If you want to use that setting with other XML formats, where its behaviour is not specified, you should create an additional test and label the assertion [Exploratory test] for those formats.

  1. $encoding: Changes the encoding for the test from the default utf-8 to whatever you specify. Value is just the encoding name, eg. iso-8859-1.
  2. $htmlattributes: Usually lang and/or xml:lang, will be added to the html tag, if not present a default will be supplied.
  3. $htmldir: Allows you to set the direction for the document, eg. dir="rtl". It has the side-effect of adding dir=ltr to the assertion, notes, title (but not the instructions - that should be added in the data.php file). For some tests where direction is set on the body element, I used this with a value of a single space, in order to activate the side effect.
  4. $metaencodingdecls: Overrides any meta encoding declarations in the head element, or forces them to appear in formats that don't normally have them. If not present, UTF-8 declarations are supplied for h4, h5 and xhtml. To suppress the meta encoding declaration altogether, use the empty string.
  5. $contentlanguagep: Forces a Content-Language meta element to appear in the head element. If not present, nothing is supplied.
  6. $header: Produces an HTTP header. The value should be the whole header, eg. Content-Language: ko. NB: use $ctheaderencoding for the Content-Type header.
  7. $ctheaderencoding: Will send an HTTP Content-Type header with the value specified, eg. iso-8859-1. To get a header without encoding information, use $ctheaderencoding="".
  8. $xmldeclaration: Adds an XML declaration to the test page, whatever the format. Value should be the whole declaration, eg. <?xml version="1.0" encoding="utf-8" ?>.
  9. $addutf8bom: Adds a UTF-8 BOM to the beginning of the file. Value takes any string, but is usually yes.
  10. $notes: Use these to explain how the test works, or any other useful information. Include block level markup for your notes text. You can make multiple paragraphs, lists, etc. if you want by adding the necessary markup.
  11. $style: Style information that will be placed inside a style element on the page. If styling is used as part of the test, this is where it goes.
  12. $stylesheet: The path to a stylesheet that is part of the test.
  13. $stylesheet2: The path to a second stylesheet that is part of the test, where needed.
  14. $linkrelstyle: A whole link element containing a style sheet reference. Useful when you want to add a charset attribute.
  15. $windowtitle: Changes the title element to a specified string, eg. 'مصر ישראל Deutschland France'.
  16. $bodyattrs: Adds the specified attributes to the body element tag, eg. dir="rtl".