Implementation Report - Internationalization Tag Set (ITS) Version 1.0

0. Summary: CR Exit Criteria

The CR exit criteria proposed were:

• two implementations of Conformance Type 1: ITS Markup Declarations. These are provided in the section ITS Applied to Existing Formats of the latest public Working Draft of Best Practices for XML Internationalization. During the CR phase, the Working Group has kept these markup schemes updated to be in sync with ITS markup declarations. In response to the comment Please use XHTML Modularization for defining ITS DTD and Schema, the Working Group has defined an ITS module for XHTML.
• two implementations of the Conformance Type 2: The Processing Expectations for ITS Markup. The test results described within the Internationalization Tag Set Version 1.0 Test Suite demonstrate that implementation 1) and implementation 3) fullfill these criteria.

1. Conformance Type 1: ITS Markup Declarations

Implementations of the Conformance Type 1: ITS Markup Declarations are markup schemes containing ITS markup declarations.

Three such markup schemes are described in section ITS Applied to Existing Formats of the latest public Working Draft of Best Practices for XML Internationalization: ITS and TEI, ITS and XHTML 1.0, and ITS and XML Spec. In response to comment Please use XHTML Modularization for defining ITS DTD and Schema, the Working Group has defined an ITS module for XHTML, using the XHTML modularization framework.

The three markup schemes are available as a tarball. The following table demonstrates how these markup schemes conform to Conformance Type 1.

Detailed information is below:

• ITS and TEI: ITS rules is allowed to appear in the TEI metadata section (the teiHeader). The local ITS attributes are added to the global attribute set for all elements. ITS span is added to the content pattern model.common (most inline contexts).
• ITS and XHTML 1.0: ITS rules is allowed to appear in the XHTML head element (the group HeadOpts.mix has been redefined accordingly). The local ITS attributes are added to the global attribute set for all elements (the group Common.attrib has been redefined accordingly). Ruby is not used since the ruby specification already defines an XHTML module for ruby.
• ITS and XML Spec: ITS rules is allowed to appear in the XML Spec header element (rules has been added as the last element to the XML Spec entity header.mdl). The local ITS attributes are added to the global attribute set for all elements (they have been added to the entity common.att). ITS ruby is allowed to appear as an inline element (it has been added to the entity p.pcd.mix).

2. Conformance Type 2: The Processing Expectations for ITS Markup

2.1. Implementations

Implementations of the Conformance Type 2: The Processing Expectations for ITS Markup are applications that need to process for internationalization or localization the nodes captured by an ITS data category.

The ITS Working Group is aware of the following implementations for this conformance type:

1. Sebastian Rahtz (invited Expert from University of Oxford) has created Spritser, an implementation of all ITS data categories, both global and local selection mechanisms, as well as defaults for data categories and precedence between selections.

2. The Okapi Framework open-source project has implemented four ITS data categories: Translate, Localization Note, Terminology and Elements Within Text. All implementations process both global and local selection mechanisms, as well as defaults for data categories and precedence between selections. The CVS repository for the source code and the Rainbow and XML filter download (which apply this implementation) are freely available.

   Applying the Okapi XML filter, Enlaso has implemented a simple Web interface to test ITS. It implements the following data categories: Translate, Localization Note, Terminology, and Elements Within Text (both globally and locally).

3. Felix Saskai (W3C) has created ITSImpl, an implementation of all ITS data categories, both global and local selection mechanisms, as well as defaults for data categories and precedence between selections.
4. xml-intl has implemented the ITS data categories Translate and Elements Within Text. The implementations process global selection mechanisms.

All implementations are able to process embedded and linked rules.

The implementations 1), 2) and 3) are open source implementations. 4) is a comercial implementation. The implementations 1), 2) and 3) are also able to produce the debug format for comparison with other tools, which is applied within the test suite (see below).

2.2. Test Suite

The Internationalization Tag Set (ITS) Version 1.0 testsuite (available as a tarball) is composed of tests which apply a common debug format (example). The format lists all element and attribute nodes in an XML document after ITS processing, and the ITS information which is available for them in the <o:output> element. For each ITS data category, there is:

1. a set of input documents (example)
2. for each input document a result document in the debug format. These result documents have been used to test during the CR phase whether an implementation is conform to conformance type 2 (example)

The following features have been tested for all data categories:

• global selection mechanisms (if applicable), with embedded and linked rules
• local selection mechanisms (if applicable), in elements of the target document or within <its:span>.
• defaults for data categories
• precedence between selections

For the language information data category, two tests have been developed (Language1.xml, Language2.xml) and processed (Language1-result.xml, Language2-result.xml) by implementation 1 and implementation 3 to demonstrate output of ITS processing which conforms to RFC 4646.

An XSLT script is used for the comparison of output. It relies on the output format described below. The format uses the namespace http://www.w3.org/2005/11/its/testoutputformat. An XML Schema document describing the ITS test suite outputformat is available.

• For the Translate data category:

  • If the node is translatable:

    <o:output o:translate="yes" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

  • If the node is not translatable:

    <o:output o:translate="no" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

• For the Localization Note data category:

  • If there is no localization information associated with the node:

    <o:output xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

  • If there is a locNote text associated with the node (locNoteType is required and can have the values description or alert):

    <o:output o:locNoteType="description" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat">
     <o:locNoteText>The inner text of the node  (can be
    empty, might also contain markup)</o:locNoteText>
    </output>

  • If there is a locNote reference associated with the node (locNoteType is required and can have the values description or alert):

    <o:output o:locNoteType="description" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat">
     <o:locNoteReference>The reference string (can be
    empty, might also contain markup)</o:locNoteReference>
    </output>

• For the Terminology data category:

  • If the node is not a term:

    <o:output o:term="no" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

  • If the node is a term and the termInfo is text:

    <o:output o:term="yes" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat">
     <o:termInfoText>The inner  text of the node (can be empty, might also contain
    markup)</o:termInfoText>
    </output>

  • If the node is a term and the termInfo is a reference:

    <o:output o:term="yes" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat">
     <o:termInfoReference>The reference string (can be
    empty, might also contain markup)</o:termInfoReference>
    </output>

• For the Directionality data category:

  • If there is explicit directionality information:

    <o:output o:dir="DIRVAL" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

    (DIRVAL is ltr, rtl, lro or rlo)

  • If there is no explicit directionality information:

    <o:output o:dir="ltr" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

    (The default is ltr)

• For the Ruby data category:

  • If there is no explicit ruby information:

    <o:output xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

  • If there is ruby information available:

    <o:output xmlns:o="http://www.w3.org/2005/11/its/testoutputformat">
     <o:ruby>
      <o:rb>(can be
    empty, might also contain markup)</o:rb>
      <o:rp>(can be
    empty or ommited, might also contain markup)</o:rp>
      <o:rt>(can be
    empty, might also contain markup)</o:rt>
     </o:ruby>
    </o:output>

    Note: the example above demonstrates only the output of local ruby markup processing. For output of global rules, please refer to the ruby related tests in the Internationalization Tag Set (ITS) Version 1.0 testsuite.

• For the Language Information data category:

  • If there is no explicit language information:

    <o:output xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

  • If there is explicit language information:

    <o:output xmlns="xmlns:o="http://www.w3.org/2005/11/its/testoutputformat">
     <o:langInfo>The language information (can be
    empty, might also contain markup)</o:langInfo>
    </output>

• For the Element Within Text data category:

  • If the node is within text:

    <o:output withinText="yes" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

  • If the node is nested:

    <o:output withinText="nested" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>

  • If the node is neither within text nor nested:

    <o:output withinText="no" xmlns:o="http://www.w3.org/2005/11/its/testoutputformat"/>