Internationalization Tag Set (ITS) Version 2.0

1.1.1 Relation to ITS 1.0

ITS 2.0 has the following relations to ITS 1.0:

• It adopts and maintains the following principles from ITS 1.0:

  • It adopts the use of data categories to define discrete units of functionality

  • It adopts the separation of data category definition from the mapping of the data category to a given content format

  • It adopts the conformance principle of ITS1.0 that an implementation only needs to implement one data category to claim conformance to ITS 2.0

• ITS 2.0 supports all ITS 1.0 data category definitions and adds new definitions, with the exceptions of Directionality and Ruby.

• ITS 2.0 adds a number of new data categories not found in ITS 1.0.

• While ITS 1.0 addressed only XML, ITS 2.0 specifies implementations of data categories in both XML and HTML.

1.1.2 New Principles

ITS 2.0 also adds the following principles and features not found in ITS 1.0:

• ITS 2.0 data categories are intended to be format neutral, with support for XML, HTML, and NIF: a data category implementation only needs to support a single content format mapping in order to support a claim of ITS 2.0 conformance.

• ITS 2.0 provides algorithms to generate NIF out of HTML or XML with ITS 2.0 metadata.

• A global implementation of ITS 2.0 requires at least the XPath version 1.0. Other versions of XPath or other query languages (e.g., CSS selectors) can be expressed via a dedicated queryLanguage attribute.

The new data categories included in ITS 2.0 are:

• Domain

• Disambiguation

• Locale Filter

• Provenance

• External Resource

• Target Pointer

• Id Value

• Preserve Space

• Localization Quality Issue

• Localization Quality Rating

• MT Confidence

• Allowed Characters

• Storage Size

1.2.1 Typical Problems

The following examples sketch one of the issues that currently hinder efficient XML-related localization: the lack of a standard, declarative mechanism that identifies which parts of an XML document need to be translated. Tools often cannot automatically perform this identification.

<resources>
  <section id="Homepage">
    <arguments>
      <string>page</string>
      <string>childlist</string>
    </arguments>
    <variables>
      <string>POLICY</string>
      <string>Corporate Policy</string>
    </variables>
    <keyvalue_pairs>
      <string>Page</string>
      <string>ABC Corporation - Policy Repository</string>
      <string>Footer_Last</string>
      <string>Pages</string>
      <string>bgColor</string>
      <string>NavajoWhite</string>
      <string>title</string>
      <string>List of Available Policies</string>
    </keyvalue_pairs>
  </section>
</resources>

<dialogue xml:lang="en-gb">
  <rsrc id="123">
    <component id="456" type="image">
      <data type="text">images/cancel.gif</data>
      <data type="coordinates">12,20,50,14</data>
    </component>
    <component id="789" type="caption">
      <data type="text">Cancel</data>
      <data type="coordinates">12,34,50,14</data>
    </component>
    <component id="792" type="string">
      <data type="text">Number of files: </data>
    </component>
  </rsrc>
</dialogue>

1.3.1 Potential Users of ITS

The ITS specification aims to provide different types of users with information about what markup should be supported to enable worldwide use and effective internationalization and localization of content. The following paragraphs sketch these different types of users, and their usage of ITS. In order to support all of these users, the information about what markup should be supported to enable worldwide use and effective localization of content is provided in this specification in two ways:

• abstractly in the data category descriptions: Section 8: Description of Data Categories

• concretely in the ITS schemas: Appendix D: Schemas for ITS

1.3.2 Ways to Use ITS

The ITS specification proposes several mechanisms for supporting worldwide use and effective internationalization and localization of content. We will sketch them below by looking at them from the perspectives of certain user types. For the purpose of illustration, we will demonstrate how ITS can indicate that certain parts of content should or should not be translated.

• A content author uses an attribute on a particular element to say that the text in the element should not be translated.

<help xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <head>
    <title>Building the Zebulon Toolkit</title>
  </head>
  <body>
    <p>To re-compile all the modules of the Zebulon toolkit you need to go in the <path
        its:translate="no">\Zebulon\Current Source\binary</path> directory. Then from there, run
      batch file <cmd its:translate="no">Build.bat</cmd>.</p>
  </body>
</help>

• A content author or information architect uses markup at the top of the document to identify a particular type of element or context in which the content should not be translated.

<help xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <head>
    <title>Building the Zebulon Toolkit</title>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
      <its:translateRule selector="//path | //cmd" translate="no"/>
    </its:rules>
  </head>
  <body>
    <p>To re-compile all the modules of the Zebulon toolkit you need to go in the
        <path>\Zebulon\Current Source\binary</path> directory. Then from there, run batch file
        <cmd>Build.bat</cmd>.</p>
  </body>
</help>

• A processor may insert markup at the top of the document which links to ITS information outside of the document.

<help xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <head>
    <title>Building the Zebulon Toolkit</title>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0"
      xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="EX-ways-to-use-its-4.xml"/>
  </head>
  <body>
    <p>To re-compile all the modules of the Zebulon toolkit you need to go in the
        <path>\Zebulon\Current Source\binary</path> directory. Then from there, run batch file
        <cmd>Build.bat</cmd>.</p>
  </body>
</help>

<its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
  <its:translateRule selector="//path | //cmd" translate="no"/>
</its:rules>

• A schema developer integrates ITS markup declarations in his schema to allow users to indicate that specific parts of the content should not be translated.

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:its="http://www.w3.org/2005/11/its"
  elementFormDefault="qualified">
  <xs:import namespace="http://www.w3.org/2005/11/its" schemaLocation="its.xsd"/>
  <xs:attributeGroup name="commonAtts">
    <xs:attributeGroup ref="its:att.local.with-ns.attribute.translate"/>
    <xs:attribute name="id" type="xs:ID" use="optional"/>
  </xs:attributeGroup>
  <xs:element name="help">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="head">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="title" type="xs:string"/>
            </xs:sequence>
            <xs:attributeGroup ref="commonAtts"/>
          </xs:complexType>
        </xs:element>
        <xs:element name="body">
          <xs:complexType>
            <xs:choice minOccurs="1" maxOccurs="unbounded">
              <xs:element name="p">
                <xs:complexType mixed="true">
                  <xs:choice minOccurs="0" maxOccurs="unbounded">
                    <xs:element ref="path"/>
                    <xs:element ref="cmd"/>
                  </xs:choice>
                  <xs:attributeGroup ref="commonAtts"/>
                </xs:complexType>
              </xs:element>
            </xs:choice>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
      <xs:attributeGroup ref="its:att.version.attribute.version"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="path">
    <xs:complexType mixed="true">
      <xs:attributeGroup ref="commonAtts"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="cmd">
    <xs:complexType mixed="true">
      <xs:attributeGroup ref="commonAtts"/>
    </xs:complexType>
  </xs:element>
</xs:schema>

The first two approaches above can be likened to the use of CSS in [XHTML 1.0]. Using a style attribute, an XHTML content author may assign a color to a particular paragraph. That author could also have used the style element at the top of the page to say that all paragraphs of a particular class or in a particular context would be colored red.

1.4.1 Support for legacy HTML content

ITS 2.0 does not define how to use ITS in HTML versions prior version 5. Users are encouraged to migrate their content to HTML or XHTML. While it is possible to use its-* attributes introduced for HTML5 in older versions of HTML (such as 3.2 or 4.01) and pages using these attributes will work without any problems, its-* attributes will be marked as invalid in validators.

2.1.1 Local Approach

The document in Example 10 shows how a content author may use the ITS translate attribute to indicate that all content inside the author element should be protected from translation. Translation tools that are aware of the meaning of this attribute can then screen the relevant content from the translation process.

<article xmlns="http://docbook.org/ns /docbook"
         xmlns:its="http://www.w3.org/2005/11/its"
         its:version="2.0" version="5.0" xml:lang="en">
  <info>
    <title>An example article</title>
    <author its:translate="no">
      <personname>
        <firstname>John</firstname>
        <surname>Doe</surname>
      </personname>
      <affiliation>
        <address><email>foo@example.com</email></address>
      </affiliation>
    </author>
  </info>
  <para>This is a short article.</para>
</article>

For this example to work, the schema developer will need to add the translate attribute to the schema as a common attribute or on all the relevant element definitions. Note how there is an expectation in this case that inheritance plays a part in identifying which content does have to be translated and which does not. Tools that process this content for translation will need to implement the expected inheritance.

2.1.2 Global Approach

The document in Example 11 shows a different approach to identifying non-translatable content, similar to that used with a style element in [XHTML 1.0], but using an ITS-defined element called rules. It works as follows: A document can contain a rules element (placed where it does not impact the structure of the document, e.g., in a “head” section). It contains one or more ITS rule elements (for example translateRule). Each of these specific elements contains a selector attribute. As its name suggests, this attribute selects the node or nodes to which a corresponding ITS information pertains. The values of ITS selector attributes are XPath absolute location paths (or CSS selectors if queryLanguage is set to "css"). Information for the handling of namespaces in these path expressions is taken from namespace declarations [XML Names] at the current rule element.

<myTopic xmlns="http://mynsuri.example.com" id="topic01" xml:lang="en-us">
  <prolog>
    <title>Using ITS</title>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
      <its:translateRule selector="//n:term" translate="no" xmlns:n="http://mynsuri.example.com"/>
    </its:rules>
  </prolog>
  <body>
    <p>ITS defines <term>data category</term> as an abstract concept for a particular type of
      information for internationalization and localization of XML schemas and documents.</p>
  </body>
</myTopic>

For this approach to work, the schema developer needs to add the rules element and associated markup to the schema. In some cases global rules may be sufficient to allow the schema developer to avoid adding other ITS markup (such as an translate attribute) to the elements and attributes in the schema. However, it is likely that authors will want to use attributes on markup from time to time to override the general rule.

For specification of the Translate data category information, the contents of the rules element would normally be designed by an information architect familiar with the document format and familiar with, or working with someone familiar with, the needs of the localization group.

The global, rule-based approach has the following benefits:

• Content authors do not have to concern themselves with creating additional markup or verifying that the markup was applied correctly. ITS data categories are associated with sets of nodes (for example all p elements in an XML instance)

• Changes can be made in a single location, rather than by searching and modifying local markup throughout a document (or documents, if the rules element is stored as an external entity)

• ITS data categories can designate attribute values as well as elements.

• It is possible to associate ITS markup with existing markup (for example the term element in DITA)

The commonality in both examples above is the markup translate='no'. This piece of ITS markup can be interpreted as follows:

• it pertains to the Translate data category

• the attribute translate holds a value of "no"

The ITS selector attribute allows:

• ITS data category attributes to appear in global rules (even outside of an XML document or schema)

• ITS data categories attributes to pertain to sets of XML nodes (for example all p elements in an XML document)

• ITS markup to pertain to attributes

• ITS markup to associate with existing markup (for example the term element in DITA)

5.2.1 Global, Rule-based Selection

Global, rule-based selection is implemented using the rules element. It contains zero or more rule elements. Each rule element has a mandatory selector attribute. This attribute and all other possible attributes on rule elements are in the empty namespace and used without a prefix.

If there is more than one rules element in an XML document, the rules from each section are to be processed at the same precedence level. The rules sections are to be read in document order, and the ITS rules with them processed sequentially. The versions of these rules elements MUST NOT be different.

Depending on the data category and its usage, there are additional attributes for adding information to the selected nodes, or for pointing to existing information in the document. For example, the Localization Note data category can be used for adding notes to selected nodes, or for pointing to existing notes in the document. For the former purpose, a locNote element can be used. For the latter purpose, a locNotePointer attribute can be used.

Each data category allows users to add information to the selected nodes except for language information. Pointing to existing information is not possible for data categories that express a closed set of values, that is: Translate, Directionality, Locale Filter, and Elements Within Text.

The functionalities of adding information and pointing to existing information are mutually exclusive. That is: markup for pointing and adding MUST NOT appear in the same rule element.

Global rules can appear in the XML document they will be applied to, or in a separate XML document. The precedence of their processing depends on these variations. See also Section 5.5: Precedence between Selections.

5.2.2 Local Selection in an XML Document

Local selection in XML documents is realized with ITS local attributes or the span element. span serves just as a carrier for the local ITS attributes.

The data category determines what is being selected. The necessary data category specific defaults are described in Section 8.1: Position, Defaults, Inheritance and Overriding of Data Categories.

<text xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0" xml:lang="en">
  <head its:translate="no">
    <author>Sven Corneliusson</author>
    <date>2006-09-26T17:34:04Z</date>
    <title its:translate="yes" role="header">Bidirectional Text</title>
  </head>
  <body>
    <par>In Arabic, the title <quote xml:lang="ar" its:dir="rtl">نشاط التدويل، W3C</quote> means
        <quote>Internationalization Activity, W3C</quote>.</par>
  </body>
</text>

5.3.1 Choosing Query Language

Rule elements have attributes which contain absolute and relative selectors. Interpretation of these selectors depends on the actual query language. The query language is set by queryLanguage attribute on rules element. If queryLanguge is not specified XPath 1.0 is used as a default query language.

5.3.2 XPath 1.0

XPath 1.0 is identified by xpath value in queryLanguage attribute.

<!-- Definitions for TEI -->
<its:rules version="2.0"  xmlns:its="http://www.w3.org/2005/11/its">
 <its:termRule selector="//tei:term" term="yes" xmlns:tei="http://www.tei-c.org/ns/1.0"/>
</its:rules>

<!-- Definitions for DocBook -->
<its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
  <its:termRule selector="//term" term="yes"/>
</its:rules>

5.3.3 CSS Selectors

CSS Selectors are identified by css value in queryLanguage attribute.

5.3.4 Additional query languages

ITS processors MAY support additional query languages. For each additional query language processor MUST define:

• identifier of query language used in queryLanguage;

• rules for evaluating absolute selector to collection of nodes;

• rules for evaluating relative selector to collection of nodes.

Future versions of this specification MAY define additional query languages. The following query language identifiers are reserved: xpath, css, xpath2, xpath3, xquery, xquery3, xslt2, xslt3.

5.3.5 Variables in selectors

A param element (or several ones) can be placed as the first child element(s) of the rules element to define the default values of variables used in the various selectors used in the rules.

Implementation MUST support the param element for all query languages it supports and which at the same time define how variables are bind for evaluation of selector expression. Implementations SHOULD also provide means for changing the default values of the param elements. Such means are implementation-specific.

The param element has a required name attribute. The value of the name attribute is a QName, see [XML Names]. The content of the element is a string used as default value for the corresponding variable.

<doc its:version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
  <its:rules version="2.0">
    <its:param name="LCID">0x0409</its:param>
    <its:translateRule selector="/doc" translate="no"/>
    <its:translateRule selector="//msg[@lcid=$LCID]" translate="yes"/>
  </its:rules>
  <msg lcid="0x0409" num="1">Create a folder</msg>
  <msg lcid="0x0411" num="1">フォルダーを作成する</msg>
  <msg lcid="0x0407" num="1">Erstellen Sie einen Ordner</msg>
  <msg lcid="0x040c" num="1">Créer un dossier</msg>
</doc>

8.2.1 Definition

The Translate data category expresses information about whether the content of an element or attribute should be translated or not. The values of this data category are "yes" (translatable) or "no" (not translatable).

8.2.2 Implementation

The Translate data category can be expressed with global rules, or locally on an individual element. For elements, the data category information inherits to the textual content of the element, including child elements, but excluding attributes. The default is that elements are translatable and attributes are not.

GLOBAL: The translateRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required translate attribute with the value "yes" or "no".

<its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
  <its:translateRule translate="no" selector="//code"/>
</its:rules>

LOCAL: The following local markup is available for the Translate data category:

• A translate attribute with the value "yes" or "no".

<messages its:version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
  <msg num="123">Click Resume Button on Status Display or <panelmsg its:translate="no"
      >CONTINUE</panelmsg> Button on printer panel</msg>
</messages>

<!DOCTYPE html>
<html>
  <head>
    <meta charset=utf-8>
    <title>Translate flag test: Default</title>
  </head>
  <body>
    <p>The <span translate=no>World Wide Web Consortium</span> is
      making the World Wide Web worldwide!</p>
  </body>
</html>

8.3.1 Definition

The Localization Note data category is used to communicate notes to localizers about a particular item of content.

This data category can be used for several purposes, including, but not limited to:

• Tell the translator how to translate parts of the content

• Expand on the meaning or contextual usage of a specific element, such as what a variable refers to or how a string will be used in the user interface

• Clarify ambiguity and show relationships between items sufficiently to allow correct translation (e.g., in many languages it is impossible to translate the word"enabled" in isolation without knowing the gender, number and case of the thing it refers to.)

• Indicate why a piece of text is emphasized (important, sarcastic, etc.)

Two types of informative notes are needed:

• An alert contains information that the translator must read before translating a piece of text. Example: an instruction to the translator to leave parts of the text in the source language.

• A description provides useful background information that the translator will refer to only if they wish. Example: a clarification of ambiguity in the source text.

Editing tools may offer an easy way to create this type of information. Translation tools can be made to recognize the difference between these two types of localization notes, and present the information to translators in different ways.

8.3.2 Implementation

The Localization Note data category can be expressed with global rules, or locally on an individual element. For elements, the data category information inherits to the textual content of the element, including child elements, but excluding attributes.

GLOBAL: The locNoteRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required locNoteType attribute with the value "description" or "alert".

• Exactly one of the following:

  • A locNote element that contains the note itself and allows for local ITS markup.

  • A locNotePointer attribute that contains a relative selector pointing to a node that holds the localization note.

  • A locNoteRef attribute that contains an IRI referring to the location of the localization note.

  • A locNoteRefPointer attribute that contains a relative selector pointing to a node that holds the IRI referring to the location of the localization note.

<myRes>
  <head>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0" its:translate="no">
      <its:locNoteRule locNoteType="alert" selector="//msg[@id='DisableInfo']">
        <its:locNote>The variable {0} has three possible values: 'printer', 'stacker' and 'stapler
          options'.</its:locNote>
      </its:locNoteRule>
    </its:rules>
  </head>
  <body>
    <msg id="DisableInfo">The {0} has been disabled.</msg>
  </body>
</myRes>

<Res>
  <prolog>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
      <its:translateRule selector="//msg/notes" translate="no"/>
      <its:locNoteRule locNoteType="description" selector="//msg/data" locNotePointer="../notes"/>
    </its:rules>
  </prolog>
  <body>
    <msg id="FileNotFound">
      <notes>Indicates that the resource file {0} could not be loaded.</notes>
      <data>Cannot find the file {0}.</data>
    </msg>
    <msg id="DivByZero">
      <notes>A division by 0 was going to be computed.</notes>
      <data>Invalid parameter.</data>
    </msg>
  </body>
</Res>

<myRes>
  <head>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
      <its:locNoteRule locNoteType="description" selector="//msg[@id='NotFound']"
                       locNoteRef="ErrorsInfo.html#NotFound"/>
    </its:rules>
  </head>
  <body>
    <msg id="NotFound">Cannot find {0} on {1}.</msg>
  </body>
</myRes>

<dataFile>
  <prolog>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
      <its:locNoteRule locNoteType="description" selector="//data"
                       locNoteRefPointer="../@noteFile"/>
    </its:rules>
  </prolog>
  <body>
    <string id="FileNotFound" noteFile="Comments.html#FileNotFound">
      <data>Cannot find the file {0}.</data>
    </string>
    <string id="DivByZero" noteFile="Comments.html#DivByZero">
      <data>Invalid parameter.</data>
    </string>
  </body>
</dataFile>

LOCAL: The following local markup is available for the Localization Note data category:

• One of the following:

  • A locNote attribute that contains the note itself.

  • A locNoteRef attribute that contains an IRI referring to the location of the localization note.

• An optional locNoteType attribute with the value"description" or "alert". If the locNoteType attribute is not present, the type of localization note will be assumed to be "description".

<msgList xmlns:its="http://www.w3.org/2005/11/its" xml:space="preserve" its:version="2.0">
 <data name="LISTFILTERS_VARIANT" its:locNote="Keep the leading space!" its:locNoteType="alert">
  <value> Variant {0} = {1} ({2})</value>
 </data>
 <data its:locNote="%1\$s is the original text's date in the format YYYY-MM-DD HH:MM always in GMT">
  <value>Translated from English content dated <span id="version-info">%1\$s</span> GMT.</value>
 </data>
</msgList>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>LocNote test: Default</title>
  </head>
  <body>
    <p>This is a <span its-loc-note="Check with terminology engineer" its-loc-note-type=alert>motherboard</span>.</p>
  </body>
</html>

8.4.1 Definition

The Terminology data category is used to mark terms and optionally associate them with information, such as definitions. This helps to increase consistency across different parts of the documentation. It is also helpful for translation.

8.4.2 Implementation

The Terminology data category can be expressed with global rules, or locally on an individual element. There is no inheritance. The default is that neither elements nor attributes are terms.

GLOBAL: The termRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required term attribute with the value "yes" or "no".

• None or exactly one of the following:

  • A termInfoPointer attribute that contains a relative selector pointing to a node that holds the terminology information.

  • A termInfoRef attribute that contains an IRI referring to the resource providing information about the term.

  • A termInfoRefPointer attribute that contains a relative selector pointing to a node that holds the IRI referring to the location of the terminology information.

<text>
  <its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
    <its:termRule selector="//term" term="yes" termInfoPointer="id(@def)"/>
  </its:rules>
  <p>We may define <term def="TDPV">discoursal point of view</term> as 
    <gloss xml:id="TDPV">the relationship, expressed through discourse structure, between the
    implied author or some other addresser, and the fiction.</gloss></p>
</text>

<text>
  <its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
    <its:termRule selector="//term[1]" term="yes"
                  termInfoRef="#TDPV"/>
  </its:rules>
  <p>We may define <term>discoursal point of view</term>
as <gloss xml:id="TDPV">the relationship, expressed through discourse
structure, between the implied author or some other addresser,
and the fiction.</gloss></p>
</text>

<text>
  <its:rules version="2.0"  xmlns:its="http://www.w3.org/2005/11/its">
    <its:termRule selector="//term" term="yes"
                   termInfoRefPointer="@target"/>
  </its:rules>
  <p>We may define <term target="#TDPV">discoursal point of view</term>
as <gloss xml:id="TDPV">the relationship, expressed through discourse
structure, between the implied author or some other addresser,
and the fiction.</gloss></p>
</text>

LOCAL: The following local markup is available for the Terminology data category:

• A term attribute with the value "yes" or "no".

• An optional termInfoRef attribute that contains an IRI referring to the resource providing information about the term.

• An optional termConfidence attribute with the value of a rational number in the interval 0 to 1 (inclusive). The value follows the XML Schema decimal data type with the constraining facets minInclusive set to 0 and maxInclusive set to 1. termConfidence represents the confidence of the agents producing the annotation that the values of the term and, where provided, termInfoRef, are accurate. 1 represents the highest level of confidence.

Any node selected by the terminology data category with the termConfidence attribute specified MUST be contained in an element with the annotatorsRef (or in HTML its-annotators-ref) attribute specified for the Terminology data category. See Section 5.8: ITS Tools Annotation for more information.

<book its:version="2.0" xmlns:its="http://www.w3.org/2005/11/its" its:annotatorsRef="terminology|http://example.com/term-tool">
  <head>...</head>
  <body>
    ... 
    <p>And he said: you need a new 
      <quote its:term="yes" 
        its:termInfoRef="http://www.directron.com/motherboards1.html" its:termConfidence="0.5">motherboard</quote></p>
    ...
 </body> 
</book>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>Terminology test: default</title>
  </head>
  <body>
    <p>We need a new <span its-term=yes>motherboard</span>
    </p>
  </body>
</html>

8.5.1 Definition

The Directionality data category allows the user to specify the base writing direction of blocks, embeddings and overrides for the Unicode bidirectional algorithm. It has four values: "ltr", "rtl", "lro" and "rlo".

8.5.2 Implementation

The Directionality data category can be expressed with global rules, or locally on an individual element. For elements, the data category information inherits to the textual content of the element, including child elements and attributes. The default is that both elements and attributes have the directionality of left-to-right.

GLOBAL: The dirRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required dir attribute with the value "ltr", "rtl", "lro" or "rlo".

<text xml:lang="en">
  <body>
    <par>In Hebrew, the title <quote xml:lang="he" direction="rtlText">פעילות
        הבינאום, W3C</quote> means <quote>Internationalization Activity,
        W3C</quote>.</par>
  </body>
</text>

<its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
  <its:dirRule dir="rtl" selector="//*[@direction='rtlText']"/>
</its:rules>

LOCAL: The following local markup is available for the Directionality data category:

• A dir attribute with the value "ltr", "rtl", "lro" or "rlo".

<text xml:lang="en" xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <body>
    <par>In Arabic, the title <quote xml:lang="ar" its:dir="rtl">نشاط التدويل،
        W3C</quote> means <quote>Internationalization Activity,
      W3C</quote>.</par>
  </body>
</text>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>Dir test: Default</title>
  </head>
  <body>
    <p>In Arabic, the title <quote dir=rtl lang=ar>نشاط التدويل، W3C</quote>
      means <quote>Internationalization Activity, W3C</quote>.</p>
  </body>
</html>

8.6.1 Definition

The Ruby data category is used for a run of text that is associated with another run of text, referred to as the base text. Ruby text is used to provide a short annotation of the associated base text. It is most often used to provide a reading (pronunciation) guide.

8.6.2 Implementation

The Ruby data category can be expressed with global rules, or locally. There is no inheritance.

GLOBAL: The rubyRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies. This is the ruby base text.

• An optional rubyPointer attribute that contains a relative selector pointing to a node that corresponds to the ruby element.

• An optional rpPointer attribute that contains a relative selector pointing to a node that corresponds to the ruby parenthesis.

• An optional rubyText element that contains the ruby text.

• An optional rtPointer attribute that contains a relative selector pointing to a node that corresponds to the ruby text.

<text xmlns:its="http://www.w3.org/2005/11/its">
  <head>
    ... 
    <its:rules version="2.0">
      <its:rubyRule selector="/text/body/img[1]/@alt">
        <its:rubyText>World Wide Web Consortium</its:rubyText>
      </its:rubyRule>
    </its:rules>
  </head>
  <body>
    <img src="w3c_home.png" alt="W3C"/>
    ...
  </body>
</text>

LOCAL: In a document, the Ruby data category is realized with a ruby element. It contains the following:

• The ruby base text or span element that contains the ruby base text and allows for local ITS markup.

• An rp element that contains the ruby parenthesis. It is used in case of simple markup to specify characters that can denote the beginning and end of ruby text when user agents do not have other ways to present ruby text distinctively from the base text.

• An rt element that contains the ruby text and allows for local ITS markup.

All these elements share the attributes of the span element.

<text its:version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
  <head> ... </head>
  <body>
  <p>この本は 
    <its:ruby>
      慶応義塾大学
      <its:rp>(</its:rp><its:rt>けいおうぎじゅくだいがく</its:rt><its:rp>)</its:rp>
    </its:ruby>の歴史を説明するものです。</p>
 </body>
</text>

8.7.1 Definition

The element langRule is used to express the language of a given piece of content. The langPointer attribute points to the markup which expresses the language of the text selected by the selector attribute. This markup MUST use values that conform to [BCP47]. The recommended way to specify language identification is to use xml:lang in XML, and lang in HTML. The langRule element is intended only as a fall-back mechanism for documents where language is identified with another construct.

<its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
  <its:langRule selector="//p" langPointer="@mylangattribute"/>
</its:rules>

8.7.2 Implementation

The Language Information data category can be expressed only with global rules. For elements, the data category information inherits to the textual content of the element, including child elements and attributes. There is no default.

GLOBAL: The langRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required langPointer attribute that contains a relative selector pointing to a node that contains language information.

8.8.1 Definition

The Elements Within Text data category reveals if and how an element affects the way text content behaves from a linguistic viewpoint. This information is for example relevant to provide basic text segmentation hints for tools such as translation memory systems. The values associated with this data category are:

• "yes" : The element and its content are part of the flow of its parent element. For example the element strong in [XHTML 1.0]:

  <strong>Appaloosa horses</strong> have spotted coats.

• "nested" : The element is part of the flow of its parent element, its content is an independent flow. For example the element fn in [DITA 1.0]:

  Palouse horses<fn>A Palouse horse is the same as an Appaloosa.</fn> have spotted coats.

• "no" : The element splits the text flow of its parent element and its content is an independent text flow. For example the element p when inside the element li in DITA or XHTML:

  <li>Palouse horses: <p>They have spotted coats.</p> <p>They have been bred by the Nez Perce.</p> </li>

8.8.2 Implementation

The Elements Within Text data category can be expressed with global rules, or locally on an individual element. There is no inheritance. The default is that elements are not within text.

GLOBAL: The withinTextRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required withinText attribute with the value "yes", "no" or "nested".

<its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
  <its:withinTextRule withinText="yes" selector="//b | //em | //i"/>
</its:rules>

LOCAL: The following local markup is available for the Elements Within Text data category:

• A withinText attribute with the values "yes", "no" or "nested".

<text xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <body>
    <par>Text with <bold its:withinText="yes">bold</bold>.</par>
  </body>
</text>

<!DOCTYPE html>
<html>
  <head>
    <meta charset=utf-8> 
    <title>Within text test: Default</title>
  </head>
  <body>
    <p>Text with <span its-within-text='yes'>bold</span>.</p>
  </body>
</html>

8.9.1 Definition

The Domain data category is used to identify the topic or subject of a given content. Such information allows to make more relevant lingusitic choices during various processes.

Examples of usage include:

• Allowing machine translation systems to select the most appropriate engine and rules to translate the content.

• Providing a general indication of what terminology collection should be used by a translator.

This data category addresses various challenges:

• Often domain-related information already exist in the document (e.g. keywords in the HTML meta element). The Domain data category provides a mechanism to point to this information.

• There are many flat or structured lists of domain related values, keywords, key phrases, classification codes, ontologies, etc. The Domain data category does not propose its own given list. Instead it provides a mapping mechanism to associate the values in the document with the values used by the consumer tool.

8.9.2 Implementation

The Domain data category can be expressed only with global rules. For elements, the data category information inherits to the textual content of the element, including child elements and attributes. There is no default.

The information provided by this data category is a comma-separated list of one or more values which is obtained by applying the following algorithm:

• STEP 1: Set the initial value of the resulting string as an empty string.

• STEP 2: Get the list of nodes resulting of the evaluation of the domainPointer attribute.

• STEP 3: For each node:

  • STEP 3-1: If the node value contains a COMMA (U+002C):

    • STEP 3-1-1: Split the node value into separate strings using the COMMA (U+002C) as separator.

    • STEP 3-1-2: For each string:

      • STEP 3-1-2-1: Trim the leading and trailing white spaces of the string.

      • STEP 3-1-2-2: If the first character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

      • STEP 3-1-2-3: If the last character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

      • STEP 3-1-2-4: If the value is empty: Go to STEP 3-1-2.

      • STEP 3-1-2-5: Check if there is a mapping for the string (the mapping is case-insensitive):

        • STEP 3-1-2-5-1. If a mapping is found: Add the corresponding value to the result string.

        • STEP 3-1-2-5-2. Else (if no mapping is found): Add the string (in its original cases) to the result string.

  • STEP 3-2: Else (if the node value does not contain a COMMA (U+002C)):

    • STEP 3-2-1: Trim the leading and trailing white spaces of the string.

    • STEP 3-2-2: If the first character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

    • STEP 3-2-3: If the last character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

    • STEP 3-2-4: If the value is empty: Go to STEP 3.

    • STEP 3-2-5: Check if there is a mapping for the string (the mapping is case-insensitive):

      • STEP 3-2-5-1: If a mapping is found: Add the corresponding value to the result string.

      • STEP 3-2-5-2: Else (if no mapping is found): Add the string (in its original cases) to the result string.

• STEP 4: Remove duplicated values from the resulting string.

• STEP 5: Return the resulting string.

GLOBAL: The domainRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required domainPointer attribute that contains a relative selector pointing to a node that contains the domain information.

• An optional domainMapping attribute that contains a comma separated list of mappings between values in the content and consumer tool specific values. The left part of the pair corresponds to the source content and is unique within the mapping and case-insensitive. The right part of the mapping belongs to the consumer tool. Several left parts can map to a single right part. The values in the left or the right part of the mapping may contain spaces; in that case they MUST be delimited by quotation marks, that is pairs of APOSTROPHE (U+0027) or QUOTATION MARK (U+0022).

<its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0"
           xmlns:h="http://www.w3.org/1999/xhtml">
  <its:domainRule selector="/h:html/h:body"
                  domainPointer="/h:html/h:head/h:meta[@name='keywords']/@content"/>
</its:rules>

<its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0"
           xmlns:h="http://www.w3.org/1999/xhtml">
  <its:domainRule selector="/h:html/h:body"
    domainPointer="/h:html/h:head/h:meta[@name='dcterms.subject' or @name='keywords']/@content"
    domainMapping="automotive auto, medical medicine, 'criminal law' law, 'property law' law"/>
</its:rules>

8.10.1 Definition

The Disambiguation data category is used to highlight (mark up) specific conceptual patterns that may require special treatment when localizing and translating content.

This data category can be used for several purposes, including, but not limited to:

• Informing a translation service that a certain fragment of text is subject to follow specific translation rules, e.g. for proper names, or officially regulated translations, as well as to conveying a very specific meaning of the fragment.

• Informing content management systems and translation services about the intended conceptual type of a textual entity in order to enable processing based on this specific type for source and target languages, for example, when dealing with personal names, product names, or geographic names, chemical compounds, protein names, and so forth.

Disambiguation support is achieved by associating a marked up fragment of text with an external web resource that can be dereferenced by a language review agent, i.e. by accessing the intended meaning or lexical choice of the fragment, and thereby contributing to its correct translation.

A fragment of text is disambiguated at different granularities: (1) lexical type, (2) ontological concept, or (3) named entity.

In the case of lexical type, the external resource may provide appropriate synonyms and example usage, such as what WordNet services do.

In the case of ontological concept, the external resource may provide a formalized conceptual definition arranged in a hierarchical framework of related concepts.

In the case of a named entity, the external resource may provide a fully fledged description of the associated real world entity. For instance, the word 'City' in the fragment 'I am going to the City' may be disambiguated on the basis of one of WordNet's synsets that can be represented by 'city', an ontological concept of 'City' that could represent a subclass of 'Populated Place' at the conceptual granularity level, or the central area of a particular city, e.g. 'City of London', as interpreted at the entity granularity level.

Linked data networks, such as DBpedia, further increase the interlinking of ontological concepts and named entity definitions for same things and in different languages, thereby offering the possibility to directly facilitate translation through a source language description.

Two types of disambiguation are possible:

• Disambiguation for target type class, which explicitly describes the type class of the underlying concept or entity of the fragment.

• Disambiguation for target identity, which implicitly describes the intended meaning of the fragment through a link to an external resource.

Text analysis engines, such as named entity recognizers, named entity, concept and word sense disambiguation components do offer appropriate solutions to create the needed information. Content management systems are also able to present and visualize this information, or employ it to index their content. Machine translation services may use this information for optimizing their language and translation models.

8.10.2 Implementation

The Disambiguation data category can be expressed with global rules, or locally on an individual element. There is no inheritance.

When using disambiguation specifying the target identity, the user MUST use only one of the two addressing modes:

1. Using disambigSource and one of disambigIdent or disambigIdentPointer (at a global rule) to specify the collection and the identifier itself.

2. Using one of disambigIdentRef or disambigIdentRefPointer (at a global rule) using an IRI for the disambiguation target.

GLOBAL: The disambiguationRule element contains the following:

• A required selector attribute that contains an absolute selector which selects the nodes to which this rule applies.

• An optional disambigGranularity attribute that contains a string, specifying the granularity level of the disambiguation. The value MUST be one of the following identifiers: "lexical-concept", "ontology-concept", or "entity". The default value is "entity".

• At least one of the following:

  • To specify the target type class, exactly one of the following:

    • A disambigClassPointer attribute that contains a relative selector pointing to a node specifying the type of entity or concept class behind the selector.

    • A disambigClassRefPointer attribute that contains a relative selector pointing to a node that holds an IRI that specifies the type of entity or concept class behind the selector.

  • To specify the target identity, exactly one of the following:

    • When using the addressing mode 1:

      • A disambigSourcePointer attribute that contains a relative selector to a node that holds the string representing the disambiguation identifier collection source.

      • A disambigIdentPointer attribute that contains a relative selector to a node that holds the string, representing the disambiguation identifier for the disambiguation target that is valid within the specified disambiguation source.

    • When using the addressing mode 2:

      • A disambigIdentRefPointer attribute that contains a relative selector pointing to a node that holds an IRI that represents a unique identifier for the disambiguation target.

For an example, see Example 59.

LOCAL: The following local markup is available for the Disambiguation data category:

• An optional disambigConfidence attribute with the value of a rational number in the interval 0 to 1 (inclusive). The value follows the XML Schema decimal data type with the constraining facets minInclusive set to 0 and maxInclusive set to 1. disambigConfidence represents the confidence of the agents producing the annotation that the union of the values for the other disambiguation attributes in this instance are accurate. 1 represents the highest level of confidence.

• An optional disambigGranularity attribute that contains a string, specifying the granularity level of the disambiguation. The value MUST be one of the following identifiers: "lexical-concept", "ontology-concept", or "entity". The default value is "entity".

• At least one of the following:

  • To specify the target type class:

    • A disambigClassRef attribute that contains an IRI, specifying the type of entity or concept class behind the selector.

  • To specify the target identity, exactly one of the following:

    • When using the addressing mode 1:

      • A disambigSource attribute that contains a string representing the disambiguation identifier collection source.

      • A disambigIdent attribute that contains a string, representing the disambiguation identifier for the disambiguation target that is valid within the specified disambiguation source.

    • When using the addressing mode 2:

      • A disambigIdentRef attribute that contains an IRI that represents a unique identifier for the disambiguation target.

Any node selected by the disambiguation data category with the disambigConfidence attribute specified MUST be contained in an element with the annotatorsRef (or in HTML its-annotators-ref) attribute specified for the disambiguation data category. For more information, see Section 5.8: ITS Tools Annotation.

<!DOCTYPE html>
<html lang="en" its-annotators-ref="disambiguation|http://enrycher.ijs.si">
  <head>
    <meta charset="utf-8" />
    <title>Disambiguation: Local Test</title>
  </head>      
  <body>
    <p><span 
          its-disambig-confidence="0.7"
          its-disambig-class-ref="http://nerd.eurecom.fr/ontology#Place"  
          its-disambig-ident-ref="http://dbpedia.org/resource/Dublin" 
          its-disambig-granularity="entity">Dublin</span> 
      is the <span 
          its-disambig-source="Wordnet3.0" 
          its-disambig-ident="301467919" 
          its-disambig-granularity="lexical-concept"
          its-disambig-confidence="0.5"
          >capital</span> of Ireland.</p>
  </body>
</html>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
      <link href=EX-disambiguation-html5-rdfa.xml rel=its-rules>
    <title>Entity: Local Test</title>
  </head>
  <body>
    <p><span property="http://xmlns.com/foaf/0.1/name"
             about="http://dbpedia.org/resource/Dublin"
             typeof="http:/nerd.eurecom.fr/ontology#Place">Dublin</span> is 
      the capital of Ireland.</p>
  </body>
</html>

<its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
  <its:disambiguationRule selector="//*[@typeof and @about]" 
    disambigClassRefPointer="@typeof" disambigIdentRefPointer="@about" disambigGranularity="entity"/>
</its:rules>

8.11.1 Definition

The Locale Filter data category specifies that a node is only applicable to certain locales.

This data category can be used for several purposes, including, but not limited to:

• Include a legal notice only in locales for certain regions.

• Drop editorial notes from all localized output.

The Locale Filter data category associates with each selected node a list of extended language ranges conforming to [BCP47]. The list is comma-separated and can include the wildcard extended language range "*". The list can also be empty. Whitespace surrounding language ranges is ignored.

8.11.2 Implementation

The Locale Filter data category can be expressed with global rules, or locally on an individual element. For elements, the data category information inherits to the textual content of the element, including child elements and attributes. The default is that the language range is "*".

Implementations MUST NOT combine lists of language ranges from multiple rules or local attributes.

GLOBAL: The localeFilterRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required localeFilterList attribute with a comma-separated list of extended language ranges, or an empty string value.

<book xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <info>
    <its:rules version="2.0">
      <its:localeFilterRule 
        selector="//legalnotice[@role='Canada']"
        localeFilterList="en-CA, fr-CA"/>
    </its:rules>
    <legalnotice role="Canada">
      <para>This legal notice is only for English and French Canadian locales.</para>
    </legalnotice>
  </info>
</book>

<section xmlns:its="http://www.w3.org/2005/11/its">
  <info>
    <its:rules version="2.0">
      <its:localeFilterRule selector="//remark" localeFilterList=""/>
    </its:rules>
  </info>
  <remark>Note: This section will be written later.</remark>
</section>

LOCAL: The following local markup is available for the Locale Filter data category:

• A localeFilterList attribute with a comma-separated list of extended language ranges, or an empty string value.

<book xmlns:its="http://www.w3.org/2005/11/its">
  <info>
    <legalnotice its:localeFilterList="en-CA, fr-CA">
      <para>This legal notice is only for English and French Canadian locales.</para>
    </legalnotice>
  </info>
</book>

8.12.1 Definition

The Provenance data category is used to communicate the identity of agents that have been involved in the translation of the content or the revision of the translated content. This allows translation and translation revision consumers, such as post-editors, translation quality reviewers or localization workflow managers, to assess how the performance of these agents may impact the quality of the translation. Translation and translation revision agents can be identified as a person, a piece of software or an organization that has been involved in providing a translation that resulted in the selected content.

This data category offers three types of information. First, it allows to identify translation agents. Second, it allows to identify revision agents. Third, if provenance information is needed that includes temporal or sequence information about translation processes (e.g. multiple revision cycles) or requires agents that support a wider range of activities, the data category offers a mechanism to refer to external provenance information.

Translation or translation revision tools, such as machine translation engines or computer assisted translation tools, may offer an easy way to create this information. Translation tools can then present this information to post-editors or translation workflow managers. Web applications may to present such information to consumers of translated documents.

The data category defines seven pieces of information:

8.12.2 Implementation

The Provenance data category can be expressed with global rules, or locally on individual elements. For elements, the data category information inherits to the textual content of the element, including child elements and attributes.

GLOBAL: The provRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A provenanceRecordsRefPointer attribute that contains a relative selector pointing to a node containing a list of provenance records. These are related to the content selected via the selector attribute.

The global rule does not apply to HTML as local markup is provided for direct annotation in HTML.

<text xmlns:dc="http://purl.org/dc/elements/1.1/"
  xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <dc:creator>John Doe</dc:creator>
  <its:provenanceRecords xml:id="pr1">
    <its:provenanceRecord
      toolRef="http://www.onlinemtex.com/2012/7/25/wsdl/"
      org="acme-CAT-v2.3"
      revToolRef="http://www.mycat.com/v1.0/download"
      revOrg="acme-CAT-v2.3"
      provRef="http://www.examplelsp.com/excontent987/production/prov/e6354"/>
  </its:provenanceRecords>
  <its:provenanceRecords xml:id="pr2">
    <its:provenanceRecord
      person="John Doe"
      orgRef="http://www.legaltrans-ex.com/"
      revPerson="Tommy Atkins"
      revOrgRef="http://www.vistatec.com/"
      provRef="http://www.vistatec.com/job-12-7-15-X31/reviewed/prov/re8573469"/>
    <its:provenanceRecord
      revPerson="John Smith"
      revOrgRef="http://john-smith.qa.example.com"/>
  </its:provenanceRecords>
  <its:rules version="2.0">
    <its:provRule selector="//*[@ref]" provenanceRecordsRefPointer="@ref"/>
  </its:rules>
  <title>Translation Revision Provenance Agent: Global Test in XML</title>
  <body>
    <par ref="#pr1"> This paragraph was translated from the machine.</par>
    <legalnotice postediting-by="http://www.vistatec.com/" ref="#pr2">This text was
      translated directly by a person.</legalnotice>
  </body>
</text>

LOCAL: Using the inline markup to represent the data category locally is limited to a single occurrence for a given content (e.g. one cannot have different toolRef attributes applied to the same span of text because the inner-most one would override the others). A local standoff markup is provided to allow such cases.

The following local markup is available for the Provenance data category:

• Either (inline markup): at least one of the following attributes:

  • A person or personRef attribute that implement the human provenance information.

  • An org or orgRef attribute that implement the organisational provenance information.

  • A tool or toolRef attribute that implement the tool related provenance information.

  • A revPerson or revPersonRef attribute that implement the human revision provenance information.

  • A revOrg or revOrgRef attribute that implement the organisational revision provenance information.

  • A revTool or revToolRef attribute that implement the tool related revision provenance information.

  • A provRef attribute that implements the reference to external provenance descriptions.

• Or (standoff markup):

  • A provenanceRecordsRef attribute. Its value is a IRI pointing to the provenanceRecords element containing the list of provenance records related to this content.

  • An element provenanceRecords which contains:

    • One or more elements provenanceRecord, each of which contains at least one of the following attributes:

      • A person or personRef attribute that implement the human provenance information.

      • An org or orgRef attribute that implement the organisational provenance information.

      • A tool or toolRef attribute that implement the tool related provenance information.

      • A revPerson or revPersonRef attribute that implement the human revision provenance information.

      • A revOrg or revOrgRef attribute that implement the organisational revision provenance information.

      • A revTool or revToolRef attribute that implement the tool related revision provenance information.

      • A provRef attribute that implements the reference to external provenance descriptions.

  When the attributes person, personRef, org, orgRef, tool, toolRef, revPerson, revPersonRef, revOrg, revOrgRef, revTool, revToolRef and provRef are used in a standoff manner, the information they carry pertains to the content of the element that refers to the standoff annotation, not to the content of the element provenanceRecord where they are declared.

  In HTML the standoff markup MUST be stored inside a script element. It MUST have a type attribute with the value application/its+xml. Its id attribute MUST be set to the same value as the xml:id attribute of the provenanceRecords element it contains.

<text xmlns:dc="http://purl.org/dc/elements/1.1/"
  xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <title>Translation Revision Provenance Agent: Local Test in XML</title>
  <body>
    <par its:toolRef="http://www.onlinemtex.com/2012/7/25/wsdl/"
      its:org="acme-CAT-v2.3"
      its:revToolRef="http://www.mycat.com/v1.0/download"
      its:revOrg="acme-CAT-v2.3"
      its:provRef="http://www.example.lsp1.com/prov/e6354 http://www.example.lsp2.com/prov/e7738"
      >This paragraph was translated from the machine.</par>
    <legalnotice its:person="John Doe"
      its:orgRef="http://www.legaltrans-ex.com/"
      its:provRef="http://www.examplelsp.com/excontent987/legal/prov/e6354"
      its:revPerson="Tommy Atkins"
      its:revOrgRef="http://www.vistatec.com/"
      >This text was translated directly by a person.</legalnotice>
  </body>
</text>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>Provenance Agent: Local Test in HTML5</title>
  </head>
  <body>
    <p its-tool-ref="http://www.onlinemtex.com/2012/7/25/wsdl/"
      its-org="acme-CAT-v2.3"
      its-prov-ref="http://www.examplelsp.com/excontent987/production/prov/e6354"
      its-rev-org="acme-CAT-v2.3"
      >This paragraph was translated from the machine.</par>
    <p class="legal-notice"
     its-person="John Doe"
     its-org-ref="http://www.legaltrans-ex.com/"
     its-prov-ref="http://www.examplelsp.com/excontent987/legal/prov/e6354"
     its-rev-person="Tommy Atkins" its-rev-org-ref="http://www.vistatec.com/"
     >This text was translated directly by a person.</legalnotice>
  </body>
</html>

<!DOCTYPE html>
<html>
  <head>
    <meta charset=utf-8>
    <title>Test</title>
    <script id=pr1 type=application/its+xml>
      <its:provenanceRecords xml:id="pr1" xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
        <its:provenanceRecord
         toolRef="http://www.onlinemtex.com/2012/7/25/wsdl/" 
         org="acme-CAT-v2.3"
         provRef="http://www.examplelsp.com/excontent987/production/prov/e6354"/>
         revToolRef="http://www.mycat.com/v1.0/download" 
         revOrg="acme-CAT-v2.3" />
      </its:provenanceRecords>       
    </script>
    <script id=pr2 type=aplication/its+xml>
      <its:provenanceRecords xml:id="pr2" xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
        <its:provenanceRecord
         person="John Doe"
         orgRef="http://www.legaltrans-ex.com/"
         provRef="http://www.examplelsp.com/excontent987/legal/prov/e6354"
         revPerson="Tommy Atkins" 
         revOrgRef="http://www.vistatec.com/" />
        <its:provenanceRecord
         revPerson="John Smith" 
         revOrgRef="http://john-smith.qa.example.com" />
        </its:provenanceRecords>
      </script>
  </head>
  <body>
    <p its-provenance-records-ref="#pr1">This paragraph was translated from the machine.</p>
    <p its-provenance-records-ref="#pr2">This text was translated directly by a person.</p>
  </body>
</html>

8.13.1 Definition

The External Resource data category indicates that a node represents or references potentially translatable data in a resource outside the document. Examples of such resources are external images and audio or video files.

8.13.2 Implementation

The External Resource data category can be expressed only with global rules. There is no inheritance. There is no default.

GLOBAL: The externalResourceRefRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required externalResourceRefPointer attribute that contains a relative selector pointing to a node that provides the IRI of the external resource.

<doc xmlns:its="http://www.w3.org/2005/11/its"
  xmlns:db="http://docbook.org/ns/docbook">
  <its:rules version="2.0">
    <its:externalResourceRefRule
      selector="//db:imagedata | //db:audiodata |  //db:videodata"
      externalResourceRefPointer="@fileref"/>
  </its:rules>
  <db:mediaobject>
    <db:videoobject>
      <db:videodata fileref="movie.avi"/>
    </db:videoobject>
    <db:imageobject>
      <db:imagedata fileref="movie-frame.gif"/>
    </db:imageobject>
    <db:textobject>
      <db:para>This video illustrates the proper way to assemble an inverting
        time distortion device. </db:para>
      <db:warning>
        <db:para> It is imperative that the primary and secondary temporal
          couplings not be mounted in the wrong order. Temporal catastrophe is
          the likely result. The future you destroy may be your own. </db:para>
      </db:warning>
    </db:textobject>
  </db:mediaobject>
</doc>

<its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its"
  xmlns:html="http://www.w3.org/1999/xhtml">
  <its:externalResourceRefRule selector="//html:video/@src"
    externalResourceRefPointer="."/>
  <its:externalResourceRefRule selector="//html:video/@poster"
    externalResourceRefPointer="."/>
</its:rules>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>Video element example</title>
  </head>
  <body>
    <video
      height=360
      poster=video-image.png
      src=http://www.example.com/video/v2.mp
      width=640>
      <p>If your browser doesn't support the <code>video</code> element, you can <a href=http://www.example.com/video/v2.mp>download the video</a> instead.</p>
    </video>
  </body>
</html>

8.14.1 Definition

Some formats, such as those designed for localization or for multilingual resources, hold the same content in different languages inside a single document. The Target Pointer data category is used to associate the node of a given source content (i.e. the content to be translated) and the node of its corresponding target content (i.e. the source content translated into a given target language).

This specification makes no provision regarding the presence of the target nodes or their content: A target node may or may not exist and it may or may not have content.

This data category can be used for several purposes, including but not limited to:

• Extract the source content to translate and put back the translation at its proper location.

• Compare source and target content for quality verification.

• Re-use existing translations when localizing the new version of an existing document.

• Access aligned bi-lingual content to build memories, or to train machine translation engines.

8.14.2 Implementation

The Target Pointer data category can be expressed only with global rules. There is no inheritance. There is no default.

GLOBAL: The targetPointerRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required targetPointer attribute. It contains a relative selector that points to the node for the target content corresponding to the selected source node.

<file>
  <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
    <its:translateRule selector="/file" translate="no"/>
    <its:translateRule selector="//source" translate="yes"/>
    <its:targetPointerRule selector="//source" targetPointer="../target"/>
  </its:rules>
  <entry id="one">
    <source>Remember last folder</source>
    <target/>
  </entry>
  <entry id="two">
    <source>Custom file filter:</source>
    <target/>
  </entry>
</file>

8.15.1 Definition

The Id Value data category indicates a value that can be used as unique identifier for a given part of the content.

The recommended way to specify a unique identifier is to use xml:id or id in HTML (See the best practice “Defining markup for unique identifiers” from [XML i18n BP]). The idValueRule element is intended only as a fall-back mechanism for documents where unique identifiers are available with another construct.

Providing a unique identifier that is maintained in the original document can be useful for several purposes, for example:

• Allow automated alignment between different versions of the source document, or between source and translated documents.

• Improve the confidence in leveraged translation for exact matches.

• Provide back-tracking information between displayed text and source material when testing or debugging.

8.15.2 Implementation

The id Value data category can be expressed only with global rules. There is no inheritance. There is no default.

GLOBAL: The idValueRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required idValue attribute. It contains an XPath expression which constructs a string corresponding to the identifier of the node to which this rule applies. The identifier MUST be unique at least within the document. If the attribute xml:id is present or id in HTML for the selected node, the value of the xml:id attribute or id in HTML MUST take precedence over the idValue value.

<?xml version="1.0"?>
<resources>
  <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
    <its:translateRule translate="no" selector="/resources"/>
    <its:translateRule translate="yes" selector="//text"/>
    <its:idValueRule selector="//text" idValue="../@name"/>
  </its:rules>
  <entry name="btn.OK">
    <text>OK</text>
    <pos>1, 1</pos>
    <trig>sendOK</trig>
  </entry>
  <entry name="btn.CANCEL">
    <text>Cancel</text>
    <pos>2, 1</pos>
    <trig>cancelAll</trig>
  </entry>
</resources>

<?xml version="1.0"?>
<doc>
  <its:rules version="2.0" xmlns:its="http://www.w3.org/2005/11/its">
    <its:idValueRule selector="//text" idValue="concat(../@name, '_t')"/>
    <its:idValueRule selector="//desc" idValue="concat(../@name, '_d')"/>
  </its:rules>
  <msg name="settingsMissing">
    <text>Can't find settings file.</text>
    <desc>The module cannot find the default settings file. You need to
      re-initialize the system.</desc>
  </msg>
</doc>

<?xml version="1.0"?>
<file>
  <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
    <its:idValueRule selector="//res" idValue="@name"/>
  </its:rules>
  <res name="retryBtn" xml:id="btnAgain">Try Again</res>
  <res name="retryTip">click this to re-run the process with the current
    settings.</res>
</file>

8.16.1 Definition

The Preserve Space data category indicates how whitespace should be handled in content. The possible values for this data category are "default" and "preserve" and carry the same meaning as the corresponding values of the xml:space attribute. The default value is "default".

8.16.2 Implementation

The Preserve Space data category can be expressed with global rules, or locally using the xml:space attribute. For elements, the data category information inherits to the textual content of the element, including child elements and attributes.

GLOBAL: The preserveSpaceRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required space attribute with the value "default" or "preserve".

<book>
 <info>
  <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
   <its:preserveSpaceRule selector="//verse" space="preserve"/>
  </its:rules>
 </info>
 <verse>
’Twas brillig, and the slithy toves
 Did gyre and gimble in the wabe;
All mimsy were the borogoves,
 And the mome raths outgrabe.
 </verse>
</book>

LOCAL: The xml:space attribute, as defined in section 2.10 of [XML 1.0], maps exactly to the Preserve Space data category.

<book>
 <verse xml:space="preserve">
'Twas brillig, and the slithy toves
 Did gyre and gimble in the wabe;
All mimsy were the borogoves,
 And the mome raths outgrabe.
</verse>
</book>

8.17.1 Definition

The Localization Quality Issue data category is used to express information related to localization quality assessment tasks. Such tasks can be conducted on the translation of some source text into a target language or on the source text itself where its quality may impact on the localization process.

This data category can be used in a number of ways, including the following example scenarios:

• An automatic quality checking tool flags a number of potential quality issues in an XML or HTML file and marks them up using ITS 2.0 markup. Other tools in the workflow then examine this markup and decide whether the file needs to be reviewed manually or passed on for further processing without a manual review stage.

• A quality assessment process identifies a number of issues and adds the ITS markup to a rendered HTML preview of an XML file along with CSS styling that highlights these issues. The resulting HTML file is then sent back to the translator to assist his or her revision efforts.

• A human reviewer working with a web-based tool adds quality markup, including comments and suggestions, to a localized text as part of the review process. A subsequent process examines this markup to ensure that changes were made.

The data category defines five pieces of information:

8.17.2 Implementation

The Localization Quality Issue data category can be expressed with global rules, or locally on individual elements. For elements, the data category information inherits to the textual content of the element, including child elements, but excluding attributes.

GLOBAL: The locQualityIssueRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• Exactly one of the following:

  • Exactly one of the following:

    • A locQualityIssuesRef attribute. Its value is an IRI pointing to the locQualityIssues element containing the list of issues related to this content.

    • A locQualityIssuesRefPointer attribute that contains a relative selector pointing to a node with the exact same semantics as locQualityIssuesRef.

  • At least one of the following:

    • A locQualityIssueType attribute that implements the type information.

    • A locQualityIssueComment attribute that implements the comment information.

• An optional locQualityIssueSeverity attribute that implements the severity information.

• An optional locQualityIssueProfileRef attribute that implements the profile reference information.

• An optional locQualityIssueEnabled attribute that implements the enabled information.

<?xml version="1.0"?>
<doc>
  <header>
    <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
      <its:locQualityIssueRule selector="//image[@id='i1']/@text"
        locQualityIssueType="typographical"
        locQualityIssueComment="Sentence without capitalization"
        locQualityIssueSeverity="50"/>
    </its:rules>
  </header>
  <para>Click the button <image id="i1" src="button.png"
                          text="start button"/>.</para>
</doc>

<?xml version="1.0"?>
<doc xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <file>
    <header>
      <its:rules version="2.0">
        <its:locQualityIssueRule selector="//mrk[@type='x-itslq']"
          locQualityIssuesRefPointer="@ref"/>
      </its:rules>
    </header>
    <unit id="1">
      <segment>
        <source>This is the content</source>
        <target><mrk type="x-itslq" ref="#lq1">c'es</mrk> le contenu</target>
      </segment>
      <its:locQualityIssues xml:id="lq1">
        <its:locQualityIssue locQualityIssueType="misspelling"
          locQualityIssueComment="'c'es' is unknown. Could be 'c'est'"
          locQualityIssueSeverity="50"/>
        <its:locQualityIssue locQualityIssueType="typographical"
          locQualityIssueComment="Sentence without capitalization"
          locQualityIssueSeverity="30"/>
      </its:locQualityIssues>
    </unit>
  </file>
</doc>

LOCAL: Using the inline markup to represent the data category locally is limited to a single occurrence for a given content (e.g. one cannot have different locQualityIssueType attributes applied to the same span of text because the inner-most one would override the others). A local standoff markup is provided to allow such cases.

The following local markup is available for the Localization Quality Issue data category:

• Either (inline markup):

  • At least one of the following attributes:

    • A locQualityIssueType attribute that implements the type information.

    • A locQualityIssueComment attribute that implements the comment information.

  • An optional locQualityIssueSeverity attribute that implements the severity information.

  • An optional locQualityIssueProfileRef attribute that implements the profile reference information.

  • An optional locQualityIssueEnabled attribute that implements the enabled information.

• Or (standoff markup):

  • A locQualityIssuesRef attribute. Its value is an IRI pointing to the locQualityIssues element containing the list of issues related to this content.

  • An element locQualityIssues with a xml:id attribute set to the identifier specified in the locQualityIssuesRef attribute. The locQualityIssues element contains:

    • One or more elements locQualityIssue, each of which contains:

      • At least one of the following attributes:

        • A locQualityIssueType attribute that implements the type information.

        • A locQualityIssueComment attribute that implements the comment information.

      • An optional locQualityIssueSeverity attribute that implements the severity information.

      • An optional locQualityIssueProfileRef attribute that implements the profile reference information.

      • An optional locQualityIssueEnabled attribute that implements the enabled information.

  When the attributes locQualityIssueType, locQualityIssueComment, locQualityIssueSeverity, locQualityIssueProfileRef and locQualityIssueEnabled are used in a standoff manner, the information they carry pertains to the content of the element that refers to the standoff annotation, not to the content of the element locQualityIssue where they are declared.

  In HTML the standoff markup MUST be stored inside a script element. It MUST have a type attribute with the value application/its+xml. Its id attribute MUST be set to the same value as the xml:id attribute of the locQualityIssues element it contains.

<?xml version="1.0"?>
<doc xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <para><span its:locQualityIssueType="typographical"
      its:locQualityIssueComment="Sentence without capitalization"
      its:locQualityIssueSeverity="50">this</span> is an example</para>
</doc>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>Telharmonium 1897</title>
    <style type=text/css>
         [its-loc-quality-issue-type]{
           background-color:yellow;
           margin:2px;
         }
         [its-loc-quality-issue-severity = "100"]{
           border: 2px solid red;
         }
        </style>
  </head>
  <body>
    <h1>Telharmonium (1897)</h1>
    <p>
      <span
        data-mytool-qacode=named_entity_not_found
        its-loc-quality-issue-comment="Should be Thomas Cahill."
        its-loc-quality-issue-profile-ref=http://example.org/qaMovel/v1
        its-loc-quality-issue-severity=100
        its-loc-quality-issue-type=inconsistent-entities>Christian Bale</span>(1867–1934) conceived of an instrument that could transmit its sound
      from a power plant for hundreds of miles to listeners over telegraph wiring. Beginning in
      1889 the sound quality of regular telephone concerts was very poor on account of the buzzing
      generated by carbon-granule microphones. As a result Cahill decided to set a new standard in
      perfection of sound <span
        its-loc-quality-issue-comment="should be 'quality'"
        its-loc-quality-issue-profile-ref=grammar
        its-loc-quality-issue-severity=50
        its-loc-quality-issue-type=spelling>qulaity</span> with his instrument, a standard that would not only satisfy listeners but that
      would overcome all the flaws of traditional instruments.</p>
  </body>
</html>

<?xml version="1.0"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2"
  xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <file original="example.doc" source-language="en" datatype="plaintext">
    <body>
      <trans-unit id="1">
        <source xml:lang="en">This is the content</source>
        <target xml:lang="fr"><mrk mtype="x-itslq"
            its:locQualityIssuesRef="#lq1">c'es</mrk> le contenu</target>
        <its:locQualityIssues xml:id="lq1">
          <its:locQualityIssue locQualityIssueType="misspelling"
            locQualityIssueComment="'c'es' is unknown. Could be 'c'est'"
            locQualityIssueSeverity="50"/>
          <its:locQualityIssue locQualityIssueType="typographical"
            locQualityIssueComment="Sentence without capitalization"
            locQualityIssueSeverity="30"/>
        </its:locQualityIssues>
      </trans-unit>
    </body>
  </file>
</xliff>

<!DOCTYPE html>
<html>
  <head>
    <meta charset=utf-8>
    <title>Test</title>
    <script src=qaissues.js type=text/javascript></script>
    <script type=application/its+xml id=lq1>
      <its:locQualityIssues xml:id="lq1" xmlns:its="http://www.w3.org/2005/11/its">
        <its:locQualityIssue
          locQualityIssueType="misspelling"
          locQualityIssueComment="'c'es' is unknown. Could be 'c'est'"
          locQualityIssueSeverity="50"/>
        <its:locQualityIssue
          locQualityIssueType="typographical"
          locQualityIssueComment="Sentence without capitalization"
          locQualityIssueSeverity="30"/>
      </its:locQualityIssues>
    </script>
    <style type=text/css>.qaissue { background-color: yellow; } </style>
  </head>
  <body onload=addqaissueattrs()>
    <p>
      <span its-loc-quality-issues-ref=#lq1>c'es</span> le contenu</p>
  </body>
</html>

8.18.1 Definition

The Localization Quality Rating data category is used to express an overall measurement of the localization quality of a document or an item in a document.

This data category allows to specify a quality score or a voting result for a given item or document, as well as to indicate what constitutes a passing score or vote. It also allows to point to a profile describing the quality assessment model used for the scoring or the voting.

8.18.2 Implementation

The Localization Quality Rating data category is only expressed locally on individual elements. The data category information inherits to the textual content of the element, including child elements, but excluding attributes.

LOCAL: The following local markup is available for the Localization Quality Rating data category:

• Exactly one of the following:

  • A locQualityRatingScore attribute. Its value is a rational number in the interval 0 to 100 (inclusive). The value follows the XML Schema decimal data type with the constraining facets minInclusive set to 0 and maxInclusive set to 100. The higher values represent better quality.

  • A locQualityRatingVote attribute. Its value is a signed integer with higher values indicating a better vote.

• If locQualityRatingScore is used:

  • an optional locQualityRatingScoreThreshold attribute indicating the lowest score that constitutes a passing score in the profile used. Its value is a rational number in the interval 0 to 100 (inclusive). The value follows the XML Schema decimal data type with the constraining facets minInclusive set to 0 and maxInclusive set to 100.

• If locQualityRatingVote is used:

  • an optional locQualityRatingVoteThreshold attribute indicating the lowest value that constitutes a passing vote in the profile used. Its value is a signed integer.

• An optional locQualityRatingProfileRef attribute. Its value is an IRI pointing to the reference document describing the quality assessment model used for the scoring.

<?xml version="1.0"?>
<doc xml:lang='nl'
  xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0"
  its:locQualityRatingScore="100"
  its:locQualityRatingScoreThreshold="95"
  its:locQualityRatingProfileRef="http://example.org/qaModel/v13">
  <title>De lotgevallen van Tom Sawyer</title>
  <para>Hij kwam vrij laat te huis, en toen hij voorzichtig het raam insprong,
   viel hij in eene hinderlaag, in de persoon van zijne tante, bij wie, toen zij
   den staat zag, waarin zijne kleederen verkeerden, het besluit om zijn vrijen
   Zaterdag in een gevangenschap met dwangarbeid te veranderen, onherroepelijk
   vaststond.</para>
</doc>

<html lang=fr
  its-loc-quality-rating-profile-ref=http://example.org/qaModel/v13
  its-loc-quality-rating-score=90
  its-loc-quality-rating-score-threshold=80
  >
 <head>
  <meta charset=utf-8>
  <title>Rikki-tikki-tavi</title>
 </head>
 <body>
  <p>C'est l'histoire de la grande guerre que Rikki-Tikki-Tavi a combattu tout seul, 
   à travers les salles de bain du grand bungalow au cantonnement Segowlee. Darzee,
   le tailbird, l'a aidé, et Chuchundra, le rat musqué, qui ne sort jamais jusqu'au
   milieu du plancher, mais se glisse toujours contre la paroi, lui donnait des
   conseils, mais Rikki-Tikki-Tavi fait le véritable combat.</p>
 </body>
</html>

8.19.1 Definition

The MT Confidence data category is used to communicate the self-reported confidence score from a machine translation engine of the accuracy of a translation it has provided. It is not intended to provide a score that is comparable between machine translation engines and platforms. This data category does NOT aim to establish any sort of correlation between the self-reported confidence score and either human evaluation of MT usefulness, or post-editing cognitive effort. For harmonization’s sake, MT Confidence is provided as a rational number in the interval 0 to 1 (inclusive).

This data category can be used for several purposes, including, but not limited to:

• Automated prioritising of raw machine translated text for further processing based on empirically set thresholds.

• Providing readers, translators, post-editors, reviewers and proof-readers of machine translated text with self-reported relative accuracy prediction.

MT confidence scores can be displayed e.g. on websites machine translated on the fly, by simple web-based translation editors or on Computer Aided Translation (CAT) tools.

8.19.2 Implementation

The MT Confidence category can be expressed with global rules or locally on individual elements. For elements, the data category information is inherited by the textual content of the element, including child elements, but excluding attributes.

Any node selected by the MT Confidence data category MUST be contained in an element with the annotatorsRef (or in HTML, its-annotators-ref) attribute specified for the MT Confidence data category. For more information, see Section 5.8: ITS Tools Annotation.

GLOBAL: The mtConfidenceRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• A required mtConfidence attribute with a value that represents the translation confidence score as a rational number in the interval 0 to 1 (inclusive). The value follows the XML Schema decimal data type with the constraining facets minInclusive set to 0 and maxInclusive set to 1.

<!DOCTYPE html>
<html lang=en>
 <head>
  <meta charset=utf-8>
  <link href=EX-mtconfidence-global-html5-1-external-rules.xml rel=its-rules>
   <title>Machine translated title attributes of img elements give MT 
           confidence scores using global rules</title>
  </head>
  <body its-annotators-ref="mt-confidence|file:///tools.xml#T1">
   <p>     
    <img src="http://upload.wikimedia.org/wikipedia/commons/9/93/Trinity_College.jpg" 
               title="Front gate of Trinity College Dublin"/>
   <img src="http://upload.wikimedia.org/wikipedia/commons/c/cc/Molly_alone.jpg" 
               title="A tart with a cart"/>
  </p>  
 </body>
</html>

Where the external ITS rules file is as shown:

<?xml version="1.0" encoding="UTF-8"?>
<its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0"
    xmlns:h="http://www.w3.org/1999/xhtml">
    <its:mtConfidenceRule mtConfidence="0.785" selector="//h:img[@title='Front gate of Trinity College Dublin']/@title"/>        
    <its:mtConfidenceRule mtConfidence="0.805" selector="//h:img[@title='A tart with a cart']/@title"/>        
</its:rules>

LOCAL: the following local markup is available for the MT Confidence data category:

• A mtConfidence attribute with a value that represents the translation confidence score as a rational number in the interval 0 to 1 (inclusive). The value follows the XML Schema decimal data type with the constraining facets minInclusive set to 0 and maxInclusive set to 1.

<text xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0" its:annotatorsRef="mt-confidence|file:///tools.xml#T1">
  <body>
    <p>
      <span its:mtConfidence="0.8982">Dublin is the capital city of
        Ireland.</span>
    </p>
  </body>
</text>

<!DOCTYPE html>
<html lang=en >
   <head>
      <meta charset=utf-8>
      <title>Sentences about Dublin and Prague 
         machine translated from Czech with mtConfidence locally.</title>
   </head>
    <body its-annotators-ref="mt-confidence|file:///tools.xml#T1">
       <p>     
         <span its-mt-confidence=0.8982>Dublin is the capital of Ireland.</span>
         <span its-mt-confidence=0.8536 >The capital of the Czech Republic is Prague.</span>
      </p>  
   </body>
</html>

8.20.1 Definition

The Allowed Characters data category is used to specify what characters are allowed in a given piece of content.

This data category can be used for various purposes, including the following examples:

• Limiting the characters that may be used in the UI of a game due to font restrictions.

• Preventing illegal characters from being entered as text content that represents file or directory names.

• Controlling what characters can be used when translating examples of a login name in content.

The set of characters that are allowed is specified using a regular expression. That is, each character in the selected content MUST be included in the set specified by the regular expression.

The regular expression is a character class construct as defined in the section Character Classes of XML Schema [XML Schema Part 2], with the assumption that the . metacharacter matches also CARRIAGE RETURN (U+000D) and LINE FEED (U+000F). That is with the dot-all option set.

Example of expressions (shown as XML source):

• "[abc]" : allows the characters 'a', 'b' and 'c'.

• "[a-c]" : allows the characters 'a', 'b' and 'c'.

• "[a-zA-Z]" : allows the characters from 'a' to 'z' and from 'A' to 'Z'.

• "[^abc]" : allows any characters except 'a', 'b', and 'c'.

• "[^a-c]" : allows any characters except 'a', 'b', and 'c'.

• "\w" : allows any character except the set of "punctuation", "separator" and "other" characters.

• "[ --[<>:"\\/|\?*]]" : allows only the characters valid for Windows file names.

• "." : allows any character.

• "" : allows no character.

• "[a-ÿ-[\s]]" : allows all characters between U+0061 and U+00FF except the characters SPACE (U+0020), TABULATION (U+0009), CARRIAGE RETURN (U+000D) and LINE FEED (U+000F).

8.20.2 Implementation

The Allowed Characters data category can be expressed with global rules, or locally on individual elements. For elements, the data category information inherits to the textual content of the element, including child elements, but excluding attributes.

GLOBAL: The allowedCharactersRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• Exactly one of the following:

  • An allowedCharacters attribute that contains the regular expression indicating the allowed characters.

  • An allowedCharactersPointer attribute that contains a relative selector pointing to a node with the exact same semantics as allowedCharacters.

<?xml version="1.0"?>
<myRes xmlns:its="http://www.w3.org/2005/11/its">
  <head>
    <its:rules version="2.0">
      <its:allowedCharactersRule allowedCharacters="[^*+]" selector="//content"/>
    </its:rules>
  </head>
  <body>
    <content>Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam
      nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed
      diam voluptua.</content>
  </body>
</myRes>

<?xml version="1.0"?>
<res xmlns:its="http://www.w3.org/2005/11/its">
  <head>
    <its:rules version="2.0">
      <its:allowedCharactersRule selector="//record" allowedCharactersPointer="@set"/>
    </its:rules>
  </head>
  <record id="a1" set="[ &#xFF01;–&#xFF5E;]">ＦＵＬＬ ＷＩＤＴＨ ＯＮＬＹ</record>
</res>

LOCAL: the following local markup is available for the Allowed Characters data category:

• A allowedCharacters attribute that contains the regular expression indicating the allowed characters.

<?xml version="1.0"?>
<messages xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <msg num="123">Click the <panelmsg its:allowedCharacters="[&#x0020;-&#x00FE;]"
      >CONTINUE</panelmsg> Button on the printer panel</msg>
</messages>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>Example</title>
  </head>
  <body>
    <p>Login names can only use letters from A to Z (upper or lowercase)
    and the character underscore (_) and minus (-).
    For example: <code its-allowed-characters=[a-zA-Z_\-]>Huck_Finn</code>.</p>
  </body>
</html>

8.21.1 Definition

The Storage Size data category is used to specify the maximum storage size of a given content.

This data category can be used for various purposes, including the following examples:

• Verify during translation if a string fits into a fixed-size database field.

• Control the size of a string that is stored in a fixed-size memory buffer at run-time.

The storage size is expressed in bytes and is provided along with the character set encoding used to store the content.

8.21.2 Implementation

The Storage Size data category can be expressed with global rules, or locally on individual elements. There is no inheritance. The default value of the character set encoding is UTF-8.

GLOBAL: The storageSizeRule element contains the following:

• A required selector attribute. It contains an absolute selector which selects the nodes to which this rule applies.

• Exactly one of the following:

  • A storageSize attribute. It contains the maximum number of bytes the text of the selected node is allowed in storage.

  • A storageSizePointer attribute that contains a relative selector pointing to a node with the exact same semantics as storageSize.

• None or exactly one of the following:

  • A storageEncoding attribute. It contains the name of the character set encoding used to calculate the number of bytes of the selected text. The name MUST be one of the names or aliases listed in the IANA Character Sets registry [IANA Character Sets]. The default value is "UTF-8".

  • A storageEncodingPointer attribute that contains a relative selector pointing to a node with the exact same semantics as storageEncoding.

• An optional lineBreakType attribute. It indicates what type of line breaks the storage uses. The possible values are: cr for CARRIAGE RETURN (U+000D), lf for LINE FEED (U+000A), crlf for CARRIAGE RETURN (U+000D) followed by LINE FEED (U+000A), or nel for NEXT LINE (U+0085). The default value is lf.

<?xml version="1.0"?>
<db>
  <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
    <its:storageSizeRule selector="//country" storageSize="25"
      storageEncoding="ISO-8859-1"/>
  </its:rules>
  <data>
    <country id="123">Papouasie-Nouvelle-Guinée</country>
    <country id="139">République Dominicaine</country>
  </data>
</db>

<?xml version="1.0"?>
<fields>
  <its:rules xmlns:its="http://www.w3.org/2005/11/its" version="2.0">
    <its:storageSizeRule selector="//field" storageSizePointer="@max"/>
  </its:rules>
  <field type="country" id="123" max="25">Papouasie-Nouvelle-Guinée</field>
  <field type="country" id="139" max="25">République Dominicaine</field>
</fields>

LOCAL: the following local markup is available for the Storage Size data category:

• A storageSize attribute. It contains the maximum number of bytes the text of the selected node is allowed in storage.

• An optional storageEncoding attribute. It contains the name of the character set encoding used to calculate the number of bytes of the selected text. The name MUST be one of the names or aliases listed in the IANA Character Sets registry [IANA Character Sets]. The default value is "UTF-8".

• An optional lineBreakType attribute. It indicates what type of line breaks the storage uses. The possible values are: cr for CARRIAGE RETURN (U+000D), lf for LINE FEED (U+000A), crlf for CARRIAGE RETURN (U+000D) followed by LINE FEED (U+000A), or nel for NEXT LINE (U+0085). The default value is lf.

<?xml version="1.0"?>
<messages xmlns:its="http://www.w3.org/2005/11/its" its:version="2.0">
  <var num="panelA1_Continue" its:storageSize="8" its:storageEncoding="UTF-16">CONTINUE</var>
  <var num="panelA1_Stop" its:storageSize="8" its:storageEncoding="UTF-16">STOP</var>
  <var num="panelB5_Cancel" its:storageSize="12" its:storageEncoding="UTF-16">CANCEL</var>
</messages>

<!DOCTYPE html>
<html lang=en>
  <head>
    <meta charset=utf-8>
    <title>Example</title>
  </head>
  <body>
    <p>String to translate:</p>
    <p contenteditable=true id=123 its-storage-size=25>Papua New-Guinea</p>
      <p contenteditable=true id=139 its-storage-size=25>Dominican Republic</p>
  </body>
</html>