This document defines data categories and their
implementation as a set of elements and attributes called the
This document defines data categories and their implementation as a set of elements and attributes called the Internationalization Tag Set (ITS). ITS is designed to be used with schemas to support the internationalization and localization of schemas and documents. An implementation is provided for three schema languages: XML DTD, XML Schema and RELAX NG.
This is a
This document has been reviewed by W3C Members, by software developers, and by other W3C groups and interested parties, and is endorsed by the Director as a W3C Recommendation. It is a stable document and may be used as reference material or cited from another document. W3C's role in making the Recommendation is to draw attention to the specification and to promote its widespread deployment. This enhances the functionality and interoperability of the Web.
Please make comments about this document using
W3C's
This document incorporates minor changes made against the Proposed Recommendation of 21 November 2006; please see the
This document was produced by a group operating under the
This is an updated version of this document.
ITS is a technology to easily create XML which is internationalized and can be localized effectively. On the one hand, the ITS specification identifies concepts (such as "directionality") which are important for
internationalization and localization. On the other hand, the ITS specification defines implementations of these concepts (termed "ITS data categories") as a set of elements and attributes called the Internationalization Tag Set (ITS). The document provides implementations for three schema languages: XML DTD
This document aims to realize many of the ideas formulated in
Requirements for this document are formulated in
This document covers the following requirements:
The following requirements will be addressed in
The Working Group decided not to cover the following requirements at this time to be able to focus on the most important ones.
Content or software that is authored in one language (so-called source language) is often made available in additional languages or adapted with regard to other cultural aspects. This is done through a process called localization, where the original material is translated and adapted to the target audience.
In addition, document formats expressed by schemas may be used by people in different parts of the world, and these people may need special markup to support the local language or script. For example, people authoring in languages such as Arabic, Hebrew, Persian or Urdu need special markup to specify directionality in mixed direction text.
From the viewpoints of feasibility, cost, and efficiency,
it is important that the original material should be
suitable for localization. This is achieved by appropriate
design and development, and the corresponding process is
referred to as internationalization. For a detailed
explanation of the terms "localization" and
"internationalization", see
The increasing usage of XML as a medium for
documentation-related content (e.g. DocBook and DITA as
formats for writing structured documentation, well suited to
computer hardware and software manuals) and software-related
content (e.g. the eXtensible User Interface Language
The following examples sketch one of the issues that currently hinder efficient XML-related localization: the lack of a standard, declarative mechanism which identifies which parts of an XML document need to be translated. Tools often cannot automatically do this identification.
In this document it is difficult to make distinction between the
Even when metadata are available to identify non-translatable text, the conditions may be quite complex and not directly indicated with a simple flag. Here, for instance, only the text in the nodes matching the expression //component[@type!='image']/data[@type='text'] is translatable.
The ITS specification aims to provide different types of users with information about what markup should be supported to enable worldwide use and effective internationalization and localization of content. The following paragraphs sketch these different types of users, and their usage of ITS.
Schema developers who start a schema from ground up
This type of user will find proposals for attribute and element names to be included in their new schema (also called "host vocabulary"). Using the attribute and element names proposed in the ITS specification may be helpful because it leads to easier recognition of the concepts represented by both schema users and processors. It is perfectly possible, however, for a schema developer to develop his own set of attribute and element names. The specification sets out, first and foremost, to ensure that the required markup is available, and that the behavior of that markup meets established needs.
Schema developers who work with an existing schema
This type of user will be working with schemas such as DocBook, DITA, or perhaps a proprietary schema. The ITS Working Group has sought input from experts developing widely used formats such as the ones mentioned.
The question
"How to use ITS with existing popular markup schemes?"
is covered in more details (including examples) in a separate document:
Developers working on existing schemas should check whether their schemas support the markup proposed in this specification, and, where appropriate, add the markup proposed here to their schema.
In some cases, an existing schema may
already contain markup equivalent to that
recommended in ITS. In this case it is not necessary
to add duplicate markup since ITS provides
mechanisms for associating ITS
markup with markup in the host vocabulary which
serves a similar purpose (see
Vendors of content-related tools
This type of user includes companies which provide tools for authoring, translation or other flavors of content-related software solutions. It is important to ensure that such tools enable worldwide use and effective localization of content. For example, translation tools should prevent content marked up as not for translation from being changed or translated. It is hoped that the ITS specification will make the job of vendors easier by standardizing the format and processing expectations of certain relevant markup items, and allowing them to more effectively identify how content should be handled.
Content producers
This type of user comprises authors, translators and other types of content author. The markup proposed in this specification may be used by them to mark up specific bits of content. Aside: The burden of inserting markup can be removed from content producers by relating the ITS information to relevant bits of content in a global manner (see global, rule-based approach). This global work, however, may fall to information architects, rather than the content producers themselves.
In order to support all of these users, the information about what markup should be supported to enable worldwide use and effective localization of content is provided in this specification in two ways:
The ITS specification proposes several mechanisms for supporting worldwide use and effective internationalization and localization of content. We will sketch them below by looking at them from the perspectives of certain user types. For the purpose of illustration, we will demonstrate how ITS can indicate that certain parts of content should or should not be translated.
A content author uses an attribute on a particular element to say that the text in the element should not be translated.
The its:translate="no" attributes indicate that the
A content author or information architect uses markup at the top of the document to identify a particular type of element or context in which the content should not be translated.
The
A processor may insert markup at the top of the document which links to ITS information outside of the document.
A
The
A schema developer integrates ITS markup declarations in his schema to allow users to indicate that specific parts of the content should not be translated.
The declarations for the commonAtts. This allows to use the
The first two approaches above can be likened to the
use of CSS in
This standard does not cover all mechanisms and data formats (sometimes called
"XML localization properties" is a generic term to name the mechanisms and data formats that allow localization tools to be configured in order to process a specific XML format. Examples of "XML localization properties" are the Trados "DTD Settings" file, and the SDLX "Analysis" file.
Abstraction via
Powerful
Content authors need, for example, a simple way to work
with the Translate data category in order to express whether the content of an
element or attribute should be translated or
not. Localization coordinators, on the other hand, need an
efficient way of managing translations of large document
sets based on the same schema. This could by realized by a
specification of defaults for the Translate data category and exceptions
from the defaults (e.g. all
To meet these requirements this specification introduces mechanisms that add ITS information to XML documents, see
The ITS selection mechanisms allows you to provide information about content locally (specified at the XML node to which it pertains) or globally (specified in another part of the document). Global selection mechanisms can be in the same document, or in a separate file.
This specification has been developed using the ODD (
XSLT transformations are provided by the TEI to create documentation into HTML, XSL FO or LaTeX forms, and to generate RELAX NG documents and DTD. From the RELAX NG documents, James Clark's trang can be used to create XML Schema documents.
Information (e.g. "translate this") captured by ITS markup
(e.g. its:translate='yes') always pertains to
one or more XML nodes (mainly element and attribute nodes). In
a sense, ITS markup "selects" the XML node(s). Selection may
be explicit or implicit. ITS distinguishes two approaches to
selection: local, and with global rules.
The mechanisms defined for ITS selection resemble those
defined in
ITS markup can be used with XML documents (e.g. a DocBook article), or schemas (e.g. an XML Schema document for a proprietary document format). Since each usage defines some specific requirements, ITS markup may take different shapes.
The following two examples sketch the distinction between the local and global approaches.
The document in
For this to work, the schema developer will need to add the
The document in
Caveat Related to XSLT-based Processing of ITS Selector Attributes
The values of ITS
myElement/descendant-or-self::*/@*
Unfortunately, values like this cause trouble when they are used in
XSLT-based processing of ITS where the values of the ITS
Basically the following restrictions hold for patterns:
Using only XSLT patterns in ITS
*[self::myElement]/@* | myElement//*/@*
For this approach to work, the schema developer needs to add the
In some cases this may allow the schema developer to avoid adding
other ITS markup (such as an
For specification of the
Translate data category information, the contents of the
The global, rule-based approach has the following benefits:
The commonality in both examples above is the markup
translate='no'. This piece of ITS markup can
be interpreted as follows:
The ITS
term element in DITA)The power of the ITS selection mechanisms comes at a price: rules related to overriding/precedence, and inheritance, have to be established.
The document in its:translate="yes". Note that the global rule is processed first, regardless of its position inside the document. In the main body of the document, the default applies, and here it is its:translate="no" that is used to set "faux pas" as non-translatable.
For some data categories, special attributes add or point to information about the selected nodes. For example, the Localization Note data category can add information to selected nodes (using a
The functionality of adding information to the selected
nodes is available for each data category except Language Information. Pointing to
existing information is not possible for data categories that
express
The functionalities of adding information and pointing to
existing information are
The keywords "MUST", "MUST NOT",
"REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT",
"RECOMMENDED", "MAY", and "OPTIONAL" in this document are to
be interpreted as described in
The namespace URI that MUST be used by implementations of this specification is:
The namespace prefix used in this specification for this URI is "its". It is recommended that implementations of this specification use this prefix.
In addition, the following namespaces are used in this document:
http://www.w3.org/2001/XMLSchema for
the XML Schema namespace, here used with the prefix
"xs"http://relaxng.org/ns/structure/1.0 for
the RELAX NG namespace, here used with the prefix
"rng"http://www.w3.org/1999/xlink for the XLink namespace, here used with the prefix "xlink"
This specification provides schemas in the format of XML DTD, XML Schema or RELAX NG. However, these schemas are only non-normative; conformance for ITS markup declarations defines only mandatory positions of ITS declarations in schemas. This makes it possible to use ITS with any schema language that allows for using these positions.
For each data category, ITS distinguishes between the following:
The Translate data category conveys information as to whether a piece of content should be translated or not.
The simplest formalization of this prose description on
a schema language independent level is a
Selection relies on the information that is
given in the XML Information Set
The selection of the ITS data categories applies to textual values contained within element or attribute nodes. In
some cases these nodes form pointers to other resources; a well-known
example is the
The
attributes
The ITS schemas in
The usage of the term
This specification defines two types of conformance: conformance of 1) ITS markup declarations , and conformance of 2) processing expectations for ITS Markup. These conformance types complement each other. An implementation of this specification MAY use them separately or together.
Full implementations of this conformance type will implement all markup declarations for ITS. Statements related to this conformance type MUST list all markup declarations they implement.
Since the ITS markup declarations are schema language
independent, each schema language can use its own,
possibly multiple, mechanisms to implement the conformance
clauses for ITS markup declarations. For example, an XML
DTD can use parameter entities to encapsulate the ITS local attributes, or
declare them directly for each element. The appropriate
steps to integrate ITS into a schema depend on the design
of this schema (e.g. whether it already has a
customization layer that uses parameter entities). The
ITS schemas in the format of XML DTD, XML Schema and RELAX
NG in
Application-specific processing (that is processing that goes beyond the computation of ITS information for a node) such as automated filtering of translatable content based on the Translate data category is not covered by the conformance clauses below.
The ITS Working group provides a test suite to help implementers to write applications that support the ITS specifications. The test suite provides pairs of input and output files.
Statements related to this conformance type MUST list all data categories they implement, and for each data category which type of selection they support.
The version of the ITS schema defined in this specification is
its:version) MUST be provided at the
root element of the document. If there is both a
Each XML document can have a different
version. That is: if external rules are linked via an XLink
ITS data categories can appear in two places:
The two locations are described in detail below.
Global, rule-based selection is implemented using the
If there is more than one
Depending on the data
category and its usage, there are additional attributes
for adding information to the selected nodes, or for
pointing to existing information in the document. For
example, the Localization Note data category can be used for adding notes to
selected nodes, or for pointing to existing notes in
the document. For the former purpose, a
Each data category allows you to add information to the
selected nodes except for language
information. Pointing to existing information is not
possible for data categories that express
The functionalities of adding information and pointing
to existing information are
Another difference between adding and pointing is the usage of XPath:
The value of the /". That is, it must
be an
AbsoluteLocationPath as described in XPath 1.0 or its successor. This ensures that
the selection is not relative to a specific
location. The resulting nodes MUST be either element or
attribute nodes.
Attributes that point to existing information in the document, i.e. attributes whose name ends in ...Pointer, MUST use a RelativeLocationPath as described in XPath 1.0 or its successor. The XPath expression is evaluated relative to the nodes selected by the selector attribute. The following attributes point to existing information:
If namespaces
The term element from the TEI is in a
namespace http://www.tei-c.org/ns/1.0.
The
Global rules can appear in the XML document they will
be applied to, or in a separate XML document. The
precedence of their processing depends on these
variations. See also
Markup for global, rule-based selection is defined as follows.
Local selection in XML documents is realized with local ITS attributes, the
The content model of
The data category determines what is being selected.
The necessary data category specific defaults
are described in
By default the content of all elements in a document is translatable. The attribute its:translate="no" in the its:translate="yes" in the its:translate="no" in
The default directionality of a document is left-to-right. The its:dir="rtl" in
the
Markup for local selection is defined as follows. The attribute group att.local.no-ns.attributes contains ITS attributes in no namespace and is used with the ITS elements
One way to associate a document with a set of external
ITS rules is to use the optional XLink
The rules contained in the referenced document MUST be processed as if they were
at the top of the
The example demonstrates how metadata can be added to ITS rules.
The result of processing the two documents above is the same as processing the following document.
Applications processing global ITS markup MUST recognize the XLink
External rules may also have links to other external rules. The linking mechanism is recursive, the deepest rules being overridden by the top-most rules, if any.
The following precedence order is defined for selections of ITS information in various positions (the first item in the list has the highest precedence):
Global selections
in documents (using a
Inside each
If identical selections are defined in different rules elements within one document, the selection defined by the last takes precedence.
ITS doesn't define precedence related to rules defined or linked based on non-ITS mechanisms (such as processing instructions for linking rules).
In case of conflicts between global selections via multiple rule elements, the last selector has higher precedence.
The precedence order fulfills the same purpose as the
built-in template rules of
The two elements
The first rule specifies that
The second rule indicates that when
Some markup schemes provide markup which can be used to
express ITS data categories. ITS data categories can be associated with such existing markup, using
the global selection mechanism described in
Associating existing markup with ITS data categories can be done only if the processing expectations of the host markup are the same as, or greater than, those of ITS.
Global rules can be associated with a given XML document using different means:
By using an
The following table summarizes for each data category which selection, default value, and inheritance and overriding behavior applies.
In this example, the content of all the its:translate="no" attribute in the
The localization note for the two first its:locNote attribute.
The data categories differ with respect to defaults. This is due to existing standards and practices. It is common practice for example that information about translation refers only to textual content of an element. Thus, the default selection for the Translate data category is the textual content.
The Translate data category expresses information
about whether the content of an element or attribute
should be translated or not. The values of this data
category are
The Translate data category can be expressed with global rules, or locally on an individual element. The information applies to the textual content of the element,
GLOBAL: The
The
LOCAL: The following local markup is available for the Translate data category:
It is not possible to override the Translate data category settings of attributes using local markup. This limitation is consistent with the advised practice of not using translatable attributes.
The local its:translate="no" specifies that the content of
The Localization Note data category is used to communicate notes to localizers about a particular item of content.
This data category can be used for several purposes, including, but not limited to:
enabledin isolation without knowing the gender, number and case of the thing it refers to.)
Two types of informative notes are needed:
Editing tools may offer an easy way to create this type of information. Translation tools can be made to recognize the difference between these two types of localization notes, and present the information to translators in different ways.
The Localization Note data category can be expressed with global
rules, or locally on an individual element. The information applies to the textual
content of the element,
GLOBAL: The
Exactly one of the following:
The
The
The
The
LOCAL: The following local markup is available for the Localization Note data category:
One of the following:
It is generally recommended to avoid using attributes to store text, however, in this specific case, the need to provide the notes without interfering with the structure of the host document is outweighing the drawbacks of using an attribute.
The Terminology data category is used to mark terms and optionally associate them with information, such as definitions. This helps to increase consistency across different parts of the documentation. It is also helpful for translation.
Existing terminology standards such as
The Terminology data category can be expressed with global rules, or locally on an individual element. There is no inheritance. The default is that neither elements nor attributes are terms.
GLOBAL: The
Exactly one of the following:
LOCAL: The following local markup is available for the Terminology data category:
The Directionality data category allows the user to specify the base writing direction of blocks, embeddings and overrides for the Unicode bidirectional
algorithm. It has four values:
ITS defines only the values of the Directionality data category and their inheritance. The behavior of text labeled in this way may vary, according to the implementation. Implementers are encouraged, however, to model the behavior on that described in the CSS 2.1 specification or its successor. In such a case, the effect of the data category's values would correspond to the following CSS rules:
Data category value:
CSS rule: *[dir="ltr"] { unicode-bidi: embed; direction: ltr}
Data category value:
CSS rule: *[dir="rtl"] { unicode-bidi: embed; direction: rtl}
Data category value:
CSS rule: *[dir="lro"] { unicode-bidi: bidi-override; direction: ltr}
Data category value:
CSS rule: *[dir="rlo"] { unicode-bidi: bidi-override; direction: rtl}
More information about how to use this data category is provided by
The Directionality data category can be expressed with global rules, or locally on an individual element. The information applies to the textual content of the element,
GLOBAL: The
In this document the right-to-left directionality is marked using a
The direction="rtlText" have right-to-left content.
LOCAL: The following local markup is available for the Directionality data category:
On the first its:dir="rtl" attribute indicates a right-to-left content.
The Ruby data category is used for a run of text that is associated with another run of text, referred to as the base text. Ruby text is used to provide a short annotation of the associated base text. It is most often used to provide a reading (pronunciation) guide.
The Ruby data category can be expressed with global rules, or locally. There is no inheritance.
GLOBAL: The