Internationalization Tag Set (ITS) Version 1.0

1.1.1 Potential Users of ITS

The ITS specification aims to provide different types of users with information about what markup should be supported to enable worldwide use and effective internationalization and localization of content. The following paragraphs sketch these different types of users, and their usage of ITS.

• Schema developers who start a schema from ground up

  This type of user will find proposals for attribute and element names to be included in their new schema (aka “host vocabulary”). Using the attribute and element names proposed in the ITS specification may be helpful because it leads to easier recognition of the concepts represented by both schema users and processors. It is perfectly possible, however, for a schema developer to develop his own set of attribute and element names. The specification sets out, first and foremost, to ensure the required markup is available, and that the behaviour of that markup meets established needs.

• Schema developers who work with an existing schema

  This type of user will be working with schemas such as DocBook, DITA, or perhaps a proprietary schema.

  The ITS Working Group has sought input from experts developing widely used formats such as the ones mentioned, and the ITS specification provides examples of how those formats (aka “host vocabulary”) could be used with ITS.

  Note:

  The Working Group covers the question “How use ITS with existing popular markup schemes?” in more detail in a separate document [XML i18n BP].

  Developers working on existing schemas should check whether their schemas support the markup proposed in this specification, and, where appropriate, add the markup proposed here to their schema.

  In some cases, an existing schema may already contain markup equivalent to that recommended in ITS. In this case it is not necessary to add duplicate markup since ITS provides mechanisms for associating ITS markup with markup in the host vocabulary which serves a similar purpose (see Section 5.6: Associating ITS Data Categories with Existing Markup). The developer should, however, check that the behaviour associated with the markup in their own schema is fully compatible with the expectations described in this specification.

• Vendors of content-related tools

  This type of users encompasses companies which provide tools for authoring, translation or other flavours of content-related software solutions. It is important to ensure that such tools enable worldwide use and effective localization of content. For example, translation tools should prevent content marked up as not for translation from being changed or translated. It is hoped that the ITS specification will make the job of vendors easier by standardising the format and processing expectations of certain relevant markup items, and allowing them to more effectively identify how content should be handled.

• Content producers

  This type of users comprises authors, translators and other types of content authors. The markup proposed in this specification may be used by them to mark up specific bits of content. Aside: The burden of inserting markup should be removed from content producers by relating the ITS information to relevant bits of content in a global manner (see global, rule-based approach). This global work, however, may fall to information architects, rather than the content producers themselves.

In order to support all of these users, the information about what markup should be supported to enable worldwide use and effective internationalization and localization of content is provided in this specification in two ways:

• abstractly in the data category descriptions: Section 6: Description of Data Categories
• concretely in the ITS schemas: Appendix A: Schemas for ITS

1.1.2 Ways to Use ITS

The ITS specification proposes several mechanisms for supporting worldwide use and effective internationalization and localization of content. We will sketch them below by looking at them from the perspectives of certain user types. For the purpose of illustration, we will answer the question, how ITS can indicate that certain parts of content should or should not be translated.

• A content author uses an attribute on a particular element to say that the text in the element should not be translated

<book its:version="1.0">
 <head>...</head>
 <body> ...  <p>And he said: you need a
   new <quote its:translate="no">T-Model</quote>
  </p> ...
  </body>
</book>

• A content author or information architect uses markup at the top of the document to identify a particular type of element or context in which the content should not be translated.

<text>
 <head>
  <its:rules its:version="1.0">
   <its:translateRule translate="no" selector="//dt"/>
  </its:rules>
 </head>
 <body> ...
   <p> ... <dl>
    <dt>...</dt>
    <dd>...</dd>
   </dl>
  </p>
 </body>
</text>

• A processor may inject markup at the top of the document which links to ITS information outside of the document.

<text>
 <head>
  <its:rules its:version="1.0" xlink:href="myRules.xml"/>
 </head>
 <body> ...
   <p> ... <dl>
    <dt>...</dt>
    <dd>...</dd>
   </dl>
  </p>
 </body>
</text>

• A schema developer integrates ITS markup declarations in his schema to allow users to indicate that specific parts of the content should not be translated.

The first two approaches above can be likened to the use of CSS in XHTML. Using a style attribute, an XHTML content author may assign a colour to a particular paragraph. That author could also have used the style element at the top of the page to say that all paragraphs of a particular class or in a particular context would be coloured red.

5.3.1 Global, Rule-based Selection

Global, rule-based selection is implemented using the rules element. It contains zero or more rule elements. Each rule element has a mandatory selector attribute. This attribute and all other possible attributes at rule elements are in the empty namespace and used without a prefix.

If there is more than one rules element in an XML document, the rules from each section are to be processed at the same precedence level. The rules sections are to be read in document order, and the ITS rules with them processed sequentially. The versions of these rules elements MUST NOT be different.

Depending on the data category and its usage, there are additional attributes for adding information to the selected nodes, or for pointing to existing information in the document. For example, the data category localization information can be used for adding information to selected nodes, or for pointing to existing information in the document. For the former purpose, an locInfo element can be used. For the latter purpose, a locInfoPointer attribute can be used.

The functionality of adding information to the selected nodes is available for each data category except language information. Pointing to existing information is not possible for data categories which express a closed set of values, that is: translatability, directionality and elements within text.

The functionalities of adding information and pointing to existing information are mutually exclusive. That is: markup for pointing and adding MUST NOT appear at the same rule element.

Another difference between adding and pointing is the usage of XPath:

• [Ed. note: The working group is looking for feedback on a proposal to group rule elements.]

  The value of the selector attribute MUST be an XPath expression which starts with "/". That is, it must be an AbsoluteLocationPath as described in [XPath 1.0]. This ensures that the selection is not relative to a specific location. The resulting nodes MUST be either element or attribute nodes.

• As for data category specific attributes like locInfoPointer which point to existing information in the document, a RelativeLocationPath as described in [XPath 1.0] MUST be used. The XPath expression is evaluated relative to the nodes which are selected via the selector attribute.

If namespaces [XML Names] are used in XPath expressions in the selector attribute or the pointing attributes, the following rules MUST be applied while processing XPath:

1. For each prefix, there MUST be an ns element as a child of the rules element. The ns element has two attributes prefix (for the namespace prefix) and uri (for the namespace URI).
2. Element and attribute names without a prefix are interpreted as having no namespace.
3. To avoid a conflict with rule 2., default namespaces MUST NOT be used in the XPath expressions.

<its:rules its:version="1.0">
 <its:ns prefix="tei" uri="http://www.tei-c.org/ns/1.0"/>
 <its:termRule selector="//tei:term"/>
</its:rules>

<its:rules its:version="1.0">
 <its:termRule selector="//qterm"/>
</its:rules>

Global rules can appear in the XML document they will be applied to, or in a separate XML document. The precedence of their processing depends on these variations. See also Section 5.5: Precedence between Selections.

Markup for global, rule-based selection is defined as follows.

rules

[1]	rules	::=	element its:rules { rules.content, rules.attributes }
[2]	rules.content	::=	ns*, ( translateRule | locInfoRule | termRule | dirRule | rubyRule | langRule | withinTextRule )*
[3]	rules.attributes	::=	att.version.attributes, attribute xlink:href { xsd:anyURI }?

ns

[4]	ns	::=	element its:ns { ns.content, ns.attributes }
[5]	ns.content	::=	empty
[6]	ns.attributes	::=	attribute prefix { xsd:NCName }, attribute uri { xsd:anyURI }

att.selector

[7]	att.selector.attributes	::=	att.selector.attribute.selector
[8]	att.selector.attribute.selector	::=	attribute selector { string }

att.version

[9]	att.version.attributes	::=	att.version.attribute.version
[10]	att.version.attribute.version	::=	attribute its:version { xsd:float }

5.3.2 Local Selection in an XML Document

Local selection in XML documents is realized with local ITS attributes, the ruby element, or the span element. span serves just as a wrapper for the local ITS attributes and ruby.

It depends on the data category what is being selected. The necessary data category specific defaults are described in Section 6.1: Position and Default Selections of Data Categories.

<text its:version="1.0">
 <head its:translate="no"> ... </head>
 <body its:translate="yes" its:dir="ltr"> ... </body>
</text>

Markup for local selection is defined as follows.

span

[11]	span	::=	element its:span { span.content, span.attributes }
[12]	span.content	::=	( text | ruby )*
[13]	span.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes

6.2.1 Definition

The data category translatability expresses information about whether the content of an element or attribute should be translated or not. The values of this data category are "yes" (translatable) or "no" (not translatable).

6.2.2 Implementation

Translatability can be expressed with global rules, or locally on an individual element.

As for global rules, translatability is expressed with a translateRule element with a translate attribute. The attribute has the values "yes" or "no". In addition, a selector attribute is required.

<its:rules its:version="1.0">
 <its:translateRule translate="yes" selector="//p"/>
<!-- All p elements should be translated-->

</its:rules>

Locally, translatability is expressed with a translate attribute with the values "yes" or "no" The selection is the textual content of the element, including child elements, but excluding attributes.

<book its:version="1.0">
 <head>...</head>
 <body its:translate="yes"> ...  
   <p>And he said: you need a new 
    <quote its:translate="no">motherboard</quote>
  </p> ...  </body>
</book>

6.2.3 Markup Declarations for Translatability

translateRule

[14]	translateRule	::=	element its:translateRule { translateRule.content, translateRule.attributes }
[15]	translateRule.content	::=	empty
[16]	translateRule.attributes	::=	att.selector.attributes, attribute translate { "yes" | "no" }

att.translate

[17]	att.translate.attributes	::=	att.translate.attribute.translate
[18]	att.translate.attribute.translate	::=	attribute its:translate { "yes" | "no" }?

6.3.1 Definition

The data category localization information is used to communicate information to localizers about a particular item of content.

This data category has several purposes:

• Tell the translator how to translate parts of the content
• Expand on the meaning or contextual usage of a specific element, such as what a variable refers to or how a string will be used on the user interface
• Clarify ambiguity and show relationships between items sufficiently to allow correct translation (e.g. in many languages it is impossible to translate the word "enabled" in isolation without knowing the gender, number and case of the thing it refers to.)
• Indicate why a piece of text is emphasized (important, sarcastic, etc.)

Two types of informative notes are needed:

• An alert contains information that the translator must read before translating a piece of text. Example: an instruction to the translator to leave parts of the text in the source language.
• A description provides useful background information that the translator will refer to only if they wish. Example: a clarification of ambiguity in the source text.

6.3.2 Implementation

Localization information can be expressed with global rules, or locally on an individual element.

Using global rules, addition of localization information to selected nodes is achieved with a locInfoRule element, which has a mandatory selector attribute and a mandatory locInfoType attribute. The information associated to the selection can be indicated in different ways using either a locInfo element, a locInfoRef attribute, a locInfoPointer attribute, or a locInfoRefPointer attribute. The locInfo element, locInfoRef attribute, locInfoPointer attribute, and locInfoRefPointer MUST NOT occur together.

<its:rules its:version="1.0">
 <its:locInfoRule locInfoType="alert" selector="/body/p[1]">
  <its:locInfo>This p element has to be handled carefully</its:locInfo>
 </its:locInfoRule>
</its:rules>

Pointing to existing localization information is provided by a locInfoPointer attribute or a locInfoRefPointer attribute on the locInfoRule element. These two attributes MUST NOT occur together. The selector attribute is mandatory.

The locInfo element contains the information itself.

<myDoc>
 <head>
  <its:rules its:version="1.0">
   <its:locInfoRule locInfoType="alert" selector="/body/p[1]">
    <its:locInfo>This p element has to be handled carefully.</its:locInfo>
   </its:locInfoRule>
  </its:rules>
 </head>
 <body>
  <p>Paragraph to translate.</p>
 </body>
</myDoc>

The locInfoRef attribute is an URI referring to the location of the information.

<myRes>
 <head>
  <its:rules its:version="1.0">
   <its:locInfoRule
     locInfoType="description"
     selector="/body/msg[@id='FileNotFound']"
     locInfoRef="ErrorsInfo.html/#FileNotFound"/>
  </its:rules>
 </head>
 <body>
  <msg id="FileNotFound">Cannot find the file {0}.</msg>
 </body>
</myRes>

The locInfoPointer attribute is a relative XPath expression pointing to a node that holds the information.

<Res>
 <prolog>
  <its:rules its:version="1.0">
   <its:locInfoRule
     locInfoType="description"
     selector="//msg/data"
     locInfoPointer="../notes"/>
  </its:rules>
 </prolog>
 <body>
  <msg id="FileNotFound">
   <notes>Indicates that the resource file {0} could not be loaded.</notes>
   <data>Cannot find the file {0}.</data>
  </msg>
  <msg id="DivByZero">
   <notes>A division by 0 was going to be computed.</notes>
   <data>Invalid parameter.</data>
  </msg>
 </body>
</Res>

The locInfoRefPointer attribute contains a relative XPath expression pointing to a node that holds the URI referring to the location of the information.

<data>
 <prolog>
  <its:rules its:version="1.0">
   <its:locInfoRule
     locInfoType="description"
     selector="//string"
     locInfoRefPointer="@noteFile"/>
  </its:rules>
 </prolog>
 <body>
  <string id="FileNotFound" noteFile="Comments.html/#FileNotFound">
   <data>Cannot find the file {0}.</data>
  </string>
  <string id="DivByZero" noteFile="Comments.html/#DivByZero">
   <data>Invalid parameter.</data>
  </string>
 </body>
</data>

Locally in a document, localization information is expressed with the attributes locInfo or locInfoRef, and locInfoType. locInfo and locInfoRef MUST NOT occur together. If the locInfoType attribute is not present, the type of localization information will be assumed as description. The selection is the textual content of element, including child elements, but excluding attributes.

<book its:version="1.0">
 <head>...</head>
 <body> ...
   <p
    its:locInfo="This p element has to be handled carefully"
    its:locInfoType="alert">And he said: you need a new
     <quote>motherboard</quote>
  </p> ...
  </body>
</book>

6.3.3 Markup Declarations for Localization Information

locInfoRule

[19]	locInfoRule	::=	element its:locInfoRule { locInfoRule.content, locInfoRule.attributes }
[20]	locInfoRule.content	::=	locInfo?
[21]	locInfoRule.attributes	::=	att.selector.attributes, attribute locInfoPointer { string }?, attribute locInfoType { "alert" | "description" }, attribute locInfoRef { xsd:anyURI }?, attribute locInfoRefPointer { string }?

locInfo

[22]	locInfo	::=	element its:locInfo { locInfo.content, locInfo.attributes }
[23]	locInfo.content	::=	text
[24]	locInfo.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes

att.locInfo

[25]	att.locInfo.attributes	::=	att.locInfo.attribute.locInfo, att.locInfo.attribute.locInfoType, att.locInfo.attribute.locInfoRef
[26]	att.locInfo.attribute.locInfo	::=	attribute its:locInfo { string }?
[27]	att.locInfo.attribute.locInfoType	::=	attribute its:locInfoType { "alert" | "description" }?
[28]	att.locInfo.attribute.locInfoRef	::=	attribute its:locInfoRef { xsd:anyURI }?

6.4.1 Definition

The terminology data category is used to mark terms. This helps to increase consistency across different parts of the documentation. It is also helpful for translation.

6.4.2 Implementation

The terminology data category can be expressed with global rules, or locally on an individual element.

As for global rules, identifying terminology information at selected nodes is realized with a termRule element with a mandatory selector attribute. In addition, an optional termInfoRef attribute can be used to refer to external information about the term. The datatype of termInfoRef is xs:anyURI. To point to a link to a term reference in the current document, a termInfoRefPointer attribute can be used. termInfoRef and termInfoRefPointer MUST NOT be used together.

<text> ...
 <its:rules its:version="1.0">
  <its:termRule selector="//term" termInfoRefPointer="@target"/>
 </its:rules> ...
 <p>We may define <term target="#TDPV">discoursal point of view</term>
 as <gloss xml:id="TDPV">the relationship, expressed through discourse
  structure, between the implied author or some other addresser,
  and the fiction.</gloss>
 </p> ...
</text>

Locally in a document, the terminology data category is expressed with a term attribute, which has the value "yes", and an optional termInfoRef attribute. The selection is the textual content of the element, including content of child elements, but excluding attributes.

<book its:version="1.0">
 <head>...</head>
 <body> ...  <p>And he said: you need a
   new <quote its:term="yes">motherboard</quote>
  </p> ...
  </body>
</book>

6.4.3 Markup Declarations for Terminology

termRule

[29]	termRule	::=	element its:termRule { termRule.content, termRule.attributes }
[30]	termRule.content	::=	empty
[31]	termRule.attributes	::=	att.selector.attributes, attribute termInfoRef { xsd:anyURI }?, attribute termInfoRefPointer { string }?

att.term

[32]	att.term.attributes	::=	att.term.attribute.termInfoRef, att.term.attribute.term
[33]	att.term.attribute.termInfoRef	::=	attribute its:termInfoRef { xsd:anyURI }?
[34]	att.term.attribute.term	::=	attribute its:term { "yes" }?

6.5.1 Definition

This data category expresses the directionality of a piece of text. Its values are "ltr", "rtl", "lro" or "rlo". This definition is compliant with the dir attribute in [XHTML 2.0], except that [XHTML 2.0] does not allow for rule-based, global selection. An implementation of the directionality data category MUST follow the XHTML family user agent conformance critera defined in that specification.

6.5.2 Implementation

The dir attribute is used for the implementation of the directionality data category. It has the four values "ltr", "rtl", "lro" or "rlo".

Directionality can be expressed with global rules or locally on an individual element.

As for global rules, directionality is expressed in rules using a dirRule element with the dir attribute. In addition, a selector attribute is required.

<its:rules its:version="1.0">
 <its:dirRule dir="rtl" selector="/body/p[1]/quote[xml:lang='he']"/>
<!-- Some Hebrew quotation -->

</its:rules>

Locally in a document, directionality is expressed with a dir attribute. The selection is the textual content of the element, including all child elements and attributes.

<book its:version="1.0">
 <head>...</head>
 <body> ...  <p>And he said: <quote its:dir="rtl"> ... a Hebrew quotation ... </quote>
  </p>
  ...  </body>
</book>

6.5.3 Markup Declarations for Directionality

dirRule

[35]	dirRule	::=	element its:dirRule { dirRule.content, dirRule.attributes }
[36]	dirRule.content	::=	empty
[37]	dirRule.attributes	::=	att.selector.attributes, attribute dir { "ltr" | "rtl" | "lro" | "rlo" }

att.dir

[38]	att.dir.attributes	::=	att.dir.attribute.dir
[39]	att.dir.attribute.dir	::=	attribute its:dir { "ltr" | "rtl" | "lro" | "rlo" }?

6.6.1 Definition

The data category ruby is used for a run of text that is associated with another run of text, referred to as the base text. Ruby text is used to provide a short annotation of the associated base text. It is most often used to provide a reading (pronunciation) guide.

6.6.2 Implementation

Ruby can be expressed locally in a document or with global rules.

Locally in a document, Ruby is realized with a ruby element.

<text its:version="1.0">
 <head> ... </head>
 <body>
  <p>This is about the
      <its:ruby>
    <its:rb>W3C</its:rb>
    <its:rt>World Wide Web Consortium</its:rt>
   </its:ruby>
  </p>
 </body>
</text>

The structure of the content model for the ruby element is identical with the structure of ruby markup as defined in [Ruby-TR]. An implementation of the ruby data category MUST follow the conformance critera for ruby defined in that specification.

The functionality of pointing to existing ruby markup is realized with various pointer attributes for ruby. There is a pointer attribute for the ruby element and each element within its content model.

6.6.3 Handling Legacy Content

In legacy situations, where one cannot change the element markup and there one wants to apply ruby text to an attribute or existing element content, then the following approach can be used.

A rubyRule element can be used with two attributes:

• A rubyText attribute contains the ruby text (corresponding to the rt element in the case of no selections)
• A selector attribute contains the selector. It selects the ruby base text, corresponding to the rb element in the case of no selection.

<text>
 <head> ... </head>
 <its:rules its:version="1.0">
  <its:rubyRule rubyText="World Wide Web Consortium" selector="/body/img[1]/@alt"/>
 </its:rules>
 <body>
  <img src="w3c_home.png" alt="W3C"/> ...
  </body>
</text>

6.6.4 Markup Declarations for Ruby

rubyRule

[40]	rubyRule	::=	element its:rubyRule { rubyRule.content, rubyRule.attributes }
[41]	rubyRule.content	::=	empty
[42]	rubyRule.attributes	::=	att.selector.attributes, attribute rubyPointer { string }?, attribute rbPointer { string }?, attribute rtPointer { string }?, attribute rpPointer { string }?, attribute rbcPointer { string }?, attribute rtcPointer { string }?, attribute rubyText { string }?, attribute rbspanPointer { string }?

ruby

[43]	ruby	::=	element its:ruby { ruby.content, ruby.attributes }
[44]	ruby.content	::=	( rb, ( rt | ( rp, rt, rp ) ) ) | ( rbc, rtc, rtc? )
[45]	ruby.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes

rb

[46]	rb	::=	element its:rb { rb.content, rb.attributes }
[47]	rb.content	::=	text
[48]	rb.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes

rt

[49]	rt	::=	element its:rt { rt.content, rt.attributes }
[50]	rt.content	::=	text
[51]	rt.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes, attribute rbspan { string }?

rbc

[52]	rbc	::=	element its:rbc { rbc.content, rbc.attributes }
[53]	rbc.content	::=	rb+
[54]	rbc.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes

rtc

[55]	rtc	::=	element its:rtc { rtc.content, rtc.attributes }
[56]	rtc.content	::=	rt+
[57]	rtc.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes

rp

[58]	rp	::=	element its:rp { rp.content, rp.attributes }
[59]	rp.content	::=	text
[60]	rp.attributes	::=	att.translate.attributes, att.locInfo.attributes, att.term.attributes, att.dir.attributes

6.7.1 Definition

The element langRule is used to express that a given piece of content (selected by the attribute langPointer) is used to express language information as defined by [RFC 3066bis].

<its:rules its:version="1.0">
 <its:langRule selector="//p" langPointer="@mylangattribute"/>
</its:rules>

6.7.2 Markup Declarations for Language Information

langRule

[61]	langRule	::=	element its:langRule { langRule.content, langRule.attributes }
[62]	langRule.content	::=	empty
[63]	langRule.attributes	::=	att.selector.attributes, attribute langPointer { string }

6.8.1 Definition

The data category elements within text expresses information about how elements should affect the flow of the content. In this context the flow of the content represents how the nodes of the elements should be treated as a single unit for linguistic purposes. Sometimes, a flow can be nested within another one. The values associated with this data category are:

• "yes" (the element and its content are part of the flow of its parent element)
• "nested" (the element is part of the flow of its parent element, its content is an independent flow)
• "no" (the element splits the text flow of its parent element and its content is an independent text flow)

Elements not listed are considered to have the value "no".

6.8.2 Implementation

This data category can be expressed only in a set of rules. It cannot be expressed as local markup on an individual element.

Element within text is expressed with a withinTextRule element with a withinText attribute. The attribute has the values "yes", "no" or "nested". In addition, a selector attribute is required.

<its:rules its:version="1.0">
 <its:withinTextRule withinText="yes" selector="//b | //em | //i"/>
</its:rules>

6.8.3 Markup Declarations for Elements within Text

withinTextRule

[64]	withinTextRule	::=	element its:withinTextRule { withinTextRule.content, withinTextRule.attributes }
[65]	withinTextRule.content	::=	empty
[66]	withinTextRule.attributes	::=	att.selector.attributes, attribute withinText { "yes" | "no" | "nested" }