Authoring Techniques for XHTML & HTML Internationalization: Specifying the language of content 1.0

Declaring the text processing language in the html tag sets the default text processing language for the whole document. It can be overridden for portions of the document as required.

You should always declare the text processing language of the page using the lang and/or xml:lang attributes. This is already important for applications such as accessibility and searching, but many other possible applications for this information may develop over time.

The following example declares an HTML document to be in Canadian French:

Note that language declarations in the HTTP header or the Content-Language meta tag should be used to describe the primary language, ie. metadata about the document as a whole, rather than the default text processing language. Most documents have one primary language, but where there are more it may not be necessary to declare a single default text processing language in the html tag. The relevance will depend on the structure used for the document.

For details of which language attribute to use, and how to use language codes use, see section 6 Specifying language.

For a comparison of 'primary language' and 'text processing language', see section 2 Definitions.

The Content-Language declaration, whether it is used in the HTTP header or a meta tag, can be useful for expressing metadata about the primary language(s) of the document being served.

Content-Language information sent in the HTTP header is defined on the server.

The following example shows how the language would be declared in a meta tag inside a document:

Note that this is different from expressing the default language of content for text processing, which must be done using a language attribute on the html tag.

The extent to which metadata information in the HTTP header or a meta tag is used by applications, or which of the two is preferred is not clear at this point.

There are potential issues surrounding use of the HTTP Content-Language header related to the maintenance and use of server-side information. Many authors may find it difficult to access server settings, particularly when dealing with an ISP. Also, pages may not always be located on servers. So this approach is not a solution that is always available.

For a comparison of 'primary language' and 'text processing language', see section 2 Definitions.

There is generally a lot of confusion about the difference between declaring language information using Content-Language in HTTP or meta elements, and using language attributes. In particular, much of the informal advice on the Web about how to declare the language of the document tells you to use the a meta tag to declare the language of the document. At least one popular authoring tool automatically inserts into the meta element language information that you declare in the page properties dialog box. Unfortunately, we have yet to identify any user agent or application that recognizes information declared in this way, whereas language information declared in the html tag is consistently recognized.

Techniques in this document recommend that Content-Language be used for describing primary language metadata, and that attributes be used for describing the default text processing language of the document. (In fact, this question only arises when describing the language of a document at the highest level, since the language of embedded fragments in a document can only be expressed using attributes.)

Because the language attribute can only declare a single language at a time, it is a poor candidate for describing documents with multiple primary languages. Content-Language declarations, however, can declare a list of languages.

On the other hand, Content-Language declarations are not specific enough for declaring the text processing language, and it is strangely inconsistent not to use attributes to declare the default text processing language when they have to be used for embedded text.

In addition, as mentioned above, from a practical perspective the information contained in HTTP Content-Language headers is rarely used by mainstream browsers for identifying the default processing language, and such implementation as there is is inconsistent. The information in the meta tag is typically not recognized at all by current user agents in a processing context.

For a fuller discussion on this topic, see Using HTTP and meta for language information.

For a shorter comparison of 'primary language' and 'text processing language', see section 2 Definitions.

The body tag is the wrong place to express this information because it only refers to a portion of the text in the document. For example, the text in the title element is natural language text that should also inherit the language information. If language is declared in the body element, however, this is not the case.

The html element is the highest level element in the document, and is therefore most appropriate for declaring the default text processing language of the document. All elements within the document will inherit that value.

The HTTP specification provides for more than one language to be expressed as the value of the Content-Language header.

The following example declares a document to have three primary languages: German, French and Italian:

The meta element provides a similar possibility:

Remember that the language attribute on the html tag should be used to declare the default text processing language for the document. Given that only one language can be defined at a time as the text processing language, there may appear to be little point in using an attribute on the html if all primary languages are used in a completely unbiased way in the document.

If, however, the page header information or navigation is in one particular language, or there is a bias of some other kind towards one particular language, you may want to use a language attribute on the html tag.

Note that there is a definite problem when dealing with multilingual title elements. Only one language can be declared for this element in HTML 4.01, since the only content allowed is character data.

[Ed. note: Should we mention the use of 'mul' as a language code? If we do, should we recommend it or not?]

For details of how to use language attributes, see the section 6 Specifying language.

Dividing parallel text at the highest possible level, can simplify the process of guiding users to the text via searching, links, etc. It also reduces the work of labelling the language of document fragments.

For details of how to use language attributes, see the section 6 Specifying language.

Where the language of the text is different from the language declared in the html tag, you should indicate this using the lang or xml:lang attributes. For example, in HTML you would write:

The lang attribute can be used on all HTML elements except applet, base, basefont, br, frame, frameset, iframe, param and script. (Note, by the way, that this means that you could use language attributes on things like bitmaps and audio files that are language specific. Such information may be particularly useful for script-based processing of documents.)

If there is no markup around the text in a different language, use a span element to delimit the boundaries. Here is an example in XHTML 1.0 served as text/html:

For details of which language attribute to use, and how to use language codes use, see section 6 Specifying language.

When serving HTML you should use the lang attribute to declare the language of the document or a range of text. For example, the following declares a document to be in Canadian French:

When serving XHTML as text/html, you should use both the lang attribute and the xml:lang attribute. The xml:lang attribute is the standard way to identify language information in XML. The following shows how you would mark up the previous example for XHTML 1.0 served as text/html.

The xml:lang attribute is not actually useful for handling the file as HTML, but takes over from the lang attribute any time you treat the document as XML for, say, scripting or validation.

If you are serving XHTML 1.0 pages as XML (ie. using a MIME type such as application/xhtml+xml), or serving pages as XHTML 1.1, you do not need the lang attribute, since lang is part of the HTML language. The xml:lang attribute alone will suffice.

RFC 3066 is the IETF document that defines how to use language tags to identify languages. It obsoletes the RFC 1766 referred to by earlier specifications.

RFC 3066 merely expands and clarifies the possibilities for specifying languages. If you have been using RFC 1766 you should not need to make any changes to your code in order to start using RFC 3066.

A proposed successor to RFC 3066 is currently being developed, but it aims to retain backwards compatibility with tags created using RFC 3066.

Note also that you can only specify one language or language variant per element.

For an introduction to the RFC3066 rules for language codes, see Specifying language attribute values in the tutorial Using Language Information in XHTML, HTML and CSS.

RFC3066 specifies that the two letter codes should be used where available, since this aids interoperability by ensuring that a single code is used everywhere to refer to a particular language.

This also avoids the question of which 3-letter code to use for those languages that have two 3-letter codes, since all such languages have a 2-letter code also.

RFC3066 specifies how to identify a language. Simplified vs. Traditional Chinese is a distinction based on script. In the past zh-CN (Chinese spoken in Mainland China) was commonly used to label Simplified Chinese, and zh-TW (Chinese spoken in Taiwan) was commonly used for Traditional Chinese. Apart from the fact that this is mislabelled, you could not guarantee that others would recognize these conventions, or even follow them. For example, some people used zh-HK to represent Traditional Chinese.

Now the IANA registry makes available the codes zh-Hans and zh-Hant for Simplified and Traditional Chinese, respectively. The following two examples illustrate the use of these tags.

It is expected that these tags will persist for the foreseeable future, so on the one hand it would be good to use them as soon as possible in order to improve interoperability sooner rather than later.

On the other hand, you need to assess the impact of changing the tags. This is not really an issue for self-describing usage, such as with :lang for application of language-based styling. It may be more of an issue where external applications are looking for tags related to Chinese but are unaware of the zh-Hans and zh-Hant variants.

There is one particular area where this may be an issue for the display of text on a user agent. Some (but not all) user agents use language information to automatically choose a font for CJK ideographic text (see the test page). Note that this assumes that (a) you have appropriate fonts set in your preferences, that (b) the document styling does not apply a font, and that (c) the user agent supports this behaviour (not all do). The following table summarizes support for this feature in baseline user agents:

IE 6.0	Does not recognise either of these codes.
Firefox 1.0PR	Handles both codes correctly.
Mozilla 1.7.2	Recognizes the tags but treats them both as Simplified Chinese.
Netscape 7.0	Recognizes the tags but treats them both as Simplified Chinese.
Opera 7.54	Doesn't automatically apply fonts in this fashion, so is irrelevant.

The hreflang attribute on an a element indicates the language of the document at the other end of the link. In practise, hreflang is typically not used by mainstream browsers. Besides that it is much better to ensure that the target document uses the language attribute in the html tag, so that this information is not needed.

A common alternative use for this attribute is to generate a visible marker attached to link text that indicates the language of the destination page for the reader. The idea is to allow the reader to decide in advance whether or not to follow the link, according to their language skill. If the user has to waste time following the link to find out that they cannot read the target document this introduces fatigue, and lack of confidence when faced with links that do go to readable pages.

The approach relies on CSS selectors that detect the value of the hreflang attribute and use the CSS content property to display an indicator of the language.

For example, the following link points to a page in Swedish.

The code to enable this in CSS may be something like:

The markup would read as follows:

This says: for each a element with an hreflang attribute, add the value of that attribute in square parentheses after the link.

There are, however, potential problems with this approach.

Firstly, not all user agents support the CSS required to enable it (see the test). Internet Explorer does not support :after.

Secondly, a newly translated version may become available. Assume, for example that a French page has used this approach some time ago to point to the Apache documentation, which at that time was only in English. Later, the Apache documentation is translated into French and language negotiation is put in place. Unless the French page referred to earlier is updated, it will now be incorrectly warning French readers that the Apache documentation is in English, and possibly discouraging them from following a link to what is actually a perfectly legible document.

Thirdly, if a resource is available in multiple languages (say you are linking from an English overview to detailed descriptions that are available in multiple languages) it is not possible to express that, since the hreflang attribute accepts only a single language as its value.

Flags represent countries, not languages. There are many countries that use the same language, and numerous countries that have more than one official language.

A much better approach is to generate some text. In the previous technique, an example is shown that uses the actual attribute value, since these two-letter codes are typically recognisable by speakers of the language.