$Id: html-tech.html,v 1.23 2003/10/07 16:13:19 rishida Exp $

XHTML & HTML Internationalization Techniques Repository 1.0

W3C Working Draft dd mmmm 2003

This version:
http://www.w3.org/International/geo/html-tech/html-tech.html
Editor:
Richard Ishida, W3C <ishida@w3.org>

Abstract

This document is a database of techniques for developing internationalized HTML using XHTML 1.0 or HTML 4.01. Its content is reused by other documents that are more focused on a specific audience.

Status of this Document

This document is an editors' copy that has no official standing.

This section describes the status of this document at the time of its publication. Other documents may supersede this document. The latest status of this series of documents is maintained at the W3C.

This is a very early working draft. It is undergoing constant and frequent modification.

This document is published as part of the W3C Internationalization Activity by the Internationalization Working Group, with the help of the Internationalization Interest Group. The Internationalization Working Group will not allow early implementation to constrain its ability to make changes to this specification prior to final release. Publication as a Working Draft does not imply endorsement by the W3C Membership. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress". A list of current W3C Recommendations and other technical documents can be found at http://www.w3.org/TR/.

Table of Contents

1 Document structure & metadata
2 Character sets, character encodings and entities
3 Fonts
4 Specifying the language of content
5 Text direction
6 Text markup
7 Lists
8 Tables
9 Links
10 Objects
11 Images
12 Multimedia
13 Forms
14 Keyboard shortcuts
15 Writing source text
16 Handling data that varies by locale
17 Supplying data for localization
18 Navigation

Appendix

A References


1 Document structure & metadata

2 Character sets, character encodings and entities

Prereading: Draw out the distinction between the document character set (always Unicode) and the document encoding.

Use a Unicode encoding

Choose UTF-8 or another Unicode encoding for all content.

When selecting a page encoding, consider both current and future localization requirements, and the benefits of using the same encoding across all pages and all languages. These considerations make the use of Unicode an attractive choice for the following reasons:

  • Unicode supports many languages, enabling the use of a single encoding across all pages and forms, regardless of language.

  • Unicode allows many more languages to be mixed on a single page than almost any other choice. If the set of languages to be represented on a single page cannot be represented directly by any single native encoding (such as ISO-8859-1, Shift-JIS, etc.), then Unicode is almost certainly the best choice.[Ed. note: How is this different from the previous point?]

  • For dynamically-generated pages, a single encoding for all pages eliminates the need for server-side logic to determine the character encoding for each page served.

  • For interactive applications using forms, a single encoding eliminates the need for server-side logic to determine the character encoding of incoming form data.

  • Unicode enables a form in one language (e.g. English) to accept input in a different language (e.g. Chinese).

  • Unicode (UTF-8) forms will be easier to migrate to XForms.[Ed. note: We should add some justification for this.]

UTF-8 and UTF-16 are both Unicode encodings. Since support for Unicode is currently limited to UTF-8 in many user agents, UTF-8 is usually the appropriate Unicode encoding. However, as user agent support for UTF-16 expands, UTF-16 will become an increasingly viable alternative.

Although there are other multi-script encodings (such as ISO-2022 and GB18030), Unicode generally provides the best combination of user agent and script support.

Resources:
implementation[Unicode] The Unicode Standard 4.0
The Unicode Standard is very readable and contains a large amount of useful information besides code point listings.
 IE(Win)  NNav  Opera 
Choosing a non-Unicode encoding

If you don't use a Unicode encoding, select an encoding that best supports the languages / characters to be included in the page text. [Ed. note: What does this mean? Does it mean, which maximizes the opportunity to directly represent characters and minimizes the need to represent characters by markup means such as character escapes? Does it include the idea that you should choose the most commonly used encoding for a region?]

There are some situations where selecting a Unicode encoding is not practical. If content is encoded in a native encoding (legacy content or content originating from an external source) and the system lacks functionality for converting content between encodings, Unicode may greatly complicate implementation. If such a site is only required to serve single-script pages (containing languages that can be represented by a single native encoding), then the cost of using a Unicode encoding may outweigh the benefits. In this case, a native encoding (such as ISO-8859-1, Shift-JIS, etc.) may be a better choice.

Be sure to select an encoding that covers most [Ed. note: all? ]of the characters required for the content, and (if it is a form) all of the characters that must be accepted as input.

Resources:
other Alan Wood’s Unicode Resources
Various resources about Unicode and multilingual support in HTML, fonts, web browsers and other applications.
 IE(Win)  NNav  Opera 
Use the XML declaration where practical for HTML.

For XHTML served as text/html, where practical use an XML declaration with an encoding attribute.

To do this, use an XML declaration as shown below at the top of the file, and assign an IANA charset name to the encoding label of the declaration.

Example:

<?xml version="1.0" encoding="UTF-8"?>

The checklist item above uses the phrase 'where practical' because authors serving XHTML as text/html often choose not to include the XML declaration. This is because the declaration can cause display problems for some HTML browsers. For example, anything that appears before the DOCTYPE declaration forces Internet Explorer browsers, including version 6, into 'quirks' mode rather than 'standards' mode.

Because an XHTML document served as text/html is actually handled as HTML, the XML declaration is not actually required when the document is served. Note, however, that the XHTML specification recommends the use of both XML declaration and the meta charset declaration when XHTML is served as text/html.

In theory it is not necessary to specify the character encoding in an XML declaration for documents encoded in UTF-8 and UTF-16, since the XML parser treats these as the default. In practise, however, it is a good idea to label the document explicitly (as shown in the example above). For example, developers, testers, or translation production managers may want to perform a visual check of a document, or process the document using tools other than XML parsers.

You should declare the encoding inside the document even if the HTTP Content-Type parameter has been sent by the server. This ensures that the character encoding is always declared, even if the document is at some point not read from that server (eg. a local copy is read from disk, or the file is moved to another server that is not set up to serve the Content-Type parameter).

According to the XHTML 1.0 specification, in XHTML-conforming user agents, the value of the encoding declaration of the XML declaration takes precedence over the meta charset statement. It has a lower priority, however, than the HTTP Content-Type parameter.

Resources:
source[XHTML 1.0] 3.1.1. Strictly Conforming Documents (towards the bottom of the section)
General requirements for specification of encoding in XHTML documents.
source[XHTML 1.0] C.9 Character encoding
How to specify character encoding for XHTML served as text/html using compatibility markup.
 IE(Win)  NNav  Opera 
Use the XML declaration to specify charset in XML.

For XHTML served as application/xhtml+xml, always use an XML declaration with an encoding attribute.

To do this, use an XML declaration as shown below at the top of the file, and assign an IANA charset name to the encoding label of the declaration.

Example:

<?xml version="1.0" encoding="UTF-8"?>

If you are serving XHTML as application/xhtml+xml, the encoding attribute is mandatory unless you are using UTF-8 or UTF-16 or declaring the encoding in the HTTP header.

In theory it is not necessary to specify the character encoding in an XML declaration for documents encoded in UTF-8 and UTF-16, since the XML parser treats these as the default. In practise, however, it is a good idea to label the document explicitly (as shown in the example above). For example, developers, testers, or translation production managers may want to perform a visual check of a document, or process the document using tools other than XML parsers.

You should declare the encoding inside the document even if the HTTP Content-Type parameter has been sent by the server. This ensures that the character encoding is always declared, even if the document is at some point not read from that server (eg. a local copy is read from disk, or the file is moved to another server that is not set up to serve the Content-Type parameter).

According to the XHTML 1.0 specification, in XHTML-conforming user agents, the value of the encoding declaration of the XML declaration takes precedence over the meta charset statement. It has a lower priority, however, than the HTTP Content-Type parameter.

The meta charset declaration is not needed when XHTML is served as application/xhtml+xml.

Resources:
source[XHTML 1.0] C.9 Character encoding
XHTML 1.0 spec (2nd Edition)
 IE(Win)  NNav  Opera 
Use the meta statement to specify HTML encoding.

For HTML documents and XHTML documents served as text/html, always use the meta element to explicitly declare the document's character encoding.

To do this, assign an IANA charset name as the charset value of a meta http-equiv statement.

Example:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xml:lang="en" lang="en">

<head>

<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>

<title>Sample document</title>

...

This is good practise even if the character encoding has already been specified in the HTTP Content-Type parameter or any XML declaration (in XHTML). It ensures that the character encoding is always declared even if the document is at some point not read from that server (eg. a local copy is read from disk, or the file is moved to another server that is not set up to serve the Content-Type parameter).

Note that the XHTML specification recommends that the character encoding be declared in both the meta charset declaration and the XML declaration.

Note also that you should include a character encoding declaration even if your document uses a basic Latin encoding such as ISO 8859-1. For example, Japanese user agents will default to a Japanese encoding that does not include the accented letters, so they may not see you text correctly unless you specified the encoding.

In case of conflict, the Content-Type charset declaration and the XML declaration have precedence over the meta charset statement, according to the HTML 4.01 and XHTML 1.0 specifications. [Ed. note: Is this true in practise? esp wrt IE?]

Resources:
source[XHTML 1.0] C.9 Character encoding
XHTML 1.0 spec (2nd Edition)
 IE(Win)  NNav  Opera 

3 Fonts

Don't use <font>

Do not use <font> tags - use CSS styles instead.

Note that <font> and <basefont> tags are deprecated in the HTML4.01 Recommendation.

Easier maintenance

Faster translation AND localization.

[Ed. note: Describe the evils of using <font> to cheat on the charset and represent other scripts.]

 IE(Win)  NNav  Opera 
Use font fallbacks

Always use the serif and sans-serif fallbacks

 IE(Win)  NNav  Opera 
Don't assume you know what fonts will be available

Don't assume you know which fonts will be available on the client.

Also don't assume that the font you've chosen will contain the characters needed for localized pages

 IE(Win)  NNav  Opera 
Don't rely on text just fitting in a space

Don't rely on text just fitting in a space

tbd

 IE(Win)  NNav  Opera 
Font size

Something about font size??

 IE(Win)  NNav  Opera 
Font coverage

don't assume that all versions of a font cover the same characters

 IE(Win)  NNav  Opera 
Fonts

Some guidelines for content authors who know that users won't have all the necessary fonts.

tbd

 IE(Win)  NNav  Opera 

4 Specifying the language of content

5 Text direction

Don't add dir to the body tag

Do not add dir="rtl" to the body tag.

Although the HTML specification recommends the use of the dir attribute on the html element, this guideline is motivated more by practical considerations relating to user agent behavior.

According to the Microsoft article Authoring HTML for Middle Eastern Content, the following behaviors can only be expected in Internet Explorer 5 if the dir attribute is on the html element, rather than the body element.

  • The OLE/COM ambient property of the document is set to AMBIENT_RIGHTTOLEFT

  • The document direction can be toggled through the document object model (DOM) (document.direction="ltr/rtl")

  • An HTML Dialog will get the correct extended windows styles set so it displays as a RTL dialog on a Bidi enabled system.

  • If the document has vertical scrollbars, they will be used on the left side if dir="rtl".

[Ed. note: check whether similar things apply to other user agents]

Resources:
other Authoring HTML for Middle Eastern Content
Explains why you should add dir to the html element rather than the body element.
 IE(Win)  NNav  Opera 
Don't use visual ordering

Use logical order, not visual ordering for Hebrew.

'Visual ordering' of text was common for old user agents that didn't support the Unicode bidirectional algorithm. Text was stored in the source code in the same order you would expect to see it displayed. This also involved such things as disabling any line wrapping, explicit right-alignment of text in paragraphs and table cells, and reverse-ordering of table columns when translating from English to a language using a bidi script. The result is very fragile code that is difficult to maintain. For example, if you want to add a few words in the middle of a paragraph, you would have to move text to and from every line that followed it in the paragraph.

Visually ordered bidirectional HTML does not conform to the HTML specification.

With 'logical ordering' text is stored in memory in the order in which it would normally be typed (and usually pronounced). The Unicode bidirectional algorithm is then applied by the browser to render the correct visual display.

Visual ordering isn't really seen much for Arabic. Since the Arabic letters are all joined up there was a stronger motivation on the part of Arabic implementers to enable the logical ordering approach.

Resources:
background What you need to know about the bidi algorithm and inline markup, Visual vs. logical order
Provides examples and explanations of visual versus logical order for pages in bidirectional scripts.
 IE(Win)  NNav  Opera 
Choose an appropriate Hebrew ISO encoding

If using an ISO character encoding for Hebrew, choose iso-8859-8-i and use logical ordering.

It is usually best to use an Unicode encoding, such as UTF-8. This technique applies if, for some reason, you choose to serve your Hebrew page in an ISO encoding instead.

According to RFC1555 and RFC1556, there are special conventions for the use of charset parameter values to indicate bidirectional treatment in MIME mail, in particular to distinguish between visual, implicit, and explicit directionality. 'Visual' refers to the practise of typing in the Hebrew characters in reverse order and preventing automatic line breaks. Formatting the document visually in this way is typically done to ensure reasonable display on older user agents that do not handle bidirectionality. Such documents do not conform to the HTML specification. 'Implicit' is also called logical ordering, and refers to an approach where all characters in memory in the order in which it would normally be typed. Correct ordering for display is then done by a special algorithm (this is the preferred approach). 'Explicit' refers to the use of explicit markers in the text to indicate directional changes.

The charset parameter value ISO-8859-8 for Hebrew denotes visual ordering, ISO-8859-8-i denotes implicit bidirectionality, and ISO-8859-8-e denotes explicit directionality.

Because HTML uses the Unicode bidirectional algorithm, conforming documents encoded using ISO 8859-8 must be labeled as ISO-8859-8-i. Explicit directional control is also possible with HTML, but cannot be expressed with ISO 8859-8, so "ISO-8859-8-e" should not be used.

Contrary to what is said in RFC1555 and RFC1556, ISO-8859-6 (Arabic) is not visual ordering.

 IE(Win)  NNav  Opera 
Do not use CSS styling

Do not use CSS styling to control directionality in XHTML/HTML. Use markup.

Because directionality is an integral part of the document structure, markup should always be used to set the directionality for a document or chunk of information, or to indicate places in the text where the Unicode bidi algorithm is insufficient to achieve desired directionality.

The CSS2 specification recommends the use of markup for bidi text in HTML. In fact it goes as far as to say that conforming HTML user agents may ignore CSS bidi properties. This is because the HTML specification clearly defines the expected behavior of user agents with respect to the bidi markup.

See CSS vs. markup for bidi support for a fuller explanation.

 IE(Win)  NNav  Opera 
Use bidi markup only when necessary.

Only use bidi markup where it is needed.

Once you have established the appropriate directionality for the html element you will only need to apply bidi markup to a block element if you want that element's directionality to be different. The same applies for inline markup. Do not use inline bidi markup unless the Unicode bidi algorithm is insufficient on its own.

The following Arabic example shows bad usage. None of the dir attributes are needed if dir="rtl" was added to the html element. Removing them will significantly simplify the document, and reduce bandwidth - which may be an important consideration in countries where Arabic is spoken.

Example:

Bad practise. Do not copy!

<h2 dir="rtl">القاموس</h2>

<dl>

<dt dir="rtl">المنالية</dt>

<dd dir="rtl">سهولة منال للويب من قبل الجميع بصرف النّظر عن إعاقةهم . </dd>

<dt dir="rtl">برنامج التصديق</dt>

<dd dir="rtl">

أو "الفاليديتور" أداة للتّحقّق من صلاحيّة صفحة ويب. على سبيل المثال، للتّحقّق من صلاحيّة

<span dir="ltr">HTML</span> ، يمكن أن تستخدم بزنامج تصديق

<span dir="ltr">W3C</span>

</dd>

<dt dir="rtl">التّدويل</dt>

<dd dir="rtl">

تدويل الويب يسمح و يجعله سهل لاستخدام موقعك باللّغات و السّيناريوهات و الثّقافات المختلفة.

</dd>

</dl>

The Unicode Bidirectional Algorithm is applied to text that is stored in logical order, and determines the appropriate display direction of a sequence of characters. It does this on the basis of semantics associated with those characters by the Unicode Standard.

Example:

The following Arabic text contains the number 1996 that runs left to right within the overall right to left flow of the Arabic letters. No special markup or styling is needed to achieve this. The bidirectional algorithm alone is enough.

بدأ تطوير إكس إم إل في 1996 و صارت...

Occasionally the Unicode bidirectional algorithm is not sufficient to correctly order chunks of embedded text. Alternatively, you may want to override the effects of the bidirectional algorithm for a part of the page. In these cases you can apply additional markup to produce the ordering you want.

Resources:
background What you need to know about the bidi algorithm and inline markup, Visual vs. logical order
Provides examples and explanations of visual versus logical order for pages in bidirectional scripts.
 IE(Win)  NNav  Opera 
Use the dir attribute on block elements.

Add the dir attribute to a block level element (only) to change its directionality.

The following example illustrates the effect of applying a change in directionality to a block level element using the dir attribute.

Example:

The following paragraph inherits the LTR directionality of this page, and its source contains some Hebrew text, followed by punctuation, followed by a graphic.

להוביל את הרשת למיצוי הפוטנציאל שלה…  Small picture of a globe.

The following is exactly the same code, but with an explicit dir="rtl" added to the paragraph tag to turn this into a right-to-left paragraph embedded in this left-to-right page.

להוביל את הרשת למיצוי הפוטנציאל שלה…  Small picture of a globe.

Note, in particular, that the positions of the image and punctuation in the example above change relative to the text, because the overall directional flow has been changed. Note also, however, that the Hebrew characters are still read in the same direction. Their sequence is determined by the Unicode bidirectional algorithm, not by the dir attribute.

The content of all nested block elements will inherit directionality (unless of course a nested element explicitly changes its directionality using dir). Remember that the base directionality for a document should already be established by the html element. There is no need to add dir attributes to block level elements unless you want to apply a different direction to that set by the html tag or an explicit setting on a parent block element.

Visual user agents that support bidirectional display will typically right-align block elements in a rtl context, and vice versa. (See the example above.)

The dir attribute setting also affects the flow of columns in a table.

Example:

The following table element has a dir attribute set to rtl.

123
مكتب W3C הישראליمكتب W3C הישראליمكتب W3C הישראלי

Here is the same table element with the dir attribute removed. The directionality of the columns is now set by the next ancestor element that specifies directionality - in this case the default ltr setting of the html tag of this document.

123
مكتب W3C הישראליمكتب W3C הישראליمكتب W3C הישראלי

Note how the cells inherit the directionality set for the table. This produces the alignment of text in the cell, the order of text relative to the number, and the position of the question mark.

Note also that in most browsers, unlike other block elements, adding a dir attribute to the table will not cause the table to be aligned differently. It will only affect the order of columns and table content. If you want the table to be aligned with the other side of the content area you will need to wrap the table in another block element (eg. a div) that carries a dir attribute.[Ed. note: Check that this applies for Mac browsers.]

 IE(Win)  NNav  Opera 
Use RLM and LRM to place neutral characters

Use a Unicode right-to-left mark (RLM) or left-to-right mark (LRM) to make neutral characters such as punctuation and spaces appear in the right place when they fall between different directional runs.

You need to be familiar with the concepts in What you need to know about the bidi algorithm and inline markup to understand this technique.

Unfortunately, the bidirectional algorithm may not always produce the desired result with regard to the placement of punctuation. For instance, the overall context of the example below is LTR. If we introduce some punctuation between the Arabic and Latin letters it will produce the following (undesirable) result.

Example:

The title is "مفتاح معايير الويب!" in Arabic.

The exclamation mark is part of the Arabic phrase and should have appeared to its left. It appears to the right because it is between an Arabic and Latin character and the overall paragraph direction is LTR. It is therefore treated as part of the English text.

An easy way to fix this is to insert the Unicode character U+200F, called the RIGHT-TO-LEFT MARK, after the exclamation mark. There is a similar character, U+200E, called the LEFT-TO-RIGHT MARK.

The best way to represent these characters is with the pre-defined HTML character entities, &rlm; and &lrm;.

Now with two strong RTL characters on either side, the exclamation mark too will be treated as part of the RTL directional run and we will get the following (correct) result.

Example:

The title is "مفتاح معايير الويب!‏" in Arabic.

Note that it is possible to use actual Unicode characters or Numeric Character References (ie. &#x200E; and &#x200F;) rather than the character entities mentioned above. The character entity is recommended because it provides maximum clarity in the code. A character code would not be visible, and a numeric value may be easily mistaken.

[Ed. note: Actually that's not quite true. It looks fine in a LTR paragraph (see above), but not in a RTL context, where the entity name falls foul of the same problem! see below. You may be able to avoid this in some cases by breaking the line - as long as this doesn't introduce unwanted spaces.]

Example:

مشس هخصث خهس title in english!&lrm; تخت تخهثز.

مشس هخصث خهس title in english!&#x200E; تخت تخهثز.

 IE(Win)  NNav  Opera 
Use RLM and LRM to resolve same script ordering

Use a Unicode right-to-left mark (RLM) or left-to-right mark (LRM) to correctly order separate runs of same direction text separated by neutral characters such as punctuation and spaces.

You need to be familiar with the concepts in What you need to know about the bidi algorithm and inline markup to understand this technique.

The Unicode characters RLM (right-to-left mark) and LRM (left-to-right mark) can be useful to achieve the correct ordering of text items that are only separated by directionally neutral characters. We will show two examples of this.

In our first example, below, the list order is incorrect because the first two Arabic words should be reversed and the intervening comma, which is part of the English text, should appear immediately to the right of the first word. The reason for the failure is that, with a strongly typed right-to-left (RTL) character on either side, the bidirectional algorithm sees the neutral comma as part of the Arabic text.

Example:

Incorrect:

The names of these states in Arabic are مصر, البحرين and الكويت respectively.

Corrected:

The names of these states in Arabic are مصر,‎ البحرين and الكويت, respectively.

The correct result was obtained by simply placing a &lrm; entity immediately after the comma. This has the effect of placing the neutral comma between two strongly typed characters, one left-to-right and the other right-to-left. Because neutral characters in this position take on the directionality of the overall context (here the paragraph), the bidi algorithm will now see it as part of the English left-to-right flow and will see the two Arabic words as separate.

In the second example, this time in a RTL Hebrew paragraph, the beginning of the sentence looks a real mess. This is because the text from "W3C" to "Consortium" is seen as a single directional run of LTR characters. (The second parenthesis from the right falls between LTR and RTL characters, so assumes the directionality of the paragraph - RTL.)

Example:

Incorrect:

W3C - (World Wide Web Consortium) מעביר את שירותי הארחה באירופה ל - ERCIM.

Correct:

W3C -‏ (World Wide Web Consortium) מעביר את שירותי הארחה באירופה ל - ERCIM.

It is very simple to obtain the correct result. Simply put a &rlm; entity immediately after the hyphen. This causes the hyphen and the nearby parenthesis to be seen as part of the paragraph's text flow.

Note that the dir attribute is not appropriate to resolve this case.

 IE(Win)  NNav  Opera 
Use the dir attribute.

Use the dir attribute on an inline element to resolve problems with nested directional runs.

At a simple level the Unicode bidirectional algorithm takes care of the reordering of inline text, but where there is nesting of directionality the dir attribute may need to be used.

The Unicode bidirectional algorithm organizes characters into directional runs - sequences of characters with the same directionality. Directionally neutral characters such as spaces and punctuation take on the directionality of surrounding characters, allowing directional runs to span several words. In the example below there are three directional runs - English, Arabic, and English. These are ordered according to the prevailing directionality of the paragraph - in this case left-to-right.

Example:

The title is مفتاح معايير الويب in Arabic.

Unfortunately, the bidirectional algorithm alone does not produce the desired result if one of the directional runs contains mixed direction text, as can be seen in the following example.

The incorrect line of text is coded as a simple sequence of characters without any inline markup. Note that the order of the two Hebrew words is correct, but the text "W3C" should appear on the left hand side of the quotation and the comma should appear between the Hebrew text and "W3C".

Example:

Incorrect:

The title says "פעילות הבינאום, W3C" in Hebrew.

Correct:

The title says "פעילות הבינאום, W3C" in Hebrew.

To get the correct result we have to create a new 'embedding level' by surrounding the text within the quote marks with a span element and setting its dir attribute to rtl as shown here. (The language information has been omitted to make the example clearer.)

Example:

<p>The title says "<span dir="rtl">פעילות הבינאום, W3C</span>" in Hebrew.</p>

This causes the comma to take on the same RLT directionality as the whole span, and orders the Hebrew directional runs appropriately.

Note that we have used a span element to carry the dir attribute in this case. If the quote had already been surrounded by an element, the dir attribute should be attached to that. A span element should only be used where there is nothing else available.

Note also that we placed the span element inside the quotation marks, since these are a part of the English text.

[Ed. note: Note that it may make sense to use markup rather than control codes, but it certainly doesn't make editing any easier unless the editing tool understands the markup you are applying and reorders the text appropriately. ]

 IE(Win)  NNav  Opera 
Use Unicode control characters for PCDATA

For attribute text or element text that allows no internal markup, use Unicode control characters for bidirectional control.

@@ make sure to refer to the title element

 IE(Win)  NNav  Opera 
Use markup rather than Unicode control characters.

Do not use Unicode control characters for bidirectional control if markup is available.

There are a number of control characters in Unicode that can be used to create the same effect as markup for bidirectional text. These are:

  • U+202A LEFT-TO-RIGHT EMBEDDING

  • U+202B RIGHT-TO-LEFT EMBEDDING

  • U+202D LEFT-TO-RIGHT OVERRIDE

  • U+202E RIGHT-TO-LEFT OVERRIDE

  • U+202C POP DIRECTIONAL FORMATTING

Both Unicode in Markup Languages and the HTML 4.01 specification advise against using these when markup is available, and they particularly advise against mixing control codes and markup.

[Ed. note: The references below need checking (esp for surviving ref to CSS)]

Resources:
source[UXML] 3.4 Bidi Embedding Controls (LRE, RLE, LRO, RLO, PDF), U+202A .. U+202E
Use HTML markup rather than Unicode control characters for directional control.
source[HTML 4.01] 8.2.3 Setting the direction of embedded text
Use HTML markup rather than Unicode control characters for directional control.
source[CSS2] 9.10 Text direction: the 'direction' and 'unicode-bidi' properties
Use HTML markup rather than CSS styles for directional control.
 IE(Win)  NNav  Opera 
Watch out for white space

Do not leave white space at the end of inline elements that mark a directional boundary.

[Ed. note: Summarise and point to the bidi space Q&A]

Resources:
source Bidi space loss
W3C Internationalization FAQ: Why does my browser collapse spaces between Latin and Arabic/Hebrew text?
 IE(Win)  NNav  Opera 
Treatment of mirrored characters

Treat mirrored characters as if any word left in the name meant 'opening', and right meant 'closing'.

The shape of the glyphs used for a pair of mirrored characters will be determined at run time according to the directional context in which they appear.

 IE(Win)  NNav 
Use the bdo element.

Use the bdo element to force the directionality of a sequence of inline characters.

bdo stands for 'bidirectional override'. This inline element can be used to override the Unicode bidirectional algorithm if the dir attribute doesn't produce the desired result or if you want to produce a different result.

Example:

Illustrations of the characters as stored in memory in earlier examples are produced by simply applying a bdo tag. This causes the characters to flow left to right, regardless of the directionality of the characters involved. For instance, an example showing how text is stored in the computer's memory such as

The title says "פעילות הבינאום" in Hebrew.

can be produced using the following underlying code

<p><bdo dir="ltr">The title says "פעילות הבינאום" in Hebrew.</bdo></p>

Without the bdo tag, the Unicode bidirectional algorithm would have produced the following result. Note how the characters in the Hebrew words run in a different direction.

The title says "פעילות הבינאום" in Hebrew.

 IE(Win)  NNav  Opera 
Use lrm and rlm control characters.

Use the special entities, &lrm; and &rlm; to force directionality of directionally neutral characters.

These represent two special characters in Unicode that can be used after the neutral character whose directionality is ambiguous. Problems typically arise for punctuation that falls between characters in a bidirectional script and characters in a non-bidirectional script. The entities are Unicode characters that are strongly typed, so they help disambiguate the context for the Unicode bidirectional algorithm.

Example:

In the following sentence, despite the use of the dir attribute, the commas between the English text that are part of the Hebrew right to left flow have become confused. This is because they are surrounded by Latin text, and the Unicode bidirectional algorithm assumes that they are part of the Latin text flow that goes from left to right.

פעילות הבינאום, W3C, W3C, W3C, פעילות הבינאום, W3C

[Ed. note: [need a proper example]]

This can be easily remedied by adding a &rlm; entity immediately after the commas, as shown here

פעילות הבינאום, W3C,‏‏ W3C,‏‏ W3C,‏‏ פעילות הבינאום, W3C

The code that produced this result is

<p dir="rtl">פעילות הבינאום, W3C,&rlm; W3C,&rlm; W3C,&rlm; פעילות הבינאום, W3C</p>

 IE(Win)  NNav  Opera 

6 Text markup

7 Lists

8 Tables

9 Links

10 Objects

11 Images

12 Multimedia

13 Forms

14 Keyboard shortcuts

15 Writing source text

[Ed. note: Move this whole section to a Core techniques doc?]

16 Handling data that varies by locale

17 Supplying data for localization

18 Navigation

[Ed. note: Move this whole section to a Core techniques doc?]

A References

URL
Nobody, This is used for single web pages
TBD
Nobody, This is a dummy place-holder entry
CharMod
M. J. Dürst, F. Yergeau, R. Ishida, M. Wolf, T. Texin, Character Model for the World Wide Web 1.0, Working Draft in Last Call . (See http://www.w3.org/TR/charmod/.)
CSS2
Bert Bos, Håkon Wium Lie, Chris Lilley, Ian Jacobs, Eds., Cascading Style Sheets, level 2 (CSS2 Specification), W3C Recommendation. (See http://www.w3.org/TR/REC-CSS2.)
HTML 4.01
Dave Raggett, Arnaud Le Hors, Ian Jacobs, Eds., HTML 4.01 Specification, W3C Recommendation. (See http://www.w3.org/TR/html401.)
IANA
Internet Assigned Numbers Authority, Official Names for Character Sets. (See http://www.iana.org/assignments/character-sets.)
RFC1555
H. Nussbacher and Y. Bourvine, Hebrew Character Encoding for Internet Messages, December 1993. (See http://www.ietf.org/rfc/rfc1555.txt
RFC1556
H. Nussbacher, Handling of Bi-directional Texts in MIME, December 1993. (See http://www.ietf.org/rfc/rfc1556.txt
RFC2616
R. Fielding et al., Hypertext Transfer Protocol -- HTTP/1.1, June 1999. (See http://www.ietf.org/rfc/rfc2616.txt
Unicode
The Unicode Consortium, The Unicode Standard, Version 3, ISBN 0-201-61633-5, as updated from time to time by the publication of new versions. (See http://www.unicode.org/unicode/standard/versions for the latest version and additional information on versions of the standard and of the Unicode Character Database).
UXML
Martin Dürst and Asmus Freytag, Unicode in XML and other Markup Languages, Unicode Technical Report #20 and W3C Note. (See http://www.w3.org/TR/unicode-xml.)
XHTML 1.0
W3C HTML Working Group, XHTML™ 1.0 The Extensible HyperText Markup Language (Second Edition), W3C Recommendation. (See http://www.w3.org/TR/xhtml1/.)
ISO/IEC 10646
ISO/IEC 10646-1:2000, Information technology -- Universal Multiple-Octet Coded Character Set (UCS) -- Part 1: Architecture and Basic Multilingual Plane and ISO/IEC 10646-2:2001, Information technology -- Universal Multiple-Octet Coded Character Set (UCS) -- Part 2: Supplementary Planes, as, from time to time, amended, replaced by a new edition or expanded by the addition of new parts. (See http://www.iso.ch for the latest version.)
ISO/IEC 10646-1:2000
ISO/IEC 10646-1:2000, Information technology -- Universal Multiple-Octet Coded Character Set (UCS) -- Part 1: Architecture and Basic Multilingual Plane. (See http://www.iso.ch/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=29819.)
ISO/IEC 10646-2:2001
ISO/IEC 10646-2:2001, Information technology -- Universal Multiple-Octet Coded Character Set (UCS) -- Part 2: Supplementary Planes. (See http://www.iso.ch/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=33208.)
ISO/IEC 646
ISO/IEC 646:1991, Information technology -- ISO 7-bit coded character set for information interchange. This standard defines an International Reference Version (IRV) which corresponds exactly to what is widely known as ASCII or US-ASCII. ISO/IEC 646 was based on the earlier standard ECMA-6. ECMA has maintained its standard up to date with respect to ISO/IEC 646 and makes an electronic copy available at http://www.ecma.ch/ecma1/STAND/ECMA-006.HTM
MIME
Multipurpose Internet Mail Extensions (MIME). Part One: Format of Internet Message Bodies, N. Freed, N. Borenstein, RFC 2045, November 1996, http://www.ietf.org/rfc/rfc2045.txt. Part Two: Media Types, N. Freed, N. Borenstein, RFC 2046, November 1996. Part Three: Message Header Extensions for Non-ASCII Text, K. Moore, RFC 2047, November 1996. Part Four: Registration Procedures, N. Freed, J. Klensin, J. Postel, RFC 2048, November 1996. Part Five: Conformance Criteria and Examples, N. Freed, N. Borenstein, RFC 2049, November 1996.
RFC 2119
S. Bradner, Key words for use in RFCs to Indicate Requirement Levels, IETF RFC 2119. (See http://www.ietf.org/rfc/rfc2119.txt.)
RFC 2396
T. Berners-Lee, R. Fielding, L. Masinter, Uniform Resource Identifiers (URI): Generic Syntax, IETF RFC 2396, August 1998. (See http://www.ietf.org/rfc/rfc2396.txt.)
RFC 2732
R. Hinden, B. Carpenter, L. Masinter, Format for Literal IPv6 Addresses in URL's, IETF RFC 2732, 1999. (See http://www.ietf.org/rfc/rfc2732.txt.)
Unicode 3.0
The Unicode Consortium, The Unicode Standard, Version 3.0, ISBN 0-201-61633-5. (See http://www.unicode.org/unicode/standard/versions/Unicode3.0.html.)
Unicode 3.1
The Unicode Consortium, The Unicode Standard, Version 3.1.0 is defined by The Unicode Standard, Version 3.0 (Reading, MA, Addison-Wesley, 2000. ISBN 0-201-61633-5), as amended by the Unicode Standard Annex #27: Unicode 3.1 (see http://www.unicode.org/reports/tr27).
Unicode 3.2
The Unicode Consortium, The Unicode Standard, Version 3.2.0 is defined by The Unicode Standard, Version 3.0 (Reading, MA, Addison-Wesley, 2000. ISBN 0-201-61633-5), as amended by the Unicode Standard Annex #27: Unicode 3.1 (see http://www.unicode.org/reports/tr27) and by the Unicode Standard Annex #28: Unicode 3.2 (see http://www.unicode.org/reports/tr28).
UTR #15
Mark Davis, Martin Dürst, Unicode Normalization Forms, Unicode Standard Annex #15. (See http://www.unicode.org/unicode/reports/tr15 for the latest version).
CharReq
Martin J. Dürst, Requirements for String Identity Matching and String Indexing, W3C Working Draft. (See http://www.w3.org/TR/WD-charreq.)
Connolly
D. Connolly, Character Set Considered Harmful, W3C Note. (See http://www.w3.org/MarkUp/html-spec/charset-harmful.)
DOM Level 1
Vidur Apparao et al., Document Object Model (DOM) Level 1 Specification, W3C Recommendation. (See http://www.w3.org/TR/REC-DOM-Level-1.)
DOM3 LS
Ben Chang, Jeroen van Rotterdam, Johnny Stenback, Andy Heninger, Joe Kesselman, Rezaur Rahman Eds., Document Object Model (DOM) Level 3 Abstract Schemas and Load and Save Specification, W3C Working Draft. (See http://www.w3.org/TR/DOM-Level-3-ASLS.)
HTML 4.0
Dave Raggett, Arnaud Le Hors, Ian Jacobs, Eds., HTML 4.0 Specification, W3C Recommendation, 18-Dec-1997 (See http://www.w3.org/TR/REC-html40-971218.)
I-D IRI
Martin Dürst, Michel Suignard, Internationalized Resource Identifiers (IRIs), Internet-Draft, April 2002. (See http://www.w3.org/International/2002/draft-duerst-iri-00.txt.)
Info URI-I18N
Internationalization: URIs and other identifiers. (See http://www.w3.org/International/O-URL-and-ident.)
ISO/IEC 14651
ISO/IEC 14651:2000, Information technology -- International string ordering and comparison -- Method for comparing character strings and description of the common template tailorable ordering as, from time to time, amended, replaced by a new edition or expanded by the addition of new parts. (See http://www.iso.ch for the latest version.)
ISO/IEC 9541-1
ISO/IEC 9541-1:1991, Information technology -- Font information interchange -- Part 1: Architecture. (See http://www.iso.ch/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=17277 for the latest version.)
MathML2
David Carlisle, Patrick Ion, Robert Miner, Nico Poppelier, Eds., Mathematical Markup Language (MathML) Version 2.0, W3C Recommendation. (See http://www.w3.org/TR/MathML2.)
Nicol
Gavin Nicol, The Multilingual World Wide Web, Chapter 2: The WWW As A Multilingual Application. (See http://www.mind-to-mind.com/library/papers/multilingual/multilingual-www.html.)
RFC 2070
F. Yergeau, G. Nicol, G. Adams, M. Dürst, Internationalization of the Hypertext Markup Language, IETF RFC 2070, January 1997. (See http://www.ietf.org/rfc/rfc2070.txt.)
RFC 2277
H. Alvestrand, IETF Policy on Character Sets and Languages, IETF RFC 2277, BCP 18, January 1998. (See http://www.ietf.org/rfc/rfc2277.txt.)
RFC 2279
F. Yergeau, UTF-8, a transformation format of ISO 10646, IETF RFC 2279, January 1998. (See http://www.ietf.org/rfc/rfc2279.txt.)
RFC 2718
L. Masinter, H. Alvestrand, D. Zigmond, R. Petke, Guidelines for new URL Schemes, IETF RFC 2718, November 1999. (See http://www.ietf.org/rfc/rfc2718.txt.)
RFC 2781
P. Hoffman, F. Yergeau, UTF-16, an encoding of ISO 10646, IETF RFC 2781, February 2000. (See http://www.ietf.org/rfc/rfc2781.txt.)
SPREAD
SPREAD - Standardization Project for East Asian Documents Universal Public Entity Set. (See http://www.ascc.net/xml/resource/entities/index.html)
SVG
Jon Ferraiolo, Ed., Scalable Vector Graphics (SVG) 1.0 Specification, W3C Recommendation. (See http://www.w3.org/TR/SVG.)
UTR #10
Mark Davis, Ken Whistler, Unicode Collation Algorithm, Unicode Technical Report #10. (See http://www.unicode.org/unicode/reports/tr10.)
UTR #17
Ken Whistler, Mark Davis, Character Encoding Model, Unicode Technical Report #17. (See http://www.unicode.org/unicode/reports/tr17.)
XLink
Steve DeRose, Eve Maler, David Orchard, Eds, XML Linking Language (XLink) Version 1.0, W3C Recommendation. (See http://www.w3.org/TR/xlink.)
XML 1.0
Tim Bray, Jean Paoli, C. M. Sperberg-McQueen, Eve Maler, Eds., Extensible Markup Language (XML) 1.0, W3C Recommendation. (See http://www.w3.org/TR/REC-xml.)
XML Schema-2
Paul V. Biron , Ashok Malhotra , Eds., XML Schema Part 2: Datatypes, W3C Recommendation. (See http://www.w3.org/TR/xmlschema-2.)
XML Japanese Profile
MURATA Makoto Ed., XML Japanese Profile, W3C Note. (See http://www.w3.org/TR/japanese-xml.)
XPath
James Clark, Steve DeRose, Eds, XML Path Language (XPath) Version 1.0, W3C Recommendation. (See http://www.w3.org/TR/xpath.)
XQuery Operators
Ashok Malhotra, Jim Melton, Jonathan Robie, Norman Walsh, Eds, XQuery 1.0 and XPath 2.0 Functions and Operators, W3C Working Draft. (See http://www.w3.org/TR/xquery-operators.)
XSLT
James Clark Ed., XSL Transformations (XSLT), W3C Recommendation. (See http://www.w3.org/TR/xslt.)