Accesskey n skips to in-page navigation. Skip to the content start.

s_gotoW3cHome Internationalization
 

Introducing Character Sets and Encodings

Intended audience: anyone who is new to internationalization and needs guidance on topics to consider and ways to get into the material on the site.

Updated 2009-05-01 9:44

This page provides some orientation for newcomers to Web internationalization who don't really know where to start. The aim is to ease you gently into some of the material on the site.

You can find a selection of more detailed articles using the links to the right. Once you get some ideas from this page, you will probably just use the topic index, the techniques index, or the site search.

What's it about?

Learn more...

Character encodings for beginners explains some of the basic concepts about character encodings, and why you should care.

Essential definitions related to character encodings provides explanations of terminology such as Unicode, character sets, coded character sets, character encodings, the document character set, and character escapes.

A character set is a collection of letters and symbols used in a writing system. For example, the ASCII character set covers letters and symbols for English text, ISO-8859-6 covers letters and symbols needed for many languages based on the Arabic script, and the Unicode character set contains characters for most of the living languages and scripts in the world.

Characters in a character set are stored as one or more bytes in a computer. Each byte or sequence of bytes represents a given character. A character encoding is the key that maps a particular byte or sequence of bytes to particular characters that the font renders as text.

There are many different character encodings. If the wrong encoding is applied to the bytes in memory, the result will be unintelligible text. It is therefore important, if people are to read your content, that you correctly label the character encoding used.

Choosing an encoding

Everyone developing content, whether content authors or programmers, must decide what character encoding to use. UTF-8 is a popular recommendation these days, but there may still be things you should consider before using it.

Learn more...

HTML & CSS authors
Choosing a character encoding

Spec developers
Choosing character encodings

Server setup
Choosing a character encoding


Declaring and applying an encoding

Once it has been decided what encoding to use, content developers and programmers must ensure that it is declared in the right way.

With a technology such as XHTML, encoding declarations are not always straightforward; they require an understanding of 'standards' vs. 'quirks' modes, and the impact of the XML declaration.

You must also ensure that your data is saved in the encoding you have chosen, it is not sufficient to just label it.

Content developers and webmasters may also need to ensure that the server delivers content with the correct character encoding declarations, since server settings can override in-document declarations.

Escapes

Escapes are a way of representing a character using only ASCII text. They provide a way of representing characters that are not available in the character encoding you are using, or a way of avoiding the use of the character for other reasons (such as when they may conflict with syntax). You should be clear on when and how these escapes should be used.


Web addresses

These days web addresses can also include non-ASCII characters. The user does little other than click on the appropriate link or enter the text as they see it, the heavy lifting is done by the user agent, but you may be interested to know how this works.

Specification developers should design their specifications so that non-ASCII web addresses can be used.


By: Richard Ishida, W3C.

Valid XHTML 1.0!
Valid CSS!
Encoded in UTF-8!

Content first published 2006-01-16. Last substantive update 2009-05-01 9:44 GMT. This version 2011-10-10 10:29 GMT

For the history of document changes, search for gs-characters in the i18n blog.