Handling character encodings in HTML and CSS (tutorial)

Why should you read this?

If a browser is unable to detect the character encoding used in a page, the content may be unreadable. The information in this tutorial is particularly important for those maintaining and extending a multilingual site, but declaring the character encoding of the document is important for anyone producing HTML or CSS that uses non-ASCII characters, because, although it looks good to you, other people"s browser settings can affect readability. This tutorial will give you an understanding of the topic that will help you make the right choices.

Objectives

When you have finished this tutorial you should:

This tutorial gathers together and organizes pointers to articles that, taken together, help you understand how to handle the essential aspects of authoring HTML and CSS related to characters and character encodings.

In a nutshell

Save your pages as UTF-8.

Always declare the encoding of your document. Use the HTTP header if you can. Always use an in-document declaration too.

<meta charset="utf-8">

You can use @charset or HTTP headers to declare the encoding of your style sheet, but you only need to do so if your style sheet contains non-ASCII characters and, for some reason, you can't rely on the encoding of the HTML and the associated style sheet to be the same.

Try to avoid using the byte-order mark in UTF-8, and ensure that your HTML code is saved in Unicode normalization form C (NFC).

Avoid using character escapes, except for invisible or ambiguous characters. And don't use Unicode control characters when you can use markup instead.

Essential background information

If you are a newcomer to this topic, there are certain foundational concepts you need to understand if you are to follow various parts of the tutorial. If you are familiar with these concepts, you can skip to the next section.

Choosing and applying a character encoding

Content is composed of a sequence of characters. Characters represent letters of the alphabet, punctuation, etc. But content is stored in a computer as a sequence of bytes, which are numeric values. Sometimes more than one byte is used to represent a single character. Like codes used in espionage, the way that the sequence of bytes is converted to characters depends on what key was used to encode the text. In this context, that key is called a character encoding. There are many character encodings to choose from.

Choosing & applying a character encoding offers simple advice on which character encoding to use for your content, and how to apply it.

How to declare a character encoding

You should always specify the encoding used for an HTML or XML page. If you don't, you risk that characters in your content are incorrectly interpreted. This is not just an issue of human readability, increasingly machines need to understand your data too. You should also check that you are not specifying different encodings in different places.

Declaring character encodings in HTML provides quick recommendations for those who just want to be told what to do, and more detailed information for those who need it.

Declaring character encodings in CSS provides information for CSS.

The byte-order mark (BOM)

The byte-order mark, or BOM, is something you will come across when using a Unicode-based character encoding, such as UTF-8 and UTF-16. In some cases you will need to remove the BOM, in others you need to ensure that it is there.

The byte-order mark (BOM) in HTML helps you understand the issues.

Unicode normalization forms

Normalization is something you need to be aware of if you are authoring in UTF-8, be it HTML pages or CSS style sheets, particularly if you are dealing with text in a script that uses accents or other diacritics.

Normalization in HTML and CSS explains this further.

Using character escapes

You can use a character escape to represent any character from the Unicode character set in HTML, XML or CSS using only ASCII characters.

Using character escapes in markup and CSS provides best practices for use of escapes, and tells you how to use them when they are needed.

Characters or markup?

Finally, there are a range of control-like Unicode characters, some of which fulfill the same role as markup. The question is, which should you use, and which should you avoid?

Characters or markup? answers that question.