W3C

Character Model for the World Wide Web 1.0: Resource Identifiers

W3C Candidate Recommendation 22 November 2004

This version:
http://www.w3.org/TR/2004/CR-charmod-resid-20041122
Latest version:
http://www.w3.org/TR/charmod-resid
Previous version:
http://www.w3.org/TR/2004/WD-charmod-20040225 (prior to document split)
Editors:
Martin J. Dürst, W3C <duerst@w3.org>
François Yergeau (Invited Expert)
Richard Ishida, W3C <ishida@w3.org>
Misha Wolf (until Dec 2002), Reuters Ltd. <misha.wolf@reuters.com>
Tex Texin (Invited Expert), XenCraft <tex@XenCraft.com>

Abstract

This Architectural Specification provides authors of specifications, software developers, and content developers with a common reference for the use of resource identifiers building on the Universal Character Set, defined jointly by the Unicode Standard and ISO/IEC 10646.

For topics such as use of the terms 'character', 'encoding' and 'string', a reference processing model, choice and identification of character encodings, character escaping, and string indexing, see Character Model for the World Wide Web 1.0: Fundamentals [CharMod]. For normalization and string identity matching, see Character Model for the World Wide Web 1.0: Normalization [CharNorm].

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document is a Candidate Recommendation of the W3C. This document has been produced by the W3C Internationalization Working Group (I18N WG) (Members only), with the help of the Internationalization Interest Group, as part of the W3C Internationalization Activity. Publication as a Candidate Recommendation does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

Section 3 of this document was formerly Section 7 of the Character Model for the World Wide Web 1.0: Fundamentals Last Call Working Draft published 25 February 2004. A more detailed change log is given in Appendix C, Changes.

The I18N WG invites comments on this specification. Comments should be submitted by email to www-i18n-comments@w3.org (public archive). Please send one email per comment where possible, otherwise number comments clearly.

The WG is soliciting implementation reports on this specification and related technology. The WG plans to submit this specification for consideration as a W3C Proposed Recommendation as soon as the following conditions are met: 1) The document Internationalized Resource Identifiers (IRIs) is published as an RFC in IETF Proposed Standard status. 2) There is a test suite that tests the use of IRIs along at least the following axes: a) use of IRIs in several document formats; b) use of IRIs in several locations in the same document format; c) use of non-ASCII characters in different parts of an IRI (e.g. domain name part, path part); d) use of IRIs in documents with various widely used character encodings and with characters from various scripts; e) use of document-specific escapes in IRIs; f) use of IRIs with various URI schemes; g) setup of various servers for IRIs; h) the translation of IRIs into URIs. 3) For each of the above mentioned axes, there are at least two implementations passing the applicable tests. 4) The WG has responded formally to all issues raised against this document. The WG is expecting that this will take at least until 15 January 2005, but possibly longer.

Patent disclosures relevant to this specification may be found on the Working Group's patent disclosure page. This document has been produced under the 24 January 2002 CPP as amended by the W3C Patent Policy Transition Procedure. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification should disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1 Introduction
2 Conformance
3 Character Encoding in Resource Identifiers

Appendices

A References
    A.1 Normative References
    A.2 Other References
B List of conformance criteria (Non-Normative)
C Changes (Non-Normative)
D Acknowledgements (Non-Normative)


1 Introduction

The goal of the Character Model for the World Wide Web is to facilitate use of the Web by all people, regardless of their language, script, writing system, and cultural conventions, in accordance with the W3C goal of universal access. One basic prerequisite to achieve this goal is to be able to transmit and process the characters used around the world in a well-defined and well- understood way.

The main target audience of this specification is W3C specification developers. This specification and parts of it can be referenced from other W3C specifications. It defines conformance criteria for W3C specifications as well as other specifications.

The character model described in this specification provides authors of specifications, software developers, and content developers with a common reference for consistent, interoperable text manipulation on the World Wide Web. Working together, these three groups can build a more international Web.

The topic addressed in this part of the Character Model for the World Wide Web is resource identifiers.

The topic addressed in this part of the Character Model for the World Wide Web is the character encoding of resource identifiers. A resource identifier is a compact string of characters for identifying an abstract or physical resource.

Other parts of the Character Model address the fundamental aspects of the model ([CharMod]) and normalization and string identity matching ([CharNorm]). For more background information, please see [CharMod].

Topics as yet not addressed or barely touched include fuzzy matching, and language tagging. Some of these topics may be addressed in a future versions or parts of this specification.

At the core of the model is the Universal Character Set (UCS), defined jointly by the Unicode Standard [Unicode] and ISO/IEC 10646 [ISO/IEC 10646]. In this document, Unicode is used as a synonym for the Universal Character Set. The model will allow Web documents authored in the world's scripts (and on different platforms) to be exchanged, read, and searched by Web users around the world.

2 Conformance

This section explains the conditions that specifications, software, and Web content have to fulfill to be able to claim conformance to this specification.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY" and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC 2119].

NOTE: RFC 2119 makes it clear that requirements that use SHOULD are not optional and must be complied with unless there are specific reasons not to: "This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course."

This specification defines conformance criteria for specifications. All conformance criteria are preceded by '[S]' where 'S' stands for specifications.

A specification conforms to this document if it:

  1. does not violate any conformance criteria preceded by [S],

  2. documents the reason for any deviation from criteria where the imperative is SHOULD, SHOULD NOT, or RECOMMENDED,

  3. where applicable, requires implementations conforming to the specification to conform to this document,

  4. where applicable, requires content conforming to the specification to conform to this document.

NOTE: Requirements placed on specifications might indirectly cause requirements to be placed on implementations or content that claim to conform to those specifications. Likewise, requirements placed on content may affect implementations designed to produce such content, and so on.

Where this specification places requirements on processing, it is to be understood as a way to specify the desired external behavior. Implementations can use other means of achieving the same results, as long as observable behavior is not affected.

3 Character Encoding in Resource Identifiers

According to the definition in RFC 2396 [RFC 2396], URI references are restricted to a subset of US-ASCII, with an escaping mechanism to encode arbitrary byte values, using the %HH convention. However, the %HH convention by itself is of limited use because there is no definitive mapping from characters to bytes. Also, non-ASCII characters cannot be used directly. Internationalized Resource Identifiers (IRIs) [I-D IRI] solves both problems with an uniform approach that conforms to the Reference Processing Model.

C058  [S]  Specifications that define protocol or format elements (e.g. HTTP headers, XML attributes, etc.) which are to be interpreted as URI references (or specific subsets of URI references, such as absolute URI references, URIs, etc.) SHOULD use Internationalized Resource Identifiers (IRIs) [I-D IRI] (or an appropriate subset thereof).

C059  [S]  Specifications MUST define when the conversion from IRI references to URI references (or subsets thereof) takes place, in accordance with Internationalized Resource Identifiers (IRIs) [I-D IRI].

NOTE: Many current specifications already contain provisions in accordance with Internationalized Resource Identifiers (IRIs) [I-D IRI]. For XML 1.0 [XML 1.0], see Section 4.2.2, External Entities. XML Schema Part 2: Datatypes [XML Schema-2] provides the anyURI datatype (see Section 3.2.17). The XML Linking Language (XLink) [XLink] provides the href attribute (see Section 5.4, Locator Attribute). Further information and links can be found at Internationalization: URIs and other identifiers [Info URI-I18N].

NOTE: Document formats should allow IRIs to be used; handlers for protocols that do not currently support IRIs can convert the IRI to a URI when the IRI is dereferenced.

C060  [S]  Specifications that define new syntax for URIs, such as a new URI scheme or a new kind of fragment identifier, MUST specify that characters outside the US-ASCII repertoire are encoded using UTF-8 and %HH-escaping.

This is in accordance with Guidelines for new URL Schemes [RFC 2718], Section 2.2.5.

C061  [S]  Specifications that define new syntax for URIs SHOULD also define the normalization requirements for the syntax they introduce.

A References

B List of conformance criteria (Non-Normative)

Below is a list of the conformance criteria in this specification, in document order. This list can be used to check specifications for conformance to this specification.

When doing so, the following points should be kept in mind:

C058[S] Specifications that define protocol or format elements (e.g. HTTP headers, XML attributes, etc.) which are to be interpreted as URI references (or specific subsets of URI references, such as absolute URI references, URIs, etc.) SHOULD use Internationalized Resource Identifiers (IRIs) [I-D IRI] (or an appropriate subset thereof).
C059[S] Specifications MUST define when the conversion from IRI references to URI references (or subsets thereof) takes place, in accordance with Internationalized Resource Identifiers (IRIs) [I-D IRI].
C060[S] Specifications that define new syntax for URIs, such as a new URI scheme or a new kind of fragment identifier, MUST specify that characters outside the US-ASCII repertoire are encoded using UTF-8 and %HH-escaping.
C061[S] Specifications that define new syntax for URIs SHOULD also define the normalization requirements for the syntax they introduce.

C Changes (Non-Normative)

This document is based Section 7 of the Character Model for the World Wide Web 1.0: Fundamentals Last Call Working Draft published 25 February 2004. Changes between Section 7 of that document and Section 3 of the present document are as follows:

In addition, the remaining parts of this document have changed compared to the corresponding parts of the abovementioned Last Call Working Draft, as follows: The Introduction has been shortened to concentrate on the material in this document. The Conformance section has been reduced to take into account that this document only contains conformance criteria for specifications. The References section has been shortened by removing unrelated references. Where necessary, references have been updated. The reference to [I-D IRI] was moved to the Normative References subsection. A List of Conformance and this section on Changes have been added.

D Acknowledgements (Non-Normative)

Tim Berners-Lee and James Clark provided important details. Asmus Freytag , Addison Phillips, and in early stages Ian Jacobs, provided significant help in the authoring and editing process. The W3C I18N WG and IG, as well as many others, provided many helpful comments and suggestions.