This section describes the status of this document at the time of its
publication. Other documents may supersede this document. The latest status
of this document series is maintained at the W3C.
This document has been developed by W3C's Technical Architecture Group (TAG) (charter).
This draft is an Editor's Draft and some new content has not yet been
reviewed by the TAG. The primary changes in this draft were editorial changes
based on some comments from Stuart Williams, and the addition of a note
regarding the phrase "on the Web" based on comments from David Orchard. A
complete list of changes
since the previous Working Draft is available on the Web.
This draft remains incomplete; sections 1, 2, and 3 are the most
developed; 4 the least. The TAG has published a number of findings that address specific
architecture issues. Parts of those findings may appear in subsequent drafts.
Please also consult the list of issues under
consideration by the TAG.
This draft includes some editorial notes and also references to open TAG issues. These do
not represent all open issues in the document. They are expected to disappear
from future drafts.
Publication as a Working Draft does not imply endorsement by the W3C
Membership. This is a draft document and may be updated, replaced or
obsoleted by other documents at any time. It is inappropriate to cite this
document as other than "work in progress."
The latest information regarding patent disclosures related to this document
is available on the Web. As of this publication, there are no disclosures.
Please send comments on this document to the public W3C TAG mailing list
www-tag@w3.org (archive).
A list of current W3C
Recommendations and other technical documents can be found at the W3C Web
site.
Uniform Resource Identifiers
(URI), defined by "Uniform Resource Identifiers (URI): Generic Syntax" [URI], are central to Web Architecture.
Identifier here is used here in the sense of name. Parties
who wish to communicate about something will establish a shared vocabulary,
i.e. a shared set of bindings between identifiers and things. This shared
vocabulary has a tangible value: it reduces the cost of communication. The
ability to use common identifiers across communities is what motivates global
naming in Web Architecture.
URIs identify resources. A resource can be be anything. Certain resources
are information resources, which convey information. These are termed
information resources. Much of this document discusses information
resources, often using the term resource.
An information resource is on the Web when it can be accessed
in practice.
When a representation of one information resource refers to another
information resource with a URI, a link is formed between the two resources. The
networked information system is built of linked resources, and the
large-scale effect is a shared information space. The value of the Web grows
exponentially as a function of the number of linked resources (the "network
effect").
Principle
Use
URIs: All important resources SHOULD be identified by a URI.3
Although there's no precise definition of an "important resource," a
resource should have a URI if a third party might reasonably want to link to
it, make or refute assertions about it, retrieve or cache a representation of
it, transclude all or part of it into another resource, or annotate it.
There are many benefits to making resources identifiable by URI. Some are
by design (e.g., linking, bookmarking, and caching), others (e.g., global
search services) were not predicted.4
Communication between parties is facilitated by the ability to establish
when they are talking about the same thing. The most straightforward way to
establish that two parties are refering to the same resource is to compare
the spelling (i.e., as strings) of the identifiers the parties are using.
Section 6 of [URI] discusses this type of
analysis. In that specification, determination of equivalence or difference
of URIs is based on string comparison, perhaps augmented by reference to
additional rules provided by URI scheme definitions (e.g., for HTTP URIs, the
authority component is case-insensitive). Depending on the application, an
agent may invest more processing effort to reduce the likelihood of a false
negative (i.e., two URIs identify the same resource, but that was not
detected).
There may be other ways to establish that two parties are refering to the
same resource that are not based on string comparison; see the section on
future directions for determining
that two URIs identify the same resource.
Agents that reach conclusions about identity beyond what they are licensed
to do (e.g., by specification, or community convention, or site-specific
convention) take responsibility for any problems that result. For instance,
agents should not assume that http://weather.example.com/Oaxaca
and http://weather.example.com/oaxaca identify the same
resource, since none of the specifications involved states that the path part
of an HTTP URI is case-insensitive. Web servers may vary in how they are
configured to handle case-sensitivity. Agents that assume these URIs identify
the same resource take responsibility for any resulting problems.
Although it is possible to determine that two URIs are equivalent, it is
generally not possible by mere inspection of two URIs to be sure that they
identify different resources. Web architecture does not constrain resources
to be uniquely named.
Good practice
Spelling URIs: If an agent has been provided with a
URI to refer to a resource, the agent SHOULD use the spelling of the URI as
it was originally provided.
To help parties know when they are referring to the same resource, it
follows that URI producers should be conservative about the number of
different URIs they produce for the same resource. For instance, the parties
responsible for weather.example.com have no reason to use both
http://weather.example.com/Oaxaca and
http://weather.example.com/oaxaca to refer to the same resource;
agents will not detect the equivalence relationship. In this case, one URI
should be chosen and used consistently. See section 6.3 of [URI] for further advice on how to reduce the risk of false
negatives.
URI consumers cannot, in general, determine the meaning of a resource by
inspection of a URI that identifies it. In our travel scenario, the example URI
(http://weather.example.com/oaxaca) suggests that the identified
resource has something to do with the weather in Oaxaca. Although it is
tempetd to infer this, it is not licesed. Although short, meaningful URIs
benefit people, URI consumers must not rely on the URI string to communicate
the meaning of a resource. A site reporting the weather in Oaxaca could just
as easily be identified by the URI http://vjc.example.com/315.
And the URI http://weather.example.com/vancouver might identify
the resource "my photo album." See the section on retrieving a representation for
information about how agents convey information about a resource.
Editor's note: When finding available on URI
opacity, link from here.
In the URI http://weather.example.com/, the "http" that
appears before the colon (":") is a URI scheme name. There are other schemes,
such as "mailto" and "ftp". It is common to classify URIs by scheme; a URI
with scheme "http" is called an "HTTP URI."
Each URI begins with a URI scheme name. The scheme name
corresponds to a specification for assigning identifiers within that scheme.
As such, the URI syntax is a federated and extensible naming system wherein
each scheme's specification may further restrict the syntax and semantics of
identifiers using that scheme. Furthermore, the URI scheme specification
specifies how an agent can dereference the URI.
Several URI schemes incorporate identification mechanisms that pre-date
the Web into this syntax:
- MAILTO URIs identify mailboxes:
mailto:nobody@example.org
- FTP URIs identify identify ftp files and directories:
ftp://example.org/aDirectory/aFile
- NEWS URIs identify USENET newsgroups:
news:comp.infosystems.www
- TEL URIs identify terminals on the telephone network:
tel:+1-816-555-1212
Other URI schemes have been introduced since the advent of the Web,
including those introduced as a consequence of new protocols. Examples of
URIs for these schemes include:
- HTTP URIs identify information resources:
http://www.example.org/something?with=arg1;and=arg2
- LDAP URIs identify things in the x.509 system:
ldap://ldap.itd.umich.edu/c=GB?objectClass?one
- URNs:
urn:oasis:SAML:1.0
The Internet Assigned Numbers Authority (IANA)
maintains a registry [IANASchemes] of
mapping between URI scheme names and their specifications. For instance, the
IANA registry indicates that the "http" scheme is defined by [RFC2616]. The process for registration of new URI schemes
is defined by RFC2717.
Since many aspects of URI processing are scheme-dependent, and since a
huge amount of deployed software already processes URIs of well-known
schemes, the cost of introduction of new URI schemes is high.
Good practice
New URI schemes: Authors of specifications
SHOULD avoid introducing new URI schemes when existing schemes can be used to
meet the goals of the specifications.
Consider our travel scenario: should
the authority providing information about the weather in Oaxaca register a
new URI scheme "weather" for the identification of resources related to the
weather? They might then publish URIs such as
weather://travel.example.com/oaxaca. While the Web Architecture
allows the definition of new schemes, there is a cost to registration and
especially deployment of new schemes. When an agent dereferences such a URI,
if what really happens is that HTTP GET is invoked to retrieve an HTML
representation of the resource, then an HTTP URI would have sufficed. If a
URI scheme exists that meets the needs of an application, designers should
use it rather than invent one. Furthermore, designers should expect that it
will prove useful to be able to share a URI across applications, even if that
utility is not initially evident.
If the motivation behind registering a new scheme is to allow an agent to
launch a particular application when retrieving a representation, such
dispatching can be accomplished at lower expense by registering a new media
type instead.
The use of unregistered URI schemes is discouraged for a number of
reasons:
- There is no generally accepted way to locate the scheme
specification.
- Someone else may be using the scheme for other purposes.
- One should not expect that general purpose software will do anything
useful with URIs of this scheme; the network effect is lost.
In the URI http://weather.example.com/, the string
weather.example.com (between "//" and the next "/") called the
authority component. Many URI schemes include a hierarchical element for a
naming authority
such that governance of the name space defined by the remainder of the URI is
delegated to that authority (which may, in turn, delegate it further). The
generic syntax provides a common means for distinguishing an authority based
on a registered domain name or server address. See section 3.2 of [URI] for more information about the authority
portion of a URI.
How authority is delegated depends on the URI scheme. The deployment and
use of different URI schemes may require varying degrees of central
coordination and administration. For example, MAILTO, FTP, and HTTP URIs
depend on the use of the DNS and IANA infrastructure; see "ICP-1: Internet
Domain Name System Structure and Delegation" [IANAICP1] for more information about how the IANA
manages delegation of domain names.
Successful communication between two parties about a piece of information
relies on shared understanding of the meaning of the information. Thousands
of independent parties can identify and communicate about a Web resource. To
give these parties the confidence that they are all talking about the same
thing when they refer to "the resource identified by the following URI ..."
the design choice for the Web is, in general, that the owner of a resource
assigns the authoritative interpretation of representations of the resource.
See the draft TAG finding "Client handling of
MIME headers" for related discussion.
In our travel scenario, the agent
responsible for weather.example.com has license to create
representations of this resource and assign their authoritative
interpretation.
In our travel scenario the server
returns an XHTML representation when Dan dereferences the URI
http://weather.example.com/oaxaca. Then, by navigating within
the XHTML content, Dan finds that the URI
http://weather.example.com/oaxaca#tom refers to information
about tomorrow's weather in Oaxaca. This URI includes the fragment identifier
"tom" (the string after the "#").
The fragment
identifier component of a URI allows indirect identification of a
secondary resource by reference to a primary resource and additional
identifying information. The identified secondary resource may be some
portion or subset of the primary resource, some view on representations of
the primary resource, or some other resource.
When a resource has a URI and has a representation; and in the language of
that representation (using a syntax and semantics defined by the MIME type
specification) a given string identifies a second resource, then the
concatenation of the URI, a hash mark and the string forms a URI for that
second resource.
Although the generic URI syntax allows any URI to end with a fragment
identifier, some URI schemes do not specify the use of fragment identifiers.
For instance, fragment identifier usage is not specified for MAILTO URIs.
For URI schemes that do specify the use of fragment identifiers, the
syntax and semantics of those identifiers is defined by the set of representations that might result
from a retrieval
action on the primary resource. The presence of a fragment identifier
component in a URI does not imply that a retrieval action will take place.
Interpretation of the fragment identifier during a retrieval action is
performed solely by the user agent; the fragment identifier is not passed to
other systems during the process of retrieval. This means that some
intermediaries in the Web architecture (e.g., proxies) have no effect on
fragment identifiers and that redirection (in HTTP [RFC2616], for example) does not account for them.
Suppose that the managers of weather.example.com provide a
visual map of the meteorological conditions in Oaxaca as part of the
representation served for http://weather.example.com/oaxaca.
They might encode the same visual map in a number of image formats to meet
different needs (e.g., they might serve PNG, SVG, and JPEG/JFIF). Dan's user
agent and the server engage in HTTP content negotiation, so that Dan receives
the best image format his user agent can handle or the format he usually
prefers. The URI http://weather.example.com/oaxaca/map#zicatela
refers to a portion of the weather map that shows the Zicatela Beach, where
Dan intends to go surfing. Clients can do something useful with the fragment
identifier and the SVG representation, since SVG defines fragment identifier
semantics. Clients should not be expected to do something useful with the
fragment identifier for the PNG or JPEG/JFIF representations since those
specifications do not define fragment identifier semantics.
Good practice
Content negotiation with fragments: Authors SHOULD
NOT use HTTP content negotiation for different media types that have
incompatible fragment identifier semantics.
Given a URI, a system may attempt to perform a variety of operations on
the resource, as might be characterized by such words as "access", "update",
"replace", or "find attributes". Available operations depend on the formats
and protocols that make use of URIs. The URI specification (in [URI], section 1.2.2) defines the following terms
related to interactions through a URI.
- Dereference
- To dereference a URI is to use an access mechanism to perform an
action on the URI's resource.
- Retrieval
- A URI dereference that causes an agent to retrieve a representation
of the associated resource.
During access of a resource, an agent applies a succession of relevant
specifications, beginning with the specification of the context in which the
URI is found (e.g., a format or protocol specification, or an application).
Any one of these specifications may define more than one access mechanism
(e.g., the HTTP protocol defines a number of access methods, including GET,
HEAD, and POST). Note that the information governing the choice of access
mechanism may be found in the context, not the URI itself (e.g., the choice
of HTTP GET v. HTTP HEAD). The draft TAG finding "URIs,
Addressability, and the use of HTTP GET and POST." discusses
issues surrounding multiple access mechanisms and the relation to URI
addressability.
Some URI schemes (e.g., the URN scheme [RFC 2141]) do not define dereference mechanisms.
TAG issue metadataInURI-31:
Should metadata (e.g., versioning information) be encoded in URIs?
TAG issue siteDate-26: Web
site metadata improving on robots.txt, w3c/p3p and favicon etc.
One of the most important actions involving a resource is to retrieve a representation of it (for example,
by using HTTP GET). As stated above, the authority responsible for a URI determines what the
URI identifies and which representations are used for interaction with the
resource. The representations communicate some or all of the state of the
resource.
Good practice
Resource descriptions: Owners of important
resources SHOULD make available representations that describe those
resources.
As an example of representation retrieval, suppose that the URI
http://weather.example.com/budapest is used within an
a element of an SVG document. The sequence of specifications
applied is:
- The SVG 1.0 Recommendation [SVG10],
which imports the link semantics defined by XLink 1.0 [XLink10]. Section 17.1 of the SVG specification
suggests that interaction with an
a link involves retrieving
a representation of a resource, identified by the XLink href
attribute: "By activating these links (by clicking with the mouse,
through keyboard input, and voice commands), users may visit these
resources."
- The attribute
xlink:href is defined in section 5.4 of the
XLink 1.0 [XLink10] specification
states that "The value of the href attribute must be a URI reference as
defined in [IETF RFC 2396], or must result in a URI reference after the
escaping procedure described below is applied."
- The URI specification [URI] states that
"Each URI begins with a scheme name that refers to a specification for
assigning identifiers within that scheme." The URI scheme name in this
example is "http".
- To find out what specification defines the "http" scheme, we look up
the mapping in [IANASchemes]; the
"http" URI scheme is defined in the HTTP/1.1 specification (RFC 2616 [RFC2616], section 3.2.2).
- The HTTP/1.1 specification defines several access mechanisms. In this
SVG context, the user agent employs the GET method (defined in section
9.3 of [RFC2616]) to retrieve the
representation. The HTTP/1.1 specification explains how the server
constructs the response (section 6 of [RFC2616]), including the media type of the
representation. Section 1.4 states "HTTP communication usually takes
place over TCP/IP connections." Though not shown in this example, the
user agent would continue this process by implementing those
specifications.
- The user agent interprets the returned representation according to the
media type, by looking up which specification defines the media type in
the [IANAMIME] registry.
Note that, in general, one cannot determine the media type(s) of
representation(s) of a resource by inspecting a URI for that resource. For
example, do not assume that all representations of
http://example.com/page.html are HTML. The HTTP protocol does
not constrain the media type based on the path component of the URI; the
server is free to return a PNG image representation.
Editor's note: It is an open question whether
the TAG will define and use the phrase "on the Web" in this document.
Definitions that have been suggested include "is identified by a URI" and "is
identified by a URI and at least one representation is available for
retrieval."
Dan's retrieval of weather information qualifies as a "safe" interaction;
a safe
interaction is one where the user agent does not commit to anything
beyond the interaction and is not responsible for any consequences other than
the interaction itself (e.g., a read-only query or lookup). Other Web
interactions resemble orders more than queries. These unsafe
interactions may cause a change to the state of a resource; the
user may be held responsible for the consequences of these interactions.
Unsafe interactions include subscription services, posting to a list, or
modifying a database.
Safe interactions are important because these are interactions where users
can browse with confidence and where software programs (e.g., search engines
and browsers that pre-cache data for the user) can follow links safely. Users
(or software agents acting on their behalf) do not commit themselves to
anything by querying a resource or following a link.
Principle
Safe retrieval: Agents do not incur obligations by
retrieving a representation.
For instance, suppose in our travel
scenario that the managers of weather.example.com offer a
monthly newsletter available by subscription. It is incorrect and
harmful to publish a page
http://example.com/oxaca/aboutNewsLetter that states "... terms
and conditions..." with a link to
http://example.com/oxaca/newsLetter because search services may
link directly to http://example.com/oxaca/newsLetter and readers
that follow such links may not have seen, let alone agreed to, the terms and
conditions.
For more information about safe and unsafe operations using HTTP GET and
POST, and handling security concerns around the use of HTTP GET, see the
draft TAG finding "URIs, Addressability,
and the use of HTTP GET and POST."
The value of a URI increases with the predictability of interactions using
that URI.
Good practice
URI Persistence: Parties responsible for a URI SHOULD
service that URI predictably and consistently.
Service breakdowns include:
- No service available (i.e., dereferencing fails).
- Inconsistent representations served. Note the difference between a
resource owner changing representations predictably in light of the
nature of the resource (e.g., the weather in Oaxaca changes) and the
owner changing representations arbitrarily.
- Improper use of content negotiation, such as serving two images as
equivalent through HTTP content negotiation, where one image represents a
square and the other a circle.
There are strong social expectations that once a URI identifies a
particular resource, it should continue indefinitely to refer to that
resource; this is called URI persistence. URI persistence is
always a matter of policy and commitment on the part of authorities servicing
URIs rather than a constraint imposed by technological means.
URI Ambiguity is the use of the same URI to refer to more than
one distinct thing, and should be avoided.
In a simple example, one part of an enterprise has data about web pages,
included who created them and when. Another part of the company has data
about companies, including who created them and when. The second dataset uses
the URIs of the organization's home page as though it was the URI of the
organization itself. When data is merged, the reuse of the URI causes
information about companies to be merged with that of the home pages,
resulting in contradictory nonsense.
Ambiguity, should not be confused with indirect identification, which is
normal. One indirectly identifies things though related things, such as
people by their mailboxes, and organizations by their web pages. For example,
when you are asked to register the participants at a meeting by giving email
addreses, both sides know that they are using the identifier for a mailbox to
indirectly identify the person. The URI still in the architecture is the URI
of the mailbox, not the person.
The RDF language allows things of any kind to be described and allocated a
URI, such that dereferencing the URI then yields that description. This
avoids ambiguity between the document and any of the things it describes.
HTTP [RFC2616] has been designed to
help service URIs. For example, HTTP redirection (via some of the 3xx
response codes) permits servers to tell an agent that further action needs to
be taken by the agent in order to fulfill the request (e.g., the resource has
been assigned a new URI). In addition, content negotiation also promotes
consistency, as a site manager would not be required to define new URIs for
each new format that is supported, as would be the case with protocols that
don't support content negotiation, such as FTP.
For more discussion about URI persistence, refer to [Cool].5
As we have seen, identification of a resource is distinct from interacting
with that resource. It is reasonable to control access to the resource (e.g.,
for security reasons), but it is unreasonable to prohibit others from merely
identifying the resource.
As an analogy: A building might have a policy that the public may only
enter via the main front door, and only during business hours. People
employed in the building and in making deliveries to it might use other doors
as appropriate. Such a policy would be enforced by a combination of security
personnel and mechanical devices such as locks and pass-cards. One would not
enforce this policy by hiding some of the building entrances, nor by
requesting legislation requiring the use of the front door and forbidding
anyone to reveal the fact that there are other doors to the building.
In the travel scenario, imagine that
Dan and Norm both subscribe to the weather.example.com
newsletter. Dan wishes to point out an article of particular interest to
Norm, using a URI. The managers of weather.example.com can offer
Dan and Norm the benefits of URIs (e.g., bookmarking and linking) and still
control access to the newsletter by authorized parties. The Web provides
several mechanisms to control access to resources, none of which relies on
hiding or suppressing URIs for those resources. For more information on
identification and access control, please refer to the TAG finding "'Deep
Linking' in the World Wide Web."
The integration of internationalized identifiers (i.e., composed of
characters beyond those allowed by [URI])
into the Web Architecture is an important and open issue. See TAG issue IRIEverywhere-27
for discussion about work going on in this area.
Emerging Semantic Web technologies, including "DAML+OIL" [DAMLOIL] and "Web Ontology Language (OWL)" [OWL10], define RDF properties such as
equivalentTo and FunctionalProperty to state -- or
at least claim -- formally that two URIs identify the same resource.
There has been some discussion but no agreement that new access protocols
should provide a means to convert fragment identifiers according to media
type.
Fragment identifier semantics may differ among formats. See related TAG
issues httpRange-14
and RDFinXHTML-35
and abstractComponentRefs-37.
The Dynamic Delegation Discovery System (DDDS) ([RFC3401] and related RFCs) is used to
implement lazy binding of strings to data, in order to support dynamically
configured delegation systems. This system is designed to allow resolution of
any type of URI, in particular URNs.
One area of work involves the creation of globally unique identifiers in a
file-sharing system without centralized or hierarchical administration.
A representation is data that represents
or describes the state of a resource. It consists of:
- Electronic data expressed in one or more formats (e.g., XHTML, CSS,
PNG, XLink, RDF/XML, and SMIL animation) used separately or in
combination.
- Metadata about the representation, such as the Internet Media Type
(defined in RFC 2046 [RFC2046]). The
Internet Media Type is the key to the correct interpretation of a
resource representation, and governs the handling of fragment identifiers.
Web agents use representations to modify as well as retrieve resource
state.
When agents transfer representations via messages (see the section on interactions for information about
message protocols), the message often includes additional metadata that is
not part of the representation (e.g., the HTTP Server field or the request
method, URI, etc.). A message may even include "meta-metadata" (for
message-integrity checks). The representation consists of those bits that
would not change regardless of the transfer protocol used to exchange
them.
In our previous travel scenario the
representation Dan receives (and whether he receives one at all) depends on a
number of factors, including:
- Whether the agents responsible for
weather.example.com
respond to requests at all;
- Whether the agents responsible for
weather.example.com
make available one or more representations for the resource identified by
http://weather.example.com/oaxaca;
- Whether Dan has access privileges to such a representation;
- If the agents responsible for
weather.example.com have
provided more than one representation (in different formats such as HTML,
PNG, or RDF, in different languages such as English and Spanish, etc.),
the resulting representation may depend on negotiation between the user
agent and server that occurs as part of the HTTP transaction.
- When Dan made the request. Since the weather in Oaxaca changes, Dan
should expect that representations will change over time.
We discuss these issues in more detail below.
As discussed above, the owner of a resource assign URIs for that resource,
create representations of the resource, and assign their authoritative
interpretation. This interpretation is described in part by metadata that is
part of the representation, notably the Internet media type. At times there
may be inconsistencies between metadata and what is specified in a format.
Examples of inconsistencies between headers and format data that have been
observed in practice include:
- The Unicode encoding of a message body (XML document) is inconsistent
with the value of the charset parameter in the message headers.
- The namespace of the root element of a message body (XML document) is
inconsistent with the value of the 'Content-Type' field in HTTP
headers.
User agents should detect such inconsistencies but should not resolve them
without involving the user (e.g., by securing permission or at least
providing notification). User agents must not silently ignore authoritative
server metadata.
See the draft TAG finding "Client handling of
MIME headers" for more in-depth discussion and examples.
The Web can be used to interchange resource representations in any format.
This flexibility is important, since there is continuing progress in the
development of new data formats for new applications and the refinement of
existing ones.
For a format to be usefully interoperable between two parties, the parties
must have a shared understanding of its syntax and semantics. This is
not to imply that a sender of data can count on constraining its
treatment by a receiver; simply that making good use of electronic data
usually requires knowledge of its designers' intentions.
For a format to be widely interoperable across the Web:
- There SHOULD be a normative specification which is a stable and widely
available Web resource.
- The format specification SHOULD have an officially registered Internet
media type (see the [IANAMIME]
registry). See TAG finding finding "Internet Media Type
registration, consistency of use."
- The format specification SHOULD define the syntax and semantics of
fragment identifiers.
- The format specification SHOULD allow Web-wide linking, not just
internal document linking.
- The format specification SHOULD allow authors to use URIs without
constraining them to a limited set of URI schemes.
- The format specification MAY use Qualified Names (QNames) for
identifiers in content, but format designers should be aware of the
limitations and interoperability risks associated with QNames. See the
TAG finding "Using QNames as
Identifiers in Content" for more information.
Although the Web architecture allows for the deployment of new data
formats, the creation and deployment of new formats (and software able to
handle them) can be very expensive. Thus, before inventing a new data format,
designers should carefully consider re-using one that is already available.
For example, if a format is required to contain human-readable text with
embedded hyperlinks, it is almost certainly better to use HTML for this
purpose than to invent a new format.
As noted above, the utility of data formats starts with the availability
of a normative specification. Some of the desirable characteristics of a
format include:
- Attention to Programmer Needs
- A specification that is written to meet the needs of programmers is
more likely to be implemented, and thus more likely to be useful. In
particular, the specification SHOULD be in part formal and
mathematical, rather than relying exclusively on narrative.
- Attention to Author Needs
- Formats that allow authors to intuitively and efficiently address
problems they are trying to solve, that can be readily learned, that
are simple to use, and that are interoperable are more likely to be
adopted by authors and authoring tool developers.
- Attention to User Needs
- A number of characteristics of a format specification will address
the needs of end users, and doing so will create a market for the
format. Attention to issues of accessibility, internationalization, and
device-independence will tend to make a format specification more
flexible.
- Attention to Error-Handling
- Given that representations are generated by humans (either coded
directly or mediated by an authoring tool) and then transmitted in a
heterogeneous network, it is inevitable that errors will occur.
Specifications of data formats SHOULD be clear about behavior in the
presence of errors. It is reasonable to specify that errors should be
worked around, or should result in the termination of a transaction or
session. It is not acceptable for the behavior in the face of errors to
be left unspecified. See the draft TAG finding
"Client handling
of MIME headers" for more discussion about error
reporting.
- Issue errorHandling-20
- Information hiding
- When designing specifications that address independent functions of a
system, references between the specifications are in general harmful.
They are harmful because they impede the independent evolution of the
specifications.
- For example, it is a strength of XML that XPath cannot query the HTTP
header. It is a strength of HTTP that it does not refer to details of
the underlying TCP to the extent that it cannot be run over a different
transport service. Similarly, the RDF data graph has a significance
that is independent of the actual serialization.
- Sometimes it is necessary (and good for given application) to break
layers. For example, it is good for an HTTP agent to be aware of TCP
speeds and round trip times to different mirror servers in order to
optimize the choice of server. When designing specification, identify
the functionalities that break layers so it is clear when they are
being used.
The section on architectural
specifications includes references to additional format specification
guidelines.
Other design issues:
This section discusses important characteristics of data formats which can
together be used to describe and understand them.
A textual data format is one in which the data is specified as a linear
sequence of characters. HTML, Internet e-mail, and all XML-based languages
are textual. In modern textual data formats, the characters are usually taken
from the Unicode repertoire.
Binary data formats are those in which portions of the data are encoded
for direct use by computer processors, for example thirty-two bit
little-endian two's-complement and sixty-four bit IEEE double-precision
floating-point. The portions of data so represented include numeric values,
pointers, and compressed data of all sorts.
In principle, all data can be represented using textual formats.
The trade-offs between binary and textual data formats are complex and
application-dependent. Binary formats can be substantially more compact,
particularly for complex pointer-rich data structures. Also, they can be
consumed more rapidly by software in those cases where they can be loaded
into memory and used with little or no conversion.
Textual formats are often more portable and interoperable, since there are
fewer choices for representation of the basic units (characters), and those
choices are well-understood and widely implemented.
Textual formats also have the considerable advantage that they can be
directly read and understood by human beings. This can simplify the tasks of
creating and maintaining processing software, and allow the direct
intervention of humans in the processing chain without recourse to tools any
more complex than the ubiquitous text editor. Finally, it simplifies the
necessary human task of learning about new data formats (the "View Source"
effect).
It is important to emphasize that intuition as to such matters as data
size and processing speed are not a reliable guide in data format design;
quantitative studies are essential to a correct understanding of the
trade-offs.
TAG issue binaryXML-30: Effect
of Mobile on architecture - size, complexity, memory constraints. Binary
infosets, storage efficiency.
Final-form data formats are not designed to allow modification or uses
other than that intended by their designers. An example would be PDF, which
is designed to support the presentation of page images on either screen or
paper, and is not readily used in any other way. XSL Formatting Objects
(XSL-FO) share this characteristic.
XHTML, on the other hand, can be and is put to a variety of uses including
direct display (with highly flexible display semantics), processing by
network-sensitive Web spiders to support search and retrieval operations, and
reprocessing into a variety of derivative forms.
In general XML-based data formats are more re-usable and repurposable than
the alternatives, although the example of XSL-FO shows that this is not an
absolute.
There are many cases where final-form is an application requirement;
representations which embody legally-binding transactions are an obvious
example. In such cases, the use of digital signatures may be appropriate to
achieve immutability, whether the format is naturally final-form or some XML
vocabulary.
On the other hand, where such requirements are not in play,
representations that are reusable and repurposable are in general higher in
value, particularly in the case where the information's utility may be
long-lived.
Some data formats are explicitly designed to be used in combination with
others, while some are designed for standalone use. An example of a
standalone data format is PDF; it is not typically embedded in
representations encoded in other formats.
At the other extreme is SOAP, which is designed explicitly to contain a
"payload" in some non-SOAP vocabulary. Another example is SVG, which is
designed to be included in compound documents, and which may in turn contain
information encoded in other XML vocabularies.
This characteristic is related to, but distinct from, the
final-form/reusable distinction discussed above. For example, one can
certainly imagine cases where it is useful for a representation to include
data in multiple different formats, but be considered immutable and
display-only.
TAG issue xmlProfiles-29:
When, whither and how to profile W3C specifications in the XML Family?
TAG issue mixedUIXMLNamespace-33:
Composability for user interface-oriented XML namespaces
TAG issue xmlFunctions-34:
XML Transformation and composability (e.g., XSLT, XInclude, Encryption)
TAG issue RDFinXHTML-35:
Syntax and semantics for embedding RDF in XHTML
XML and XML Namespaces allow format designers to create and combine
vocabularies. A format is extensible if instances of the format can
include terms from other vocabularies.
Good practice
Extensibility: Format designers should create
extensible formats.
Versioning is the term for the evolution of languages and documents.
Versioning is achieved through extensibility mechanisms and language
redefinition. When a format definition changes (e.g., by addition or removal
of element or attribute definitions), this creates a new version of the
format. When an instance of a format includes other vocabulary elements, this
creates an extension of the instance; the original format definition has not
changed. An example is a SOAP message with a header block, this is called a
SOAP extension, not a new version of the SOAP format.
The following terms define important relationships among different
versions of a format:
- Compatible formats
- Format version M is compatible with format version N if agents
designed to process all instances of M can also process all instances
of N. The compatibility relationship is not always symmetric.
- Forwards
compatibility
- Format version M is forwards compatible with respect to format
version N if M is compatible with N and M predates N.
- Backwards
compatibility
- Format version M is backwards compatible with respect to format
version N if M is compatible with N and N predates M.
Good practice
Compatibility: Format designers SHOULD define
extensibility models that allow forwards compatible and backwards compatible
changes.
Naturally, even if M and N are compatible but different versions of a
format, agents will process instances of them differently. For instance, if
format version M is forwards compatible with format version N, M processors
that encounter N instances might handle unknown elements by ignoring them
entirely, or by ignoring element tags but continuing to process element
content. Different format specifications may require different compatibility
behavior.
Good practice
Compatibility behavior: Format designers
SHOULD define expected behavior when agents designed to process one version
of a format encounter a compatible version of the format.
In some cases, format designers require that new features be supported
(i.e., not ignored). In this case, the new version of the format (N) may be
backwards compatible with the earlier version (M), but M is not forwards
compatible with N. The SOAP 1.2 Recommendation [SOAP12], for example, defines the
mustUnderstand attribute in section
5.2.3.
For more information on format extensibility, refer to "Web Architecture:
Extensible Languages" [EXTLANG].
In many cases, the information contained in a separation is logically
separable from the choice of ways in which it may be presented to a human,
and the modes of interaction it may support.
While such separation is, where possible, often advantageous, it is
clearly not always possible and in some cases not desirable either.
Replacement text from C. Lilley expected.
One of the greatest strengths of HTML as a format is that it allows
authors to embed cross references (hyperlinks). The simplicity of <a
href="#foo"> as a link to "foo" and <a name="foo"> as the anchor
"foo" are partly (perhaps largely) responsible for the birth of the hypertext
Web as we know it today.
Simple, single-ended, single-direction, inline links are not the most
powerful linking paradigm imaginable. But they are very easy to understand.
And they can be authored by individuals (or other agents) that have no
control or write access to the other end point.
More sophisticated linking mechanisms have been invented for the Web.
XPointer allows links to address content that does not have an explicit,
named anchor. XLink allows links to have multiple ends and to be expressed
either inline or in "link bases" stored external to any or all of the
resources identified by the links it contains.
All of the current common linking mechanisms identify resources by URI and
optionally identify portions (or views) of a resource with the fragment
identifier.
Good practice
Link mechanisms: Format specification designers
SHOULD provide mechanisms for identifying links to other resources and to
portions of representations (via fragment identifiers).
For formats based on XML, format designers should examine XLink and the
XPointer framework for inspiration. To define fragment identifier syntax, use
at least the XPointer Framework and XPointer element() Schemes.
If a future revision of RFC 3023 identifies the XPointer Framework,
element(), and perhaps other ancillary schemes as the fragment identifier
syntax for XML documents, authors will be able to rely on at least those
schemes for all XML documents.
TAG issue: What is the scope of using XLink? xlinkScope-23.
Many resource representations are encoded in formats which are XML
vocabularies. This section discusses issues that are specific to such data
formats.
Anyone seeking guidance in this area is urged to consult the "Guidelines
For The Use of XML in IETF Protocols" [IETFXML] for the use of XML in Internet Protocols. This
document contains a very thorough discussion of the considerations that
govern whether or not XML ought to be used, as well as specific guidelines on
how it ought to be used. While it is directed at Internet applications with
specific reference to protocols, the discussion is generally applicable to
Web scenarios as well.
The discussion here should be seen as ancillary to the content of the IETF
BCP. Refer also to "XML Accessibility Guidelines" [XAG] for help designing XML formats that lower barriers to
Web accessibility for people with disabilities.
XML defines textual data formats that are naturally suited to describing
data objects which are hierarchical and processed in an in-order sequence. It
is widely but not universally applicable for format specifications. For
example, an audio or video format is unlikely to be well suited to
representation in XML. Design constraints that would suggest the use of XML
include:
- Explicit representation of a hierarchical structure.
- The data's usefulness should outlive the tools currently used to
process it.
- Ability to support internationalization in a self-describing way that
makes confusion over coding options unlikely.
- Early detection of encoding errors with no requirement to "work around"
such errors.
- A high proportion of human-readable textual content.
- Potential composition of the data format with other XML-encoded
formats.
Editor's note: Which XML Specifications make
up the XML Family?
The Web is significantly a networked information system. Authors
and applications can use URIs uniformly to identify different resources.
After representations of these resources have been retrieved, they may be
processed in a variety of ways. Some applications (and some users) will
undoubtedly build new resources by combining several representations
together. This is particularly easy, and potentially useful, when XML
representations are available for all the resources.
However, combining representations in this way moves them out of their
original context and places them in a new context. This change of context
introduces the possibility of information loss. Any information that depended
on the local context will no longer be available.
What is needed is a mechanism for establishing a global context for the
elements and attributes in the XML resources. This problem bears a strong
resemblance to the distinction between relative and absolute URIs. While the
many hundreds of relative URI references to "index.html" on a typical web
server may be entirely unambiguous in their respective contexts, they have no
unambiguous global meaning. But each such relative URI has an unambiguous
absolute URI that can be established in its local context and used when a
document is moved. This solves the problem for URI references.
For elements and attributes, their names can be seen as analogous to
relative URI. Within their original context, they have meanings that are
clear and entirely unambiguous. Namespaces in XML provides a mechanism for
establishing a globally unique name that can be understood in any context.
The "absolute" form of an XML element or attribute name is the combination
of its namespace URI and its local name. This is represented lexically in
documents by associating namespace names with (optional) prefixes and
combining prefixes and local names with a colon as described in "Namespaces
in XML" [XMLNS].
Designers that use namespaces are thus providing a global context for
documents authored with their schema. Establishing this global context allows
their documents (and portions of their documents) to be reused and combined
in novel ways not yet imagined. Failure to provide a namespace makes such
reuse more difficult, perhaps impractical in some cases.
The most significant technical drawback to using namespaces is that they
do not interact well with DTDs. DTDs perform validation based on the lexical
form of the name, making prefixes semantically significant in ways that are
not desirable. As other schema language technologies become widely deployed,
this drawback will diminish in significance.
Namespace designers SHOULD make available human-readable material to meet
the needs of those who will use the namespace vocabulary. The simplest way to
achieve this is for the namespace name to be an HTTP URI which may be
dereferenced to access this material. The resource identified by such a URI
is called a "namespace document."
There are many reasons why a person or agent might want more information
about the namespace. A person might want to:
- understand its purpose,
- find out who controls it,
- request authority to access schemas or collateral material about it,
or
- report a bug or situation that could be considered an error in some
collateral material.
A namespace document should also support the automatic retrieval of other
Web resources that support the processing markup from this vocabulary. Useful
information to processors includes:
- schemas, to use for validation,
- style sheets, to use for presentation,
- ontologies, to use for making inferences, or
- any number of other application-specific details.
It follows that there is, in general, no single type of resource that can
be returned in response to a request for the namespace name that will always
be the most appropriate; see the section on future work regarding namespace document formats for more
information.
Issue: namespaceDocument-8:
What should a "namespace document" look like?
Issue: abstractComponentRefs-37:
Definition of abstract components with namespace names and frag ids
Editor's note: Where should we put a section
on mixing namespaces; is the section on processing model more appropriate?
See issue mixedUIXMLNamespace-33.
Suppose that the URI http://example.com/oaxaca defines a
resource with representations encoded in XML. What, then, is the
interpretation of the URI http://example.org/oaxaca#weather?
The URI specification [URI] makes it clear
that the interpretation depends on the context of the media type of the
representation. It follows from this that designers of XML-based data formats
SHOULD include the semantics of fragment identifiers in their designs. The
XPointer Framework [XPTRFR] provides a
syntax designed for in such fragment identifiers, and it SHOULD be used for
this purpose.
When a representation is provided whose media type is
application/xml, there are no semantics defined for fragment
identifiers, and thus they SHOULD NOT be provided for such representations.
This is also the case if the representation is known to be XML because the
media type has a suffix of +xml as described in "XML Media
Types" [RFC3023], but there is no
normative specification of fragment semantics.
It is common practice to assume that when an element has an attribute that
is declared in a DTD to be of type ID, then the fragment identifier
#abc identifies the element which has an attribute of that type
whose value is "abc". However, there is no normative support for
this assumption and it is problematic in practice, since the only defined way
to establish that an attribute is of type ID is via a DTD, which may not
exist or may not be available.
TAG issue fragmentInXML-28:
Do fragment identifiers refer to a syntactic element (at least for XML
content), or can they refer to abstractions? See TAG issue.
TAG issue xmlIDSemantics-32:
How should the problem of identifying ID semantics in XML languages be
addressed in the absence of a DTD? See draft TAG finding
"How should the
problem of identifying ID semantics in XML languages be addressed in the
absence of a DTD?".
RFC 3023 defines the media types application/xml and
text/xml, and describes a convention whereby XML-based data
formats use media types with a +xml suffix, for example
image/svg+xml.
In general, media types beginning with text/ SHOULD NOT be
used for XML representations. They create two problems: First, intermediate
agents in the Web are allowed to "transcode", i.e., convert one character
encoding to another. Since XML documents are designed to allow them to be
self-describing, and since this is a good and widely-followed practice, any
such transcoding will make the self-description false.
Secondly, representations whose media types begin with text/
are required, unless the charset parameter is specified, to be
considered to be encoded in US-ASCII. In the case of XML, since it is
self-describing, it is good practice to omit the charset
parameter, and since XML is very often not encoded in US-ASCII, the use of
"text/" media types effectively precludes this good practice.
The Resource Directory Description Language [
RDDL] is a proposal under discussion in the community for a
variant of XHTML optimized for the construction of namespace documents which
meet the goals described in this section. Note, however, that RDDL or a
document like it, is no more universally correct than any other type of
representation. Namespace developers should give careful consideration to
choosing the most appropriate format for their application, keeping in mind
that both human- and machine-readable information is useful.