This document explains some common mistakes in user agents due to incorrect or incomplete implementation of specifications, and suggests remedies. It also suggests some "good behavior" where specifications themselves do not specify any particular behavior (e.g., in the face of error conditions). This document is not a complete set of guidelines for good user agent behavior.
This document does not incriminate specific user agents. W3C does not generally track bugs or errors in implementations. That information is generally tracked by the vendors themselves or third parties.
This document is an update of an already published Note, published on January 28th,
2003, and made available for discussion only by the editor and authors as part of their work as
W3C Team participants in the Quality Assurance
Activity.
Publication of this Note by W3C does not imply endorsement by W3C,
including the W3C Team and Membership.
This document may be updated, replaced, or obsoleted by other documents
at any time.
No formal commitment is made by W3C to invest additional
resources in topics addressed by this Note. However, comments are welcome
and the W3C Quality Assurance Team
may publish an amended version should the amount and quality
of the received comments prove it worthwhile or necessary. Though some of the previous comments has been added to this version. Some of them are still in discussion and might be added in a future version. We plan to publish in the next few months a new and improved version of this document to have the same organization than the CHIPS note. The problems related to XHTML, DOCTYPE and namespaces will be addressed in this future version.
Please send comments to the
publicly archived
mailing-list of the
Quality Assurance Interest Group:
www-qa@w3.org.
A list of acknowledged
errors and proposed corrections can be found at
http://www.w3.org/QA/2002/12/cuap-errata.
Translation of this document is welcome. However, before
starting a translation of this document, please be sure to read the
information on translations, in our
Copyright FAQ,
and check the
list of existing translations of this document (available at
http://www.w3.org/QA/translations#cuap).
.
A list of current W3C technical
reports and publications, including Working Drafts and Notes,
can be found at http://www.w3.org/TR/.
This document explains some common mistakes in user agents (browsers, spiders, etc.) due to incorrect or incomplete implementation of specifications, and suggests remedies. It also suggests "good behavior" where specifications themselves do not specify any particular behavior (e.g., in the face
of error conditions).
This document only deals with the client-side aspect of
HTTP,
people looking for HTTP implementation problems in Web servers should have a look at the Web server counterpart of this document: Common HTTP Implementation Problems [CHIPS].
This document does not address accessibility issues for user agents. Please refer to W3C's User Agent Accessibility Guidelines 1.0 [UAAG10] for information on how to design user agents that are accessible to people with disabilities.
This document is a set of known problems and/or good practices for user agents implementations and their use, aimed at:
- developers implementing browsers, Web clients,
- developers implementing spiders, proxies, when they act as a client.
Unless specifically mentioned, what is referred throughout this document as
HTTP is RFC2616,
a.k.a.
HTTP/1.1
[RFC2616].
This document is informative.
This document has no conformance per se, but since it is about
implementation of normative specifications (such as the
HTTP/1.1 specification), or their use, one should consider following the guidelines and checkpoints described here as a good step toward conformance to these normative specifications.
As often as possible, references will be mentioned for each checkpoint.
This document uses RFC 2119 [RFC2119] keywords
(capitalized MUST, MAY, SHOULD etc.) when referring to behaviors clearly defined
by a normative specification. When not capitalized, these words should be interpreted as regular language and not as RFC2119 keywords.
This section focuses on the user's experience, including
customization, user interface, and other usability issues. Some of the checkpoints suggested here depends on the user agents used and can be sometimes not applicable in terms of implementations.
- 1.1 When the user follows a link to a target anchor, highlight the target location.
Techniques:
- Put the target location at a consistent location in the viewport (e.g., at the top of a graphical viewport).
- Allow configuration to highlight (e.g., through audio cues, graphically, etc.) the target location. Ensure that highlight mechanisms are distinguishable from other highlight mechanisms.
References:
- see 'Selectors' for information about the CSS and the ':target' selector [SELECTORS]
- 1.2 If the user attempts to follow a link that is broken because it designates a missing anchor, let the user know it is broken.
There are many ways to indicate to the user that a link is broken. The recommended behavior is as follows:
- Do not scroll or otherwise change the viewport. This could make the user believe the link is not broken.
- Indicate to the user (e.g., via a text message in the status bar) that the link is broken. If no message is given to the user, they will not understand why the viewport didn't move.
- Ensure that any non-text message to the user has a text equivalent; text may be rendered as visually displayed text, synthesized speech, and braille. Audio cues or visual cues may be used in addition to text messages.
Wrong: Some user agents scroll to the top or
bottom of the document when the user attempts to follow a broken
link. This behavior is discouraged since it is indistinguishable from the
correct behavior when a target is at the beginning or end of a
document.
References:
- For information about accessible user interfaces, please refer
to the User Agent Accessibility Guidelines 1.0 [UAAG10].
1.3 Provide text messages
1. Ensure that every message (e.g., prompt, alert, or
notification) that is a non-text element and is part of the
user agent user interface has a text equivalent.
- 1.3
Allow the user to retrieve Web resources even if the browser cannot
render them.
User agents may not be able to render certain types of content on
the Web either natively or through a plug-in (e.g., XML content,
XSLT style sheets, RDF documents, DTDs, XML schemas, etc).
User agents should allow users to retrieve and save these resources,
otherwise users may not be able to access this Web content at
all.
- 1.4 When
the user requests to print a frameset, allow the user to select to
print an individual frame or the frameset.
The presentation of the frameset could be achieved, for
example, by:
- proposing a list of frames to the user.
- using a graphical representation of the organization of the frames.
Note: The authors do not encourage Web content
developers to use frames as they can cause many usability and
accessibility problems.
References:
- 1.5
Add support for new URI schemes in a straightforward way.
For instance, allow users to associate external programs with
URI schemes. The user agent should inform the user when it does not
recognize a URI scheme in content.
Example:
A user may want the "tel" scheme (e.g.,
tel:+33-4-12-34) to interact with their
telephone. Or they may want the "irc" scheme (e.g.,
irc://irc.example.org/) to activate an IRC client on
their desktop with a connection to the specified server.
Wrong: Some user agents ignore the scheme part
(before the ":") when the scheme is unknown to them, interpret the
colon character as though it were encoded as '%3A' and then treat the
URI as though it were a relative URI, usually producing a broken link
(and confusing users).
References:
- From section 3 of "Uniform Resource Identifiers (URI):
Generic Syntax" [RFC2396]:
An absolute URI contains the name of the scheme being used
followed by a colon (":") and then a string whose interpretation
depends on the scheme.
- Refer to information about URI schemes in section 3.1 of
"Uniform Resource Identifiers (URI): Generic Syntax" [RFC2396].
- For a list of known URI schemes, see
"An Index of WWW Addressing Schemes" [SCHEMES].
- For a list of the Official IANA Registry of URI Schemes, see
"Uniform Resource Identifier (URI) SCHEMES" [SCHEMES-IANA].
- To register an URI scheme, see
"Registration Procedures for URL Scheme Names" [SCHEMES].
- 1.6 Allow the user
to override any mechanism for guessing URIs or keywords.
Many user agents compensate for incomplete URIs by applying a
series of transformations with the hope of creating a URI that
works. For example, many user agents transform the string
www.w3.org into the URI http://www.w3.org/. The
user should be able to control whether, for example, typing a
keyword should invoke a Web search or whether the user agent
should prepend http://www. and append
.org/.
- 1.7
Warn users about incomplete documents and transfers.
Rendering an incomplete document as though it were complete is
very likely to confuse users. Part of the document is missing, hence
some anchors might not be present, possibly breaking some links. The
user agent should notify the user that the document is incomplete.
The HTTP/1.1 specification describes this behavior for caches at
the protocol level. Partial responses should also be made obvious to
the user with a warning.
References:
- The correct behavior is specified in
section 13.8 of the HTTP/1.1 specification [RFC2616].
A cache MUST NOT return a partial response to a client without
explicitly marking it as such, using the 206 (Partial Content) status
code. A cache MUST NOT return a partial response using a status code
of 200 (OK).
- 1.8
Provide a mechanism to allow authentication information to
expire.
Many browsers allow configuration to save HTTP authentication
[RFC2616, RFC2617] information ("remember my
password"). They should also allow users to "flush" that
authentication information on request. For instance, the user may wish
to leave the user agent running but tell it to forget the password
to access the user's bank account.
Wrong: Most user agents consider that
authentication information (e.g., password) provided by a user for a
server/realm pair during a session is immutable for the duration of
the session.
- 1.9
When a Web resource includes metadata that may be recognized by the
user agent, allow the user to view that metadata.
Metadata – data about data – can provide very useful context
to users about information on the Web. For instance, metadata about
a book might include the book's author, title, publication date,
publisher, etc. (refer to the Dublin Core [DC] for information about library-type
metadata). Authors include metadata in HTML documents through a
variety of elements and attributes (e.g., the TITLE
and ADDRESS elements, the "alt", "title", and
"summary" attributes, etc. Languages such as the Resource
Description Framework [RDF] allow users to
populate the Web with rich metadata. User agents should provide a user
interface to allow users to view metadata. The user interface may
vary according to the underlying markup language. For instance,
many graphical browsers render the HTML "title" attribute (e.g., as
a tool-tip) when the user selects or hovers over an element with
that attribute specified.
References:
- Some projects that address the display of metadata are linked
from the RDF home page at the
W3C Web site.
- 1.10 Allow
the user to keep track of completed HTTP POST requests.
Users may wish to track and archive HTTP POST requests for the
same reasons they wish to track and archive email. For instance, if
the user places a book order through a form, and that form uses a POST
request, the user should be able to store information about that
transaction.
References:
- HTTP/1.1 POST requests are described in section 9.5 of the
HTTP/1.1 specification [RFC2616].
- "Axioms of Web architecture: User Agent watch points"
[UAWP].
- 1.11 Allow the user
to bookmark negotiated resources.
-
The HTTP/1.1 protocol [RFC2616]
allows the client to request a representation of a resource which is
best suited to its needs (language, media type, etc); this mechanism
is called "content negotiation".
When a resource is negotiated, the user might want to bookmark a
particular version. For example, a document might be available in
several languages under the same URI, and the user might want to point
somebody to the Canadian version of this document, which has a different
URI.
In such a case, it should be possible to bookmark either the
original URI or the URI of the view that the user got. The original
URI can be interpreted as being the generic object and the retrieved
document as one view of this object.
Bookmarking a particular version of a
negotiated resource is not always possible under HTTP semantics, because
a) the particular version may not have its own URI and b) even if it does,
HTTP does not guarantee that the user agent will be informed of this.
HTTP/1.1 defines the Content-Location header field as the way for the
server to indicate the URI of the variant, and some servers do supply
this Content-Location when negotiation took place most of the time.
However, Content-Location is also used for some other things, and its
inclusion in a response does not necessarily mean that content
negotiation took place.
References:
- 1.12
Support time-saving transfer encoding mechanisms and send out TE headers announcing their support.
HTTP/1.1 [RFC2616] allows
transfer encoding. An example of encoding is data compression, which
speeds up Web browsing over a slow connection.
The HTTP/1.1 transfer encoding negotiation mechanism has been designed to avoid the need for the end user to get involved. Using the HTTP protocol, the server, proxy, and client implementations among themselves will be able to choose and use the most efficient transfer encoding. The more you support such mechanisms, the better it is.
Users might have enough knowledge or have help of user interfaces to fine-tune this process beyond what can be done automatically. The user agent should allow the user to set the transfer encoding in the HTTP requests sent out.
References:
- Refer to information about the "TE" request header, described
in
section 14.39 of the HTTP/1.1 specification [RFC2616].
- 1.13 Use the user interface language as the default value for language
negotiation.
The user should be allowed to specify the set of languages that
the user agent may use for language negotiation.
In case the user does not specify any language, the user agent may
specify the language of its user interface as the preferred language,
while allowing other languages with a lower preference, for example by
sending
Accept-Language: dk, *;q=0.5
References:
- 1.14 Only advertise an encoding in
Accept-encoding that you really accept.
A number of web sites suffer from bandwidth overload. By altering the server side scripting engine to support encoding compression
or by inserting a compressing proxy, it is possible to dramatically reduce the
operating costs. The down side is that a number of user agents advertise
that they can handle gzip or deflate when they really are unable to do it.
References:
- 1.15 Remember traverse redirects
When the browser traverses a redirect, it should remember both the original URI and the target URI for marking links as visited.
This section focuses on issues related to style sheets and link
types.
- 2.1
Implement user style sheets. Allow the user to select from author and
user style sheets or to ignore them.
A style sheet is a set of rules that specifies how to render a
document on a graphical desktop computer monitor, on paper, as
synthesized speech, etc. A document may have more than one style
sheet associated with it, and users should be able to select from
alternative style sheets.
References:
- 2.2
Respect media descriptors when applying style sheets.
Some markup and style sheet languages allow authors
(e.g., @media construct in [CSS2], media attribute in [HTML 4.01]) to design
documents that are rendered differently according to the
characteristics of the output device: whether graphical display,
television screen, handheld device, speech synthesizer, braille
display, etc.
References:
- 2.3 If a CSS style sheet is
missing, ignore it and continue processing.
Users must be able to view content even without CSS style
sheets.
Wrong: In some user agents, missing style sheets
result in a fatal error or result in the user agent not rendering
content.
References:
- From
section 3.2 of the CSS2 Recommendation, [CSS2]:
For each source document, [a user agent] must attempt to retrieve
all associated style sheets that are appropriate for the supported
media types. If it cannot retrieve all associated style sheets (for
instance, because of network errors), it must display the document
using those it can retrieve.
- 2.4
Implement the HTML 4 recognized link types.
Section
6.12 of the HTML 4.01 Recommendation [HTML 4.01] lists some link types that may
be used by authors to make assertions about linked Web
resources. These include alternate,
stylesheet, start, next,
prev, contents, glossary, and others.
Although the HTML 4.01 specification does not specify definitive
rendering or behavior for these link types, user agents should
interpret them in useful ways. For instance, the start,
next, prev, and contents link
types may be used to build a table of contents, or may be used to
identify the print order of documents, etc.
This section focuses on the implementation of network protocols
used to download resources from the Web.
- 3.1 Save
resources retrieved from the Web on the local system using the
appropriate system naming conventions.
-
The media type of a resource retrieved by HTTP [RFC2616] is determined by the content type
and encoding returned by the server in the response headers.
If the user wants to save a resource locally, the user agent should
respect the system naming conventions for files (e.g. PNG images
usually have a .png extension).
Example:
http://www.w3.org/TR/1999/REC-html401-19991224/html40.ps
is a view of the gzip'ed PostScript version of the HTML 4.01
specification. The HTTP headers sent by the server include:
Content-Type: application/postscript; qs=0.001
Content-Encoding: gzip
If saved locally, the filename on most computers should be
html40.ps.gz for the applications to recognize the file
type.
Wrong: Saving this compressed PostScript document
as html40.ps is likely to confuse other applications.
References:
- 3.2
Respect the media type of a resource if one is explicitly given using
the
Content-Type HTTP header.
Example:
If an HTML document is returned with a Content-Type
value of text/plain, the user agent must NOT render the
document with another guessed Content-Type (like, for example, text/html).
Reference:
- 3.3
Respect the character set of a resource when one is explicitly
given.
-
User agents must respect the character set when it is explicitly
specified in the response. The character set can be given by the HTTP
Content-Type headers and/or by the document-internal
fallback (HTML meta element, etc).
References:
- From
section 3.4.1 of the HTTP/1.1 specification, [RFC2616]:
HTTP/1.1 recipients MUST respect the charset label provided by the
sender; and those user agents that have a provision to "guess" a
charset MUST use the charset from the content-type field if they
support that charset [..].
- From section 5.2.2
of the HTML 4.01 Recommendation, [HTML
4.01]:
To sum up, conforming user agents must observe the following
priorities when determining a document's character encoding (from
highest priority to lowest):
- An HTTP "charset" parameter in a
"
Content-Type" field.
- A META declaration with "
http-equiv" set to
"Content-Type" and a value set for "charset".
- The charset attribute set on an element that designates an
external resource.
- 3.4 Do not
treat HTTP temporary redirects as permanent redirects.
-
The HTTP/1.1 specification [RFC2616] specifies several types of
redirects. The two most common are designated by the codes 301
(permanent) and 302 or 307 (temporary):
- A 301 redirect means that the resource has been moved
permanently and the original requested URI is out-of-date.
- A 302 or 307 redirect, on the other hand, means that the resource has a
temporary URI, and the original URI is still expected to work in
the future. The user should be able to bookmark, copy, or link to the
original (persistent) URI or the result of a temporary redirect.
Wrong: User agents usually show the user (in the
user interface) the URI that is the result of a temporary (302 or 307)
redirect, as they would do for a permanent (301) redirect.
References:
- For more information about HTTP/1.1 response codes 301 and 302,
refer to
section 10.3.2 and
section 10.3.3, respectively, of the HTTP/1.1 specification [RFC2616].
- Refer to "Axioms of Web architecture: User Agent watch points"
[UAWP].
- 3.5 If a
host name has multiple DNS entries, try them all before concluding that the
Web site is down.
-
Many Web sites have a single hostname like www.example.org resolve to
multiple servers for the purpose of load balancing or mirroring. If
one server is unreachable, others may still be up, so browsers should
try to contact all the servers of a Web site before concluding that
the Web site is down.
For example, the libwww implementation does it the right way and check the response time of all ip address, once done it sorts all the address to get the best one. An example is available in the open source implementation of the Domain Name Service Class.
- 3.6 List only
supported media types in an HTTP
Accept header.
-
HTTP/1.1 [RFC2616] defines
content negotiation. The client sending out a request gives a list of
media types that it is willing to accept; the server then returns a
representation of the object requested in one of the specified formats
if it is available.
When entities are embedded in a document (such as images in HTML
documents), user agents should only send Accept headers
for the formats they support.
Example:
If a user agent can render JPEG, PNG and GIF images, the list of
media types accepted should be image/jpeg,
image/png, image/gif.
Wrong: User agent agents should not send an HTTP
header of Accept: */* since the server may support
content types that the user agent does not. For instance, if a server
is configured so that SVG images are preferred to PNG images, a user
agent that only supports PNG, GIF, and JPEG will receive (unsupported)
SVG rather than (supported) PNG.
Note: Some user agents send a Accept header that has '*/*' at the end, after all of the supported content types. This way, the server is free to send the resource in any format, which can then be processed by the user with another tool.
References:
Resources are located on the Web using Uniform Resources
Identifiers [RFC2396]. This
section discusses how user agents should handle URIs.
- 4.1 Handle the
fragment identifier of a URI when the HTTP request is redirected.
When a resource (URI1) has moved, an HTTP redirect
can indicate its new location (URI2).
If URI1 has a fragment identifier #frag,
then the new target that the user agent should be trying to reach
would be URI2#frag. If URI2 already has a
fragment identifier, then #frag must not be appended and
the new target is URI2.
Wrong: Most current user agents do implement HTTP
redirects but do not append the fragment identifier to the new URI,
which generally confuses the user because they end up with the wrong
resource.
References:
- HTTP redirects are described in section 10.3 of the HTTP/1.1
specification [RFC2616].
- The required behavior is described in detail in "Handling of
fragment identifiers in redirected URLs" [RURL].
- The term "Persistent Uniform Resource Locator (PURL)"
designates a URL (a special case of a URI) that points to another
one through an HTTP redirect. For more information, refer to
"Persistent Uniform Resource Locators" [PURL].
Example:
Suppose that a user requests the resource at
http://www.w3.org/TR/WD-ruby/#changes and the server
redirects the user agent to http://www.w3.org/TR/ruby/.
Before fetching that latter URI, the browser should append the fragment
identifier #changes to it:
http://www.w3.org/TR/ruby/#changes.
The editor would like to thank the following W3C Team members for the initial input that led to the creation of this document. Hugo Haas has been the main author of the first version of this document:
The editor would also like to thank the following people for their review of
the document:
- CC/PP
- "Composite
Capability/Preference Profiles (CC/PP): A user side
framework for content negotiation", Franklin Reynolds, Johan
Hjelm, Spencer Dawkins, Sandeep Singhal, 27 July 1999. Available at
http://www.w3.org/1999/07/NOTE-CCPP-19990727/.
- CHIPS
- Common
HTTP Implementation Problems,
Olivier Thereaux, 28 January 2003. Available at http://www.w3.org/TR/2003/NOTE-chips-20030128/.
Latest version available at http://www.w3.org/TR/chips.
- CSS2
- "Cascading
Style Sheets, Level 2", Bert Bos, Håkon Wium
Lie, Chris Lilley, Ian Jacobs, 12 May 1998. Available at
http://www.w3.org/TR/1998/REC-CSS2-19980512/.
- DC
- Dublin Core. Available
at http://www.dublincore.org/.
- HTML 4.01
- "HTML
4.01 Specification", Dave Raggett, Arnaud Le Hors, Ian Jacobs,
24 December 1999. Available at
http://www.w3.org/TR/1999/REC-html401-19991224/.
- PURL
- "Introduction to
Persistent Uniform Resource Locators", Keith Shafer, Stuart
Weibel, Erik Jul, Jon Fausey. Available at
http://purl.oclc.org/OCLC/PURL/INET96.
- RDF
- "Resource
Description Framework (RDF) Model and Syntax
Specification", Ora Lassila, Ralph R. Swick, 22 February 1999.
Available at
http://www.w3.org/TR/1999/REC-rdf-syntax-19990222/.
- RFC1630
- "Universal
Resource Identifiers in WWW", T. Berners-Lee, June 1994.
Available at http://www.ietf.org/rfc/rfc1630.txt.
- RFC2119
-
"Key words for use in RFCs to Indicate Requirement Levels",
S. Bradner, March 1997. Available at
http://www.ietf.org/rfc/rfc2119.txt
- RFC2396
- "Uniform Resource
Identifiers (URI): Generic Syntax", T. Berners-Lee et al.,
August 1998. Available at http://www.ietf.org/rfc/rfc2396.txt.
- RFC2616
- "Hypertext
Transfer Protocol -- HTTP/1.1", R. Fielding et al., June 1999.
Available at http://www.ietf.org/rfc/rfc2616.txt.
- RFC2617
- "HTTP
Authentication: Basic and Digest Access Authentication",
J. Franks et al., June 1999. Available at
http://www.ietf.org/rfc/rfc2617.txt.
- RFC2717
- "Registration Procedures for URL Scheme Names",
R. Petke et al., November 1999. Available at
http://www.ietf.org/rfc/rfc2717.txt.
- RURL
- "Handling
of fragment identifiers in redirected URLs", B. Bos, 30
June 1999. Available at
http://www.ics.uci.edu/pub/ietf/http/draft-bos-http-redirect-00.txt.
- SCHEMES
- "An Index of WWW
Addressing Schemes", Dan Connolly, 2000. Available at
http://www.w3.org/Addressing/schemes.
- SCHEMES-IANA
- "Uniform Resource Identifier (URI) SCHEMES", IANA. Available at
http://www.iana.org/assignments/uri-schemes.
- SELECTORS
- "Selectors", Daniel Glazman et al., 13 November 2001. Available at
http://www.w3.org/TR/2001/CR-css3-selectors-20011113/.
- UAAG10
- "User
Agent Accessibility Guidelines 1.0", Jon Gunderson, Ian Jacobs,
17 December 2002. Available at
http://www.w3.org/TR/2002/REC-UAAG10-20021217/.
- UAWP
- "Axioms of
Web architecture: User Agent watch points", Tim Berners-Lee,
1998. Available at http://www.w3.org/DesignIssues/UserAgent.
- XML-STYLE
- "Associating
Style Sheets with XML documents Version 1.0", James
Clark, 29 June 1999. Available at
http://www.w3.org/1999/06/REC-xml-stylesheet-19990629/.