W3C

Identifying Application State

TAG Finding 01 December 2011

This version:
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState-20111201
Latest version:
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState
Previous versions:
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState-20111130
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState-20110930
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState-20110910
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState-20110830
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState-20110715
http://www.w3.org/2001/tag/doc/IdentifyingApplicationState-20110515
http://www.w3.org/2001/tag/2011/03/HashInURI-20110331
http://www.w3.org/2001/tag/2011/01/HashInURI-20110228
http://www.w3.org/2001/tag/2011/01/HashInURI-20110115
http://www.w3.org/2001/tag/2010/12/HashInURI-20101231
http://www.w3.org/TR/2009/WD-hash-in-uri-20090415/
Editors:
T.V. Raman, Google raman@google.com
Ashok Malhotra, Oracle ashok.malhotra@oracle.com

Abstract

As the Web has evolved from a Web of documents to a Web of applications, techniques for designing applications so that application state and document views of application state can be identified and bookmarked have evolved correspondingly. Originally introduced as a static "fragment identifier" to identify locations in a document, the hash sign, #, is now being used to identify application states as well as in other interesting ways, for example, by SVG and PDF to select from and render documents and as arguments to Web applications that are interpreted by JavaScript. Fragment identifiers are used to provide several different kinds of parameters to the client-side application, such as the actual URI of a video to be played to a video player, or the position and zoom to a map. In many widely deployed browsers, changing the scheme, path or query string in a URI causes an unconditional page reload, but changes to the fragment identifier do not: the characters in the URI bar after the # can be changed without incurring the overhead of reloading the page. Applications and toolkits using fragment identifiers in this way often go to some effort to maintain a history and make sure the back button works as expected. Accessibility and search can, however, be compromised, because without running JavaScript, the URI has no meaning. Such uses of the "fragment identifier" have interesting and different properties, and the usage differs from the way it is described in existing specifications. Recently added functionality in [HTML5] (history.pushState() and history.replaceState()) allows browser history to be changed without causing a page reload thereby providing an alternative to the use of fragment identifiers to identify application state.

Many Web applications are used to present or edit documents. Such applications should be designed so that the documents are readily identified with URIs that can be used for linking from other Web pages or transmitted in e-mails or used in copy/paste situations.

This document explores the issues that arise from these new uses of fragment identifiers and attempts to define best practices. We argue that, in many cases, the use of query parameters along with the new HTML5 functionality mentioned above is preferable to fragment identifiers to identify application state.

Status of this Document

This document has been produced by the W3C Technical Architecture Group (TAG). The TAG approved this finding at its 01 December 2011 teleconference. Additional TAG findings, both accepted and in draft state, may also be available. The TAG may incorporate this and other findings into future versions of the [Architecture of the World Wide Web]. Please send comments on this finding to the publicly archived TAG mailing list www-tag@w3.org (archive).

Table of Contents

1 Introduction
2 Use Case Scenarios
    2.1 Addressing Into Multimedia Streams -- CNN
        2.1.1 Things to Note
        2.1.2 Extrapolating From This Pattern
    2.2 Creating URIs for Media Fragments
    2.3 Interaction State and Browser History
    2.4 AJAX Libraries And State Management
    2.5 Other Examples
    2.6 The Naked Hash-Ref
3 Managing Browser History
4 Architectural Questions
5 Recommended Best Practices
    5.1 Use URIs to Identify Web Application States
    5.2 Use URIs to Identify Documents in Web Applications
    5.3 Manage the History
    5.4 Structure of the State or Document URI
6 Should Existing Specifications be Updated to Cover New Usage?
    6.1 New Specifications
    6.2 Existing Specifications
7 Conclusions
8 References


1 Introduction

[RFC 3986] defines the character string following the ? sign in a URI as the "query component". The character string following the # sign is known as the "fragment identifier" and used to address specific locations in a document. Nearly twenty years later a strong set of conventions has developed around how URI parameters are used. As transactional applications began moving on to the Web in the late 1990's, query parameters formed a core building block for how application state was communicated between client and server. In this phase of Web evolution, clients were still comparatively simple, and client-side URI parameters did not move beyond the use of fragment identifiers to identify parts of the document.

With Web 2.0 applications increasingly moving traditional client-side applications to the Web and highly interactive applications being built using Web parts (HTML, CSS and JavaScript component resources that are themselves Web addressible, see [TVRaman CACM 2009]), fragment identifiers are being used in a variety of ways quite different from their originally intended use. Users of the Web are beginning to discover and codify design patterns based on fragment identifiers for many of these use cases.

This document explores the issues that arise in this context, and attempts to define best practices that help Web applications to:

In this document we use the term "URI" to include URLs (as used in the [HTML5] specification), and IRIs (see [RFC 3987] and [RFC 3987bis]).

2 Use Case Scenarios

This section discusses several Web applications that identify application state using fragment identifiers or query parameters and discusses the consequences of that choice.

2.1 Addressing Into Multimedia Streams -- CNN

When publishing multimedia streams, there is often a need to address into specific points in the multimedia stream, e.g., by using a time-index. The simplest means of doing this is to pass in the start-time as a query parameter in the URI to the server, e.g. http://www.example.com/media.stream?start=03:06:09 and have the server start streaming the content starting at the specified time: 3 hours, 6 minutes and 9 seconds into the content in this example. This technique can be used to create distinct URIs for each point in the media stream and these URIs can be used to bookmark locations of interest.

It is also possible to leverage client-side parameters encoded as part of the URI (using a # ), where this pseudo fragment identifier is used by client-side scripts as an argument to be passed to an appropriate locator function. Consider the following example taken from cnn.com:

<a href="http://www.cnn.com/video/#/video/tech/2008/02/19/vo.aus.sea.spider.ap"> Giant sea spider filmed deep underwater </a>

CNN uses links like the above for all the topical video segments that are published on its site. The URI in this case has the following components:

Component Value
Protocol http
Host www.cnn.com
Path video
Client Parameter #/video/tech/2008/02/19/vo.aus.sea.spider.ap

2.1.1 Things to Note

The browser does a GET on the URI leading up to the # sign, and the processing application, in this case, the JavaScript embedded in the HTML response, processes the portion of the URI following the #. Note that in the general case, the JavaScript function that eventually processes the client parameter may not have been present in the original HTTP response. It may come from a JavaScript library that was loaded as the result of a subsequent HTTP GET request or as a result of a script in the text/html response.

The fragment identifier in this use case is intentionally referred to as a client parameter. Treating it as a regular fragment identifier in this usage would result in incorrectly inferring that the URI for the video resource being addressed is http://www.cnn.com/video. This would result in all the video links on the CNN site getting the same URI. Thus, the entire URI in this case is http://www.cnn.com/video/#/video/tech/2008/02/19/vo.aus.sea.spider.ap. A consumer of this URI who goes looking for an id within the response that matches the #-suffix of this URI will fail. The reported Content-Type for the resource is text/html. However, the behavior of the #-suffix in this case is not defined by the HTML media type registration.

As used, the #-suffix is a first-class client parameter in that it gets consumed by a script that is served as part of the HTML document returned by the server upon receiving a GET request. This embedded script examines the URI available to it as script variable content.location, strips off the # and uses the rest of the suffix as an argument to a function that generates the actual URI. Having constructed this content URI, the script then proceeds to instruct the browser to play the media at the newly constructed location.

2.1.2 Extrapolating From This Pattern

The CNN example cited above is not unique with respect to its use of a fragment identifier within the URI for encoding parameters to the receiving application. It shows that in a world of dynamic documents, the traditional fragment identifier need no longer be an idref (see [XML Schema Datatypes]) value that addresses an existing node in the serialized HTML making up the HTTP response. In addition to possibly being a static idref, the fragment identifier in the URI, the pattern demonstrated here and in other uses cases discussed in this document generalizes to the following:

  • An idref to a dynamically generated node.
  • A parameter to be consumed by the application that is delivered as the HTTP response to the original GET request.

Also, modifying the behaviour using only the fragment identifier allows caching and pushing content through Content Delivery Networks which is a nice property.

Such uses of fragment identifiers have become popular and fulfill a need but are not documented in existing specifications. See 6 Should Existing Specifications be Updated to Cover New Usage? for further discussion.

2.2 Creating URIs for Media Fragments

The Media Fragments Working Group at W3C is developing a specification to address spatial and temporal media fragments on the Web: [Media Fragments URI 1.0]. As the title of the specification states, the objective is to create URIs for media fragments. This is done by using fragment identifiers. For example: http://www.example.org/video.ogv#t=60,100. In this case, the user agent knows that the primary resource is http://www.example.org/video.ogv and that it is expected to display the portion of the primary resource that relates to the fragment #t=60,100, i.e. seconds 60 to 100. The complete syntax is very rich and allows media fragments to be identified along a number of different dimensions. The fragment identifiers are constructed as name-value pairs separated by the & sign.

The relationship between the primary resource and the secondary resource is, in these cases, quite complicated. Selection of the fragment may involve decompression, mapping fragment ranges to byte offsets and other complex calculations. Depending on the format and the capabilities of the user agent, some of this may be done at the user agent and some at the server. Retrieval of the primary resource may be accomplished using several requests to conserve bandwidth and adapt to network conditions.

In some cases, for optimization reasons or because the user agent cannot perform the fragment to byte mapping, only the bytes required for the fragment are retrieved directly from the server. HTTP header extensions are used to accomplish this.

In summary: the [Media Fragments URI 1.0] specification does create URIs for media fragments using fragment identifiers but the derivation of the secondary resource from the primary resource is complex and specialized to the type of the media stream and the storage format. This is discussed further in 6 Should Existing Specifications be Updated to Cover New Usage?.

2.3 Interaction State and Browser History

A variety of methods are available in Web Architecture to save application state. Cookies store information on the client side that is sent along with the GET request. Similarly, data can be stored on the server-side -- in a database, for example, identified by a cookie -- and can be used to change the details of the GET request. There are also specifications under development (See [Web Storage]) that extend the cookie mechanism in several directions. These specifications allow large amounts of data to be stored on the client and can also be used to encode application state.

These mechanisms, however, encode private applications states. In some cases, an application may want to allow selected states to be made public and shareable. For this we require a URI, appropriately decorated with client-side and server-side parameters. The challenge in designing a mechanism to encode state is to preserve the familiar user experience especially to make the back button do the right thing. For live examples of this design pattern, see [GMail] and [Google Maps] both of which take extreme care to ensure that the user's expectations of Web interaction are preserved. These applications use iframe proxies to achieve the desired effect.

A very early interactive Web application was the [Xerox Parc Map Viewer]. When you bring up the application it shows you, by default, a map of the world. If you select a spot on the map it changes to show you a map centered on the selected spot. Users can interact with the map in various ways: pan, zoom, select degree of detail, etc. Each interaction is encoded as a parameter in a URI which is sent back to the server. The server generates a new map from this URI and refreshes the page. The format of the URI is:

http://hostname/map/option=value/.../option=value

This application was created before JavaScript or AJAX Toolkits and essentially, it creates maps as documents, one for each URI and retains the Web paradigm of displaying a document for each URI.

[Google Maps], though many times more functional than the simple Map Viewer attempts to retain the same paradigm. Maps are displayed as documents and each interaction with a map or request for more information, such as nearby hotels, generates and displays a different map/document which has its own URI.

If you point your browser at http://maps.google.com/ an an HTML page is returned. This page loads the maps API which fetches the JavaScript. The JavaScript then determines the location to be displayed, depending on whether Geolocation is enabled, whether a default location has been set, etc. Based on this, an [XMLHttpRequest] is made to the server to pull down the tiles for the default map and these are used to generate the display. This is how the standard AJAX implementation of Google Maps works. For mobile devices there is a lighter-weight implementation that does not use JavaScript and works a bit differently.

If you work with Google Maps you will notice that even after you have customized the map by entering a different location, adding nearby attractions or scrolling, panning or zooming, the address bar has not changed - it still says http://maps.google.com/. If you want a link to the displayed map, you click the "Link" button on the right and it gives you a URI to the map displayed. For example:

http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=212+Hessian+Hills+Rd,+Croton-on-Hudson,+NY+10520&sll=37.0625,-95.677068&sspn=32.527387,51.679688&ie=UTF8&hq=&hnear=212+Hessian+Hills+Rd,+Croton-on-Hudson,+Westchester,+New+York+10520&z=16

Notice the structure of this URI; it includes the address as well as other parameters encoded as a long query parameter string but no fragment identifier. For performance reasons, some of the map manipulation such as panning, zooming and scrolling is done on the client side. If new information is requested to embellish the map, such as traffic, this requires a trip to the server. Thus, the map that you see is generated by JavaScript on the client, based on information gathered from the server, but when you use the "Link" button, the URI that is generated encapsulates all the information and the manipulations as query parameters. This is clearly a conscious choice. The URI with the query parameters is always sent to the server and, thus, it works even for clients that do not support JavaScript or have JavaScript disabled or have other limitations such as small screen sizes or low bandwidth. The information that you get back may be different for different devices but you always get something useful. If, on the other hand, some of the map manipulations were encoded as fragment identifiers then the fragment identifier part would not be sent to the server with the consequence that many maps would have the same URI and would be indistinguishable. See the discussion in 2.1.1 Things to Note re. video fragments.

[GMail], on the other hand, uses fragment identifiers to encode the state and passes them as parameters to the JavaScript that processes your mail. For example, https://mail.google.com/mail/?shva=1#inbox/12c7e6abbc328af4 identifies the inbox and a specific piece of mail in the inbox. If a piece of mail is not selected, the fragment identifier merely identifies the inbox: https://mail.google.com/mail/?shva=1#inbox Note that this "URI" only works for your mailbox. It cannot be mailed to someone else be and used by them to address into their mailbox.

The mechanisms used to implement this functionality are discussed below in more detail.

2.4 AJAX Libraries And State Management

AJAX applications use features of Dynamic HTML (DHTML) to create highly reactive user experiences. Updates to the Web user interface in response to user actions no longer require a full page reload. Consequently, the user can perform a sequence of interaction steps while remaining on the same page at least as seen from the browser's perspective of content.location. This makes for a good user experience, however, additional facilities must be provided by the application for the following:

  • Recording key points in the interaction flow, e.g., for bookmarking.
  • Supporting intuitive behavior for the browser's history mechanism.
  • Snapshotting interaction state to enable returning to a partially completed task at a later time.

Today, many of the details of AJAX programming have been abstracted away by higher level toolkits such as [Dojo AJAX Toolkit] and [Google GWT]. Management of interaction state and browser history is one of the key affordances implemented in these libraries. History mechanisms in AJAX libraries like GWT and Dojo share a lot in common, and the approach can be traced back to [Really Simple History RSH].

The basic premise is to keep track of the application's internal state in the URI fragment identifier. Here the mantra "give everything a URI" can beneficially be extended to Web applications that use active content. This works because updating the fragment identifier to change the state doesn't typically cause the page to be reloaded. This approach has several benefits:

  • Until recently it was about the only way to control the browser's history reliably. New functionality introduced in [HTML5] (history.pushState() and history.replaceState()) now allows browser history to be changed without causing a page reload. See below.
  • It provides good feedback to the user.
  • It is bookmarkable i.e., the user can create a bookmark to the current state and save it, email it, or whatever.

One of the techniques that is used to provide this functionality is to open a number of frames within a browser window. In such an architecture, parent and child frames are allowed to change each others' location URI as long as the frames display information from the same domain or have agreed to collaborate by some other means. Otherwise, changing a frame's location URI opens up a cross-site scripting vulnerability. If the frames can collaborate, then one of the frames, say the parent, passes data to the child via a fragment identifier by resetting the child's location URI. Thus, given a parent frame P and a child frame C , where the location URIs U_P and U_C may come from different domains, the parent frame might pass data to the child by resetting its location URI to U_C#data; the child picks up this data by polling for changes in its location URI. This technique is used in [Comet Programming]. As an example, the [Dojo AJAX Toolkit] uses an IFrame proxy to enable cross-domain XML HTTP Requests. this is a useful technique when writing cross-site mashups. As an example, see [XKCD and AxsJAX], a cross-site mashup that mashes together XKCD comics with their associated transcripts to create a speech-friendly XKCD experience.

2.5 Other Examples

When applications are built from Web parts, there is often a need to configure them when the application is launched. Traditional applications would call these default start-up or command-line options. We see the equivalent emerging for configuring desktop gadgets and widgets where command-line options are passed in via URI parameters. In this context, the URI is the Web command-line. For one sample implementation and its associated usage, see [Using URIs to Pass Parameters to Web Applications]. Dave Raggett's [HTML Slidy] uses URIs of the form ...#(nn) to address into a deck of slides.

Similarly, [Superfeedr] allows you to subscribe to a fragment of a document using a fragment identifier, for example, http://www.nytimes.com/weather#.wCurrent%20.summary. [Addrable] uses a fragment identifier to address into a dataset consisting of comma-separated values.

2.6 The Naked Hash-Ref

In some situations a single # sign is used as the value of the href attribute on HTML anchors. This can be thought of as a relative URI with a null fragment identifier. Web sites wishing to override the default-target behavior of anchors used this for compatibility with older browsers when attaching a JavaScript mouse-click event-handler to anchor elements. This at one time best practice is now considered outdated, though still commonly used. Note that this idiom also creates significant hurdles for users who do not use the mouse for navigation.

3 Managing Browser History

As discussed above, JavaScript applications can be designed to capture reproducible states using fragment identifiers. The characters following the hash mark, #, serve as arguments to the JavaScript. In such cases, it is also desirable to change the URI in the address bar so URIs can be emailed and bookmarked and the back-button works as expected.

In JavaScript the location.href property can be used to get and set the URI and the location.host, location.hostname, location.path, location.pathname, location.protocol and location.hash can be used to get and set various parts of the URI. These methods work in conjunction with the window.onpopstate. event. See, for example, [Manipulating Browser History]. Tracking the onhashchange event enables code that reacts to changes in the fragment identifier. Changes to any part of the URI, other than the fragment identifier, causes the page to be reloaded.

More recently, [HTML5] introduced the history.pushState() and history.replaceState() methods, which allow you to add and modify history entries, respectively. These new facilities have an interesting consequence: the path, query or fragment identifier portion of a URI in the address bar can be changed without causing a page reload. Thus, an application can choose to identify a state with a URI that includes query parameters or a different path instead of changing the fragment identifier. This has several advantages. In situations where the URI is used in another context or with another tool, the entire URI is transmitted to the server and the server can respond in a manner tailored to the capabilities of the client and do something sensible whether the client supports JavaScript or not. If the client does support JavaScript, the query parameters can be sent to the client for processing.

The methods, history.pushState() and history.replaceState() are relatively new and browser uptake is still developing. Currently, they are supported by Google Chrome, Mozilla Firefox and Apple Safari but not by Microsoft Internet Explorer 9 (IE9) or Opera. This is a problem because if you use these facilities your website will not work correctly with older browsers and, for now, with IE9 or Opera.

4 Architectural Questions

Web applications that want to identify certain states that can be bookmarked and reproduced must decide whether to mint URIs for them using fragment identifiers or query parameters. Before [HTML5] introduced pushState() and history.replaceState() the answer was clear: fragment identifiers could be changed without causing a page reload. Most of the use cases discussed here chose that route. The disadvantage of this solution is that clients have to support JavaScript and the states identified by the fragment identifier are not visible to search engines and web crawlers. As we have discussed, Google Maps goes to great lengths to get around this limitation by providing the "Link" button which mints URIs for map states using query parameters.

For applications that use fragment identifiers to pass parameters to JavaScript, what happens if the receiving client does not implement JavaScript, or has scripting turned off due to security or performance concerns? For the same URI, the behavior of user-agents that do not execute the embedded JavaScript is likely to be very different from user-agents that do. Notice, further, that the HTTP response headers do not give the client any indication that this is likely to be so. Until now, URIs have been equally useful to browsers and other kinds of user agents with different capabilities. This pattern demonstrates a case where the behavior for the same URI is different depending on whether scripting is supported. As discussed in the CNN case above, a user-agent with scripting disabled that receives a URI of the form http://www.cnn.com/video/#/video/tech/2008/02/19/vo.aus.sea.spider.ap, and sees a Content-Type of text/html might incorrectly assume that the URI for this video resource is http://www.cnn.com/video.html.

[Jeni Tennison's Blog] discusses in detail the behavior of two applications that use fragment identifiers in this way and shows how their behavior is different for agents that do and do not support scripting. She also quotes statistics that 2% of users do not have or choose to turn off JavaScript. We could choose to ignore this philistine minority but there is a deeper problem. Search engines, web crawlers and the like do not support JavaScript and so states identified by fragment identifiers do not get scanned by the Google search engine, for example. To get around this, Google introduced the #! convention which converts fragment identifiers into server-side parameters. This is discussed below.

With the introduction of history.pushState() and history.replaceState() the choice between # and ? becomes less clear. Using these facilities, states can be identified by URIs that include query parameters (or URI paths) and these URIs can be pushed onto the history stack and bookmarked without causing a page reload. This has several advantages. In situations where the URI is used in a different context, the entire URI is sent to the server and the server can decide, based on the client's capabilities, how to respond. Search engines and web crawlers work as expected. The only problem is that, as discussed above, these features are currently not supported by all browsers.

See also 6 Should Existing Specifications be Updated to Cover New Usage?

5 Recommended Best Practices

A common application pattern that emerges from the Web application use cases discussed in this document is that the browser does a GET on a URI and retrieves some JavaScript. This retrieves more information and then, guided by the client-parameters, processes it to create the display on the browser. As the user interacts with the application, the display may change and still more information may be fetched. Applications may want to identify some of these states so they can be shared and reproduced. The questions that this finding addresses are: how should these states be identified and how can the user's Web experience be preserved e.g. the back button works correctly?

RECOMMENDATIONS FOR APPLICATION DEVELOPERS:

5.1 Use URIs to Identify Web Application States

Applications should be designed so that as the state of the resource and the display changes, sharable, reproducible states can be identified by URIs.

This a special case of "use URIs to name things". The state of the application changes as you manipulate the user interface and information is fetched from the server. You can name the resulting state with a URI or you can name it some other way. If you use a URI then you have a link that you can move out of the page.

5.2 Use URIs to Identify Documents in Web Applications

Many Web applications are used to display or manage documents. For example, a Web application might be used to manage recipes, maps, musical scores, or documents of historical interest. For such applications [Architecture of the World Wide Web] provides the following guidance:

"A resource should have an associated URI if another party might reasonably want to create a hypertext link to it, make or refute assertions about it, retrieve or cache a representation of it, include all or part of it by reference into another representation, annotate it, or perform other operations on it."

Users of a Web application must be able to discover the URI for a given document managed or displayed by that application and then use it in all the different ways that URIs can be used. As discussed above in 2.3 Interaction State and Browser History and below in 5.4 Structure of the State or Document URI, the options available for doing this depend in part on whether fragment identifiers are used in the URI and whether support can be assumed for new HTML5 features such as history.pushState() and history.replaceState().

5.3 Manage the History

Once the state is identified with an URI, the address bar should be updated to reflect the change as discussed above in 3 Managing Browser History so that the user's Web experience is preserved.

5.4 Structure of the State or Document URI

The URIs assigned to identify application states or individual documents can either use query parameters or URI paths or they can use fragment identifiers.

  • If the URI uses query parameters or URI paths then, to avoid a page reload, history.pushState() and history.replaceState() have to be used to manage the browser history. This has the advantage that if the URI is moved and reused the entire URI is transmitted to the server and the server can make intelligent decisions based on the capabilities of the client. It has the disadvantage that these new capabilities may not be supported by all browsers and that may limit the reach of the website.

    Google Maps takes a somewhat different approach. When running in a browser, it uses JavaScript to manipulate the display and respond to user commands. As the display changes the address bar does not change, however. If the user wants a URI for a particular map she can use the "Link" button and Google Maps generates a URI for that map using query parameters. Google Maps was designed several years ago and its use of query parameters has the advantage discussed above, i.e., that the resulting URIs work equally well in clients that support JavaScripot and clients that do not support JavaScript. It has the disadvantage that the address bar does not update in the obvious way to identify the document being viewed, and that forward/back buttons may not always do what users would expect.

  • Using fragment identifiers for the generated URI provides some advantages in performance and caching but has a significant limitation in that the application will not run correctly, or run at all, on user-agents that do not run JavaScript. Another problem is that the states generated by JavaScript cannot be tracked by search engines and web crawlers. To get around this problem Google has invented the #! convention. See [Making AJAX Applications Crawlable]. If you use this convention then client-side fragment identifier parameters are converted into server-side parameters for the search engine. So far, this special convention works only for Googlebots. If history.pushState() and history.replaceState() become generally accepted it may wither away. [Jeni Tennison's Blog] discusses the #! convention in more detail.

6 Should Existing Specifications be Updated to Cover New Usage?

Not surprisingly, the use of fragment identifiers to identify application states and pass parameters to JavaScript code is not covered by existing specifications. [RFC 3986] says: "The fragment identifier component of a URI allows indirect identification of a secondary resource by reference to a primary resource and additional identifying information. ... The semantics of a fragment identifier are defined by the set of representations that might result from a retrieval action on the primary resource. The fragment's format and resolution is therefore dependent on the media type [RFC2046] of a potentially retrieved representation ..."

For HTML and XML the fragment identifier processing rules are defined in [RFC 2854] and [XPointer] respectively. Essentially, the fragment identifier is a pointer into a document. If the value of the fragment identifier equals the id of an element in the document, then it identifies that element.

In the CNN example, the fragment identifiers designate no element, and, thus, its fragment processing violates [RFC 2854]. If the fragment identifier identified an element, it would generate a conflict between the element identified, and the JavaScript code handling the fragment.

The media fragments case is a bit different. Most of the media type specifications do not specify fragment identifier processing rules. Thus, they do not explicitly violate [RFC 3986] although the relationship between primary and secondary resources is quite a bit more complex than [RFC 3986] describes. In some cases they fetch the fragment directly from the server rather than selecting it on the client.

This document discusses highly interactive applications built using JavaScript and Ajax libraries that run on the client and use fragment identifiers as arguments to functions and in other ways that were not envisaged by [RFC 3986]. Thus, [RFC 3986] needs to be revised to take into account these new usage patterns and emerging functionality. And, perhaps, there should be different standards for browsers that support/run JavaScript and those that do not.

RECOMMENDATIONS FOR MEDIA TYPE REGISTRATIONS:

In order to ground the semantics of fragment identifiers in applicable specifications, it is recommended that IETF media-type registration procedures be updated as suggested below.

6.1 New Specifications

All media type specifications and registrations, especially for new types, must specify fragment identifier semantics for both static use and use in active content as appropriate. The text/html and application/xhtml+xml media types defined for HTML5 need to define the use of fragment identifiers with active content.

6.2 Existing Specifications

For media types that accept "active content", like HTML and SVG, the definition should be extended to acknowledge the fact that fragment identifiers might also be used (if not in contradiction with the 'static' use of those fragment identifiers) for programmatic purposes. The media type registration needs to say how fragment identifiers are used as parameters by the active content and how they may be used to identify the portion of the state that is reproducible and can be referenced externally.

7 Conclusions

As the Web has evolved from showing things to doing things i.e. from from showing documents to running applications where the applications use code running on the client to construct the display by retreiving and manipulating bits of information gathered from several sources, the need has arisen to identify application states that are reproducible and also to identify individual documents, such as maps or emails, managed by Web applications. Such identifiers must be shareable and, where appropriate, usable for linking on the Web. This finding recommends that these states should be identified by URIs and, further, that the browser history should be managed so that users' expectations and the Web paradigm are preserved. The URIs for the application states can be constructed using query parameters or paths or using fragment identifiers. The pros and cons of these different approaches are discussed in this finding.

8 References

Addrable
Addrable.Michael Hausenblas (See https://github.com/mhausenblas/addrable.)
Architecture of the World Wide Web
Architecture of the World Wide Web I. Jacobs, N. Walsh, W3C, December 2004 (See http://www.w3.org/TR/webarch/.)
Comet Programming
Comet Programming from Wikipedia (See http://en.wikipedia.org/wiki/Comet_(programming).)
Dojo AJAX Toolkit
The Javascript Toolkit by the Dojo Foundation. (See http://dojotoolkit.org/.)
GMail
GMail (See http://mail.google.com/.)
Google GWT
Google Web Toolkit -- Java software development framework that makes writing AJAX applications like Google Maps and GMail easy for developers taking care of browser and platform details. (See http://code.google.com/webtoolkit/.)
Google Maps
Google Maps (See http://maps.google.com/.)
HTML5
HTML5: A vocabulary and associated APIs for HTML and XHTML Ian Hickson (See http://www.w3.org/TR/html5.)
HTML Slidy
HTML Slidy: Slide Shows in HTML and XHTML Dave Raggett (See http://www.w3.org/Talks/Tools/Slidy2/Overview.html.)
Jeni Tennison's Blog
Jeni Tennison on Hash URIs (See http://www.w3.org/QA/2011/05/hash_uris.html.)
JSON with Padding
JSON with Padding (See http://ajaxian.com/archives/jsonp-json-with-padding..)
Making AJAX Applications Crawlable
Making Ajax Applications Crawlable (See http://code.google.com/web/ajaxcrawling/docs/specification.html.)
Manipulating Browser History
Manipulating the browser history (See https://developer.mozilla.org/en/DOM/Manipulating_the_browser_history.)
Media Fragments URI 1.0
Media Fragments URI 1.0 (See http://www.w3.org/TR/media-frags/.)
Really Simple History RSH
Really Simple History (RSH): Ajax history and bookmarking library (See http://code.google.com/p/reallysimplehistory/.)
RFC 2854
The 'text/html' Media Type D. Connolly, L. Masinter (See http://www.ietf.org/rfc/rfc2854.txt.)
RFC 3986
Uniform Resource Identifier (URI): Generic Syntax T. Berners-Lee, R. Fielding, L. Masinter. (See http://www.ietf.org/rfc/rfc3986.txt.)
RFC 3987
Internationalized Resource Identifiers (IRIs) M. Duerst, M. Suignard. (See http://www.ietf.org/rfc/rfc3987.txt.)
RFC 3987bis
Internationalized Resource Identifiers (IRIs) M. Duerst, M. Suignard, L. Masinter. (See http://tools.ietf.org/html/draft-ietf-iri-3987bis-01.)
Superfeedr
Superfeedr Superfeedr (See http://blog.superfeedr.com/fragment-subscription/.)
TVRaman CACM 2009
Toward 2^W Beyond Web-2.0, Communications Of The ACM, ACM, New York., T.V. Raman. (See http://portal.acm.org/citation.cfm?id=1461945.)
Using URIs to Pass Parameters to Web Applications
Using URLs to pass parameters to web applications, widgets and gadgets (See http://internet-apps.blogspot.com/2007/11/using-urls-to-pass-parameters-to-web.html.)
Web Storage
Web Storage Ian Hickson (See http://dev.w3.org/html5/webstorage/.)
www-tag archive
Mail thread on WWW-TAG from 2007 that initiated some of these discussions. (See http://lists.w3.org/Archives/Public/www-tag/2007Jul/0148.html.)
Xerox Parc Map Viewer
An Interactive Map Viewer Xerox PARC (See http://www2.parc.com/istl/projects/www94/mapviewer.html.)
XKCD and AxsJAX
AxsJAX-Enhanced xkcd User Guide (See http://google-axsjax.googlecode.com/svn/trunk/docs/xkcd_userguide.html.)
XMLHttpRequest
XMLHttpRequest Anne van Kesteren (See http://www.w3.org/TR/XMLHttpRequest2/.)
XML Schema Datatypes
XML Schema Datatypes P. V. Biron, A. Malhotra (See http://www.w3.org/TR/xmlschema-2/.)
XPointer
XPointer Framework P. Grosso, E. Maler, J. Marsh, N. Walsh (See http://www.w3.org/TR/xptr-framework/.)