Content Transformation Guidelines 1o (Rev 15)

W3C Editor's Draft 30 July 2008

This version:
http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/080730
Latest version:
http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/latest
Previous versions:
Draft 1n (diff) http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/080724
Draft 1m (diff) http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/080722
Draft 1l (diff) http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/080712
Draft 1k (diff) http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/080606
Editor:
Jo Rabin, mTLD Top Level Domain (dotMobi)

Abstract

This document provides guidance to content transformation proxies and content providers as to how inter-work when delivering Web content.

Status of this Document

This document is an editors' copy that has no official standing.

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

Publication as a Group Working Draft of a proposed normative Recommendation does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document has been produced by the Content Transformation Task Force of the Mobile Web Best Practices Working Group as part of the Mobile Web Initiative . Please send comments on this document to the Working Group's public email list public-bpwg-ct@w3.org, a publicly archived mailing list .

This document was produced under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of patent disclosures made in connection with this document; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification must disclose the information in accordance with section 6 of the W3C Patent Policy.

Revision Description

Table of Contents

1 Introduction (Non-Normative)
    1.1 Purpose
    1.2 Audience
    1.3 Scope
    1.4 Summary of Requirements
2 Terminology (Normative)
    2.1 Types of Proxy
    2.2 Types of Transformation
3 Conformance (Normative)
    3.1 Classes of Product
    3.2 Normative and Informative Parts
    3.3 Normative Language for Conformance Requirements
    3.4 Content Deployment Conformance
    3.5 Transformation Deployment Conformance
4 Behavior of Components (Normative)
    4.1 Proxy Forwarding of Request
        4.1.1 Applicable HTTP Methods
        4.1.2 no-transform directive in Request
        4.1.3 Treatment of Requesters that are not Web browsers
        4.1.4 Serving Cached Responses
        4.1.5 Alteration of HTTP Header Values
            4.1.5.1 Content Tasting
            4.1.5.2 Avoiding "Request Unacceptable" Responses
            4.1.5.3 User Selection of Restructured Experience
            4.1.5.4 Sequence of Requests
            4.1.5.5 Original Headers
        4.1.6 Additional HTTP Headers
            4.1.6.1 Proxy Treatment of Via Header
    4.2 Server Response to Proxy
        4.2.1 Use of HTTP 406 Status
        4.2.2 Server Origination of Cache-Control: no-transform
        4.2.3 Varying Representations
            4.2.3.1 Use of Vary HTTP Header
            4.2.3.2 Indication of Intended Presentation Media Type of Representation
    4.3 Proxy Forwarding of Response to User Agent
        4.3.1 Receipt of Cache-Control: no-transform
        4.3.2 Receipt of Warning: 214 Transformation Applied
        4.3.3 Server Rejection of HTTP Request
        4.3.4 Receipt of Vary HTTP Header
        4.3.5 Link to "handheld" Representation
        4.3.6 Proxy Decision to Transform
            4.3.6.1 Alteration of Response
            4.3.6.2 HTTPS Link Re-writing
5 Testing (Normative)

Appendices

A References
B Example Transformation Interactions (Non-Normative)
    B.1 Basic Content Tasting by Proxy
    B.2 Optimization based on Previous Server Interaction
    B.3 Optimization based on Previous Server Interaction, Server has Changed its Operation
    B.4 Server Response Indicating that this Representation is Intended for the Target Device
    B.5 Server Response Indicating that another Representation is Intended for the Target Device
C Applicability to Transforming Solutions which are Out of Scope (Non-Normative)
D Scope for Future Work (Non-Normative)
    D.1 POWDER
    D.2 link HTTP Header
    D.3 Sources of Device Information
    D.4 Inter Proxy Communication
    D.5 Amendment to and Refinement of HTTP
E Administrative Arrangements (Non-Normative)
F Acknowledgments (Non-Normative)


1 Introduction (Non-Normative)

1.4 Summary of Requirements

This section summarizes the communication requirements of actors (users, user agents, transforming proxies and origin servers) to communicate with each other. It is recognised that several transformation proxies may be present but their interactions are not discussed in detail. The relevant scenario is as follows:

The needs of these actors are as follows:

  1. The user agent needs to be able to tell the Content Transformation proxy and the origin server:

    1. what type of mobile device and which user agent is being used;

    2. that all Content Transformation should be avoided.

  2. The Content Transformation proxy needs to be able to tell the origin server:

    1. that some degree of Content Transformation (restructuring and recoding) can be performed;

    2. that content is being requested on behalf of something else and what that something else is;

    3. that the request headers have been altered and what the original ones were.

  3. The origin server needs to be able to tell the Content Transformation proxy:

    1. that it varies the representation of its responses according to device type and other factors;

    2. that it is not permissible to perform Content Transformation;

    3. that it has media-specific representations;

    4. that is unable or unwilling to deal with the request in its present form.

  4. The Content Transformation proxy needs to be able to tell the user agent:

    1. that it has applied transformations of various kinds to the content.

  5. The Content Transformation proxy needs to be able to interact with the user:

    1. to allow the user to disable its features;

    2. to alert the user to the fact that it has transformed content and to allow access to an untransformed representation of the content.

Note:

A more extensive discussion of the requirements for these guidelines can be found in "Content Transformation Landscape" [CT Landscape].

2 Terminology (Normative)

3 Conformance (Normative)

4 Behavior of Components (Normative)

4.1 Proxy Forwarding of Request

4.1.2 no-transform directive in Request

If the request contains a Cache-Control: no-transform directive proxies must forward the request unaltered to the server, other than to comply with transparent HTTP behavior and as noted below (see 4.1.6 Additional HTTP Headers).

Note:

An example of the use of Cache-Control: no-transform is the issuing of asynchronous HTTP requests, perhaps by means of XMLHTTPRequest [XHR], which may include such a directive in order to prevent transformation of both the request and the response.

4.1.3 Treatment of Requesters that are not Web browsers

Proxies must act as though a no-transform directive is present (see 4.1.2 no-transform directive in Request) unless they are able positively to determine that the user agent is a Web browser. The mechanism by which a proxy recognizes the user agent as a Web browser should use evidence from the HTTP request, in particular the User-Agent and Accept headers.

4.1.5 Alteration of HTTP Header Values

Other than to comply with transparent HTTP operation, proxies should not modify request headers unless:

  1. the user would be prohibited from accessing content as a result of the server responding that the request is "unacceptable" (see 4.3.3 Server Rejection of HTTP Request);

  2. the user has specifically requested a restructured desktop experience;

  3. the request is part of a sequence of requests to the same Web site and either it is technically infeasible not to adjust the request because of earlier interaction, or because doing so preserves consistency of user experience.

These circumstances are detailed in the following sections.

Note:

In this section, the concept of "Web site" is used (rather than "origin server") as some origin servers host many different Web sites. Since the concept of "Web site" is not strictly defined, proxies should use heuristics including comparisons of domain name to assess whether resources form part of the same "Web site".

4.1.5.2 Avoiding "Request Unacceptable" Responses

A proxy may reissue a request with altered HTTP header values if a previous request with unaltered values resulted in the origin server rejecting the request as "unacceptable" (see 4.3.3 Server Rejection of HTTP Request). A proxy may apply heuristics of various kinds to assess, in advance of sending unaltered header values, whether the request is likely to cause a "request unacceptable" response. If it determines that this is likely then it may alter header values without sending unaltered values in advance, providing that it subsequently assesses the response as described under 4.3.4 Receipt of Vary HTTP Header below, and is prepared to reissue the request with unaltered headers, and alter its subsequent behavior in respect of the Web site so that unaltered headers are sent.

A proxy must not re-issue a POST/PUT request with altered headers when the response to the unaltered POST/PUT request has HTTP status code 200 (in other words, it may only send the altered request for a POST/PUT request when the unaltered one resulted in an HTTP 406 response, and not a "request unacceptable" response).

4.1.5.3 User Selection of Restructured Experience

Proxies may offer users an option to choose to view a restructured experience even when a Web site offers a choice of user experience. If a user has made such a choice then proxies may alter header values when requesting resources in order to reflect that choice, but must, on receipt of an indication from a Web site that it offers alternative representations (see 4.2.3.2 Indication of Intended Presentation Media Type of Representation), inform the user of that and allow them to select an alternative representation.

Proxies should assume that by default users will wish to receive a representation prepared by the Web site. Proxies must assess whether a user's expressed preference for a restructured representation is still valid if a Web site changes its choice of representations (see 4.3.4 Receipt of Vary HTTP Header).

4.2 Server Response to Proxy

4.2.3 Varying Representations

Servers should take account of user agent capabilities and formulate an appropriate experience according to those capabilities. Servers should provide a means for users to select among available representations, should default to the last selected representation and should provide a means of changing the selection.

4.2.3.1 Use of Vary HTTP Header

If a server varies its representation according to examination of received HTTP headers then it must include a Vary HTTP header indicating this to be the case. If, in addition to, or instead of HTTP headers, a server varies its representation based on other factors (e.g. source IP Address) then it must, in accordance with [RFC 2616 HTTP], include a Vary header containing the value '*'.

Servers may base their actions on knowledge of behavior of specific transforming proxies, as identified in a Via header, but should not choose an Internet content type for a response based on an assumption or heuristics about behavior of any intermediaries. (e.g. a server should not choose Content-Type: application/vnd.wap.xhtml+xml solely on the basis that it suspects that proxies will not transform content of this type).

4.2.3.2 Indication of Intended Presentation Media Type of Representation

If a server has distinct representations that vary according to the target presentation media type, it should inhibit transformation of the response by including a Cache-Control: no-transform directive (see 4.2.2 Server Origination of Cache-Control: no-transform).

In HTML content it should indicate the medium for which the representation is intended by including a link element identifying in its media attribute the target presentation media types of this representation and setting the href attribute to a valid local reference (i.e. use the fragment identifier (see [RFC 3986] section 3.5) added to the URI of the document being served to point to a valid target within the document).

In addition it should include link elements identifying the target presentation media types of other available representations by setting the media attribute to indicate those representations and the href attribute to a URI without a fragment identifier.

Note:

The presence of link elements which do not contain a valid local reference does not indicate one way or another whether this representation is formatted for the presentation media types listed.

Note:

Some examples of the use of the link element are included below in B Example Transformation Interactions.

4.3 Proxy Forwarding of Response to User Agent

4.3.1 Receipt of Cache-Control: no-transform

If the response includes a Cache-Control: no-transform directive then proxies must not alter it other than to comply with transparent HTTP behavior and other than as follows.

If a proxy determines that a resource as currently represented is likely to cause serious mis-operation of the user agent then it may advise the user that this is the case and must provide the option for the user to continue with unaltered content.

4.3.3 Server Rejection of HTTP Request

For compatibility with servers that do not implement this Recommendation (see 4.2.1 Use of HTTP 406 Status), a proxy may treat responses with an HTTP 200 Status as though they were responses with an HTTP 406 Status if it has determined that the content (e.g. "Your browser is not supported") is equivalent to a response with an HTTP 406 Status.

4.3.6 Proxy Decision to Transform

In the absence of a Vary or no-transform directive (or a meta HTTP-Equiv element containing Cache-Control: no-transform) proxies should apply heuristics to the response to determine whether it is appropriate to restructure or recode it (in the presence of such directives, heuristics should not be used.)

Examples of heuristics:

  • The Web site (see note) has previously shown that it is contextually aware, even if the present response does not indicate this;

  • a claim of mobileOK Basic [mobileOK Basic Tests] conformance is indicated;

  • the Content-Type or other aspects of the response (such as the DOCTYPE) are known to be specific to the device or class of device;

    Examples of mobile specific DOCTYPEs:

    -//OMA//DTD XHTML Mobile 1.2//EN
    -//WAPFORUM//DTD XHTML Mobile 1.1//EN 
    -//WAPFORUM//DTD XHTML Mobile 1.0//EN
    -//W3C//DTD XHTML Basic 1.1//EN
    -//W3C//DTD XHTML Basic 1.0//EN)
  • the user agent has linearization or zoom capabilities or other features which allow it to present the content unaltered;

  • the URI of the response (following redirection or as indicated by the Content-Location HTTP header) indicates that the resource is intended for mobile use (e.g. the domain is *.mobi, wap.*, m.*, mobile.* or the leading portion of the path is /m/ or /mobile/);

  • the response contains client-side scripts that may mis-operate if the resource is restructured;

  • the response is an HTML response and it includes <link> elements specifying alternatives according to presentation media type.

5 Testing (Normative)

Operators of transforming proxies should make available interfaces that facilitate testing of Web sites accessed through them and should make such interfaces available through normal Internet access paths.

A References

CT Landscape
Content Transformation Landscape 1.0, Jo Rabin, Andrew Swainston (eds), W3C Working Draft 25 October 2007 (See http://www.w3.org/TR/ct-landscape/)
RFC 2119
Key words for use in RFCs to Indicate Requirement Levels, , Request for Comments: 2119, S. Bradner, March 1997 (See http://www.ietf.org/rfc/rfc2119.txt)
RFC 2616 HTTP
Hypertext Transfer Protocol -- HTTP/1.1 Request for Comments: 2616, R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee, June 1999 (See http://tools.ietf.org/html/rfc2616)
RFC 3986
Uniform Resource Identifier (URI): Generic Syntax, Request for Comments: 3986, T. Berners-Lee, R. Fielding, L. Masinter, January 2005 (See http://tools.ietf.org/html/rfc3986)
Device Independence Glossary
W3C Glossary of Terms for Device Independence, Rhys Lewis (ed), W3C Working Draft 18 January 2005
Best Practices
Mobile Web Best Practices 1.0 Basic Guidelines, Jo Rabin, Charles McCathieNevile (eds), W3C Proposed Recommendation, 2 November 2006 1l: To update (See http://www.w3.org/TR/mobile-bp/)
mobileOK Basic Tests
W3C mobileOK Basic Tests, Sean Owen, Jo Rabin (eds), W3C Working Draft, 10 June 2008 1l: To update (See http://www.w3.org/TR/mobileOK-basic10-tests/)
XHR
The XMLHttpRequest Object, Anne van Kesteren (ed), W3C Working Draft, 15 April 2008 (See http://www.w3.org/TR/XMLHttpRequest/)

B Example Transformation Interactions (Non-Normative)

B.3 Optimization based on Previous Server Interaction, Server has Changed its Operation

Proxy receives a request for resource P, that it has previously encountered as in B.2 Optimization based on Previous Server Interaction

Proxy forwards request with altered headers

Response is 200 OK containing a Vary: User-Agent header

Proxy notices that behavior has changed and re-issues request with original headers

Response is 200 OK and proxy forwards it

C Applicability to Transforming Solutions which are Out of Scope (Non-Normative)

There are a number of well-known examples of solutions that seem to their users as though they are using a browser, but because the client software communicates with using proprietary protocols and techniques, it is the combination of the client and the in-network component that is regarded as the HTTP User Agent. The communication between the client and the in-network component is therefore out of scope of this document.

Additionally, where some kind of administrative arrangement exists between a transforming proxy and an origin server for the purposes of transforming content on the origin server's behalf, this is also out of scope of this document.

In both of the above cases, it is recommended that when forwarding requests to origin servers that proxies adhere to the provisions of this document in respect of providing information about the device and the original IP address.

D Scope for Future Work (Non-Normative)

D.1 POWDER

The BPWG believes that POWDER will represent a powerful mechanism by which a server may express transformation preferences. Future work in this area may recommend the use of POWDER to provide a mechanism for origin servers to indicate more precisely which alternatives they have and what transformation they are willing to allow on them, and in addition to provide for Content Transformation proxies to indicate which services they are able to perform.

E Administrative Arrangements (Non-Normative)

It is noted that there are means which fall outside of the scope of this document for establishing user preferences with content transformation proxies. It is anticipated that proxies will maintain preferences on a user by user and Web site by Web site basis, and will change their behavior in the light of changing circumstances as discussed under 4.3.4 Receipt of Vary HTTP Header.

F Acknowledgments (Non-Normative)

The editor acknowledges contributions of various kinds from members of the Mobile Web Best Practices Working Group Content Transformation Task Force.

The editor acknowledges significant written contributions from: