Copyright
© 2008 © 2009 W3C ® (
MIT ,
ERCIM
, Keio ), All Rights Reserved.
W3C liability
, trademark
and document
use rules apply.
This document provides guidance to content
transformation Content
Transformation proxies and content
providers as to whether and how
inter-work when delivering to transform Web content.
Content Transformation proxies alter requests sent by user agents to servers and responses returned by servers so that the appearance, structure or control flow of Web applications are modified. Content Transformation proxies are mostly used to convert Web sites designed for desktop computers to a form suitable for mobile devices.
Based on current practice and standards, this document specifies mechanisms with which Content Transformation proxies should make their presence known to other parties, present the outcome of alterations performed on HTTP traffic, and react to indications set by clients or servers to constrain these alterations.
The objective is to reduce undesirable effects on Web applications, especially mobile-ready ones, and to limit the diversity in the modes of operation of Content Transformation proxies, while at the same time allowing proxies to alter content that would otherwise not display successfully on mobile devices.
Important considerations regarding the impact on security are highlighted.
This section describes the status of this document at the
time of its publication. Other documents may supersede this
document. A list of current W3C publications and the latest
revision of this technical report can be found in the W3C technical reports index at
http://www.w3.org/TR/. http://www.w3.org/TR/ .
This is a the
second Last Call Working Draft of Guidelines for Web Content Transformation
Guidelines 1.0, Proxies 1.0 expected to become a W3C
Recommendation. The W3C Membership and other interested parties are
invited to review the document and send comments to public-bpwg-comments@w3.org
(with public
archive ) through 16 September
2008 6 November 2009
.
This document reflects group resolutions on comments received on the previous Last Call Working Draft (published on 1 August 2008). A complete list of changes is available. Many guidelines have been updated, adjusted or clarified based on feedback. Main changes are (list is not exhaustive):
"X-Device-"<original header name>
HTTP Header fields were provisionally
registered with IANA.Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document has been produced by the Mobile Web Best Practices
Working Group as part of the Mobile Web Initiative
. Since its publication as a First Public
Working Draft . Please send
comments on 14 April 2008 , the Content
Transformation Guidelines 1.0 this document has been
almost entirely re-written. The guidelines were extended, precised
and re-worded for clarity reasons. In particular: The Conformance
section was added. The statements were simplified, clarified and
follow the same normative pattern. The choice of the
"X-Device-"<original header name> HTTP header was confirmed.
The possible uses of to the
link element were detailed. The notion of Web
Site was introduced. The Working Group
notes that it has already identified Group's public email list public-bpwg-ct@w3.org , a guideline considered to be at risk : the notification of
users as defined in section 4.1.4 Serving Cached Responses
publicly archived mailing list may turn out to be difficult to implement from a user
experience's point of view and may be removed in future versions of
the document based on the feedback received. .
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy . W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy .
1 Introduction
(Non-Normative)
1.1 Purpose
1.2 Audience
1.3 Scope
1.4 Summary
Principles
1.4.1
IAB
Considerations
1.4.2
Priority of Requirements Intention
2 Terminology (Normative)
2.1 Types of
Proxy
2.2 Types of Transformation
3 Conformance (Normative)
3.1 Classes of Product
3.2 Normative and Informative
Parts
3.3 Normative
Language for Conformance Requirements
3.4 Content
Deployment Conformance 3.5 Transformation
Deployment Conformance
4 Behavior of Components
(Normative)
4.1 Proxy
Forwarding of Request
4.1.1 Applicable HTTP Methods
4.1.2 no-transform directive in
Request
4.1.3 Treatment of Requesters that are not Web
browsers
4.1.4 Serving Cached Responses
4.1.5 Alteration of HTTP Header Field Values
4.1.5.1
Content Tasting
4.1.5.2
Avoiding "Request
Unacceptable" Responses
4.1.5.3
User Selection of Restructured
Experience
4.1.5.4
Sequence of Requests
4.1.5.5
Original Headers Header
Fields
4.1.6 Additional HTTP Headers Header
Fields
4.1.6.1
Proxy Treatment of Via Header
Field
4.2 Server
Proxy
Forwarding of Response to Proxy
User Agent
4.2.1 Use of HTTP 406 Status Applicable
Responses
4.2.2 Server Origination of Cache-Control: no-transform
4.2.3 Varying
Representations
4.2.3.1
Use of Vary HTTP Header
4.2.3.2
Indication of Intended Presentation Media Type of Representation
4.3 Proxy Forwarding of Response to
User Agent Preferences
4.3.1
4.2.3
Receipt of
Cache-Control: no-transform
4.3.2
Receipt 4.2.4
Use of Warning: 214
Transformation Applied Cache-Control:
no-transform
4.3.3
4.2.5
Server Rejection of HTTP
Request
4.3.4
4.2.6
Receipt of Vary HTTP Header
Field
4.3.5
4.2.7
Link to "handheld"
Representation
4.3.6
4.2.8
WML
Content
4.2.9
Proxy Decision to
Transform
4.3.6.1
4.2.9.1
Alteration of
Response
4.3.6.2
4.2.9.2
Link
Rewriting
4.2.9.3
HTTPS Link Re-writing Rewriting
5 Testing (Normative)
A References
B Conformance Statement
C Internet Content
Types associated with Mobile Content
D DOCTYPEs Associated
with Mobile Content
E URI Patterns
Associated with Mobile Web Sites
F Summary of User Preference Handling
G Example Transformation
Interactions (Non-Normative)
B.1 G.1 Basic Content Tasting by Proxy
B.2 G.2 Optimization based on Previous
Server Interaction
B.3 G.3 Optimization based on Previous Server Interaction,
Server has Changed its Operation
B.4 G.4 Server Response Indicating that this Representation is
Intended for the Target Device
B.5 G.5 Server Response Indicating that another Representation
is Intended for the Target Device
C H
Informative Guidance for
Origin Servers (Non-Normative)
H.1 Server Response to
Proxy
H.1.1
Use of HTTP
406 Status
H.1.2
Use of HTTP
403 Status
H.1.3
Server Origination of Cache-Control:
no-transform
H.1.4
Varying Representations
H.1.4.1
Use of Vary
HTTP Header Field
H.1.4.2
Indication
of Intended Presentation Media Type of
Representation
I Applicability to Transforming Solutions which
are Out of Scope (Non-Normative)
D J
Scope for Future Work
(Non-Normative)
D.1 J.1 POWDER
D.2 J.2 link HTTP Header Field
D.3 J.3 Sources of Device Information
D.4 J.4 Inter Proxy Communication
D.5 J.5 Amendment to and Refinement of HTTP
E Administrative Arrangements (Non-Normative)
F K Acknowledgments
(Non-Normative)
From the point of view of
Within this document, document
Content Transformation is
refers to the manipulation in various ways, by proxies, of requests
made to to,
and content delivered by responses from, an origin server with a view server.
This manipulation is carried out by proxies in order to
provide a better user experience of content
that would otherwise result in an unsatisfactory experience on the
device making it more suitable for
mobile presentation. the
request.
The W3C Mobile Web Best Practices Working Group neither approves nor disapproves of Content Transformation, but recognizes that is being deployed widely across mobile data access networks. The deployments are widely divergent to each other, with many non-standard HTTP implications, and no well-understood means either of identifying the presence of such transforming proxies, nor of controlling their actions. This document establishes a framework to allow that to happen.
The overall objective of this document is to provide a means, as far as is practical, for users to be provided with at least a "functional user experience" [Device Independence Glossary] of the Web, when mobile, taking into account the fact that an increasing number of content providers create experiences specially tailored to the mobile context which they do not wish to be altered by third parties. Equally it takes into account the fact that there remain a very large number of Web sites that do not provide a functional user experience when perceived on many mobile devices.
The audience for this document is creators of Content
Transformation proxies, proxies and purchasers and operators of such
proxies and proxies. The document also contains non-normative
guidance for content providers whose services may be accessed
by means of such proxies.
The recommendations in this document refer only to "Web browsing" - i.e. access by user agents that are intended primarily for interaction by users with HTML Web pages (Web browsers) using HTTP. Clients that interact with proxies using mechanisms other than HTTP (and that typically involve the download of a special client) are out of scope, and are considered to be a distributed user agent. Proxies which are operated in the control of or under the direction of the operator of an origin server are similarly considered to be a distributed origin server and hence out of scope.
The BPWG W3C Mobile Web
Best Practices Working Group (BPWG) is not chartered to create new technology -
its role is to advise on best practice for use of existing
technology. In satisfying Content Transformation requirements,
existing HTTP headers, header fields, directives and behaviors must be
respected, and as far as is practical, no extensions to [RFC 2616 HTTP] are to be used.
This section summarizes the communication
requirements of actors (users, user agents, transforming proxies
and origin servers) The recommendations
in this document refer to communicate
with each other. It is recognised that several transformation
proxies may be present but their interactions are not discussed in detail. The relevant scenario is as
follows: The needs of these actors are
as follows: The user agent needs to be able to tell the Content
Transformation a proxy and
do not refer to any presumed aspects of
the origin server: what type
internal operation of mobile device and which user agent is being used; that
all Content Transformation should be avoided. The Content
Transformation proxy needs to be able to tell the origin server: that some degree proxy. For this reason, the document does not discuss
use of Content Transformation (
restructuring "allow" and
recoding ) can be performed;
"disallow" lists (though it does discuss
behavior that content is
being requested on behalf of something else
and what that something else is; that the request headers have been
altered and what the original ones were. The origin server needs to
be able to tell induced by the
Content Transformation proxy: that
implementation of such lists). In
addition it varies does not discuss details of how transformation is
carried out except if this is reflected in interoperability. For
this reason, it does not discuss the representation insertion or
insertion of its responses according to
device type headers and
footers or any other factors; that specific
behaviors (though it is not permissible
to perform Content Transformation; does
discuss the need for essential user interaction of some
form).
that is unable or unwilling
The BPWG made reference to deal with Internet
Architecture Board (IAB) work on "Open Pluggable Edge
Services" [RFC 3238
OPES] for various principles that
underlie behavior of proxies. In this work the request in IAB
expressed its present form.
concerns about privacy, control, monitoring,
and accountability of such services.
The Content Transformation proxy needs to
be able to tell the user agent: that it has applied
transformations Web allows users
considerable flexibility in respect of various kinds to the representation of content. The At the same time,
Content Transformation proxy needs
Providers may have a preferred manner in
which they wish their content to be able to interact with represented. Content Transformation must reconcile these
contrasting factors. In creating this Recommendation the
user: to BPWG has
determined that Content Transformation proxies should respect
Content Providers intentions, where they are expressed, but
may allow the user users to disable its
features; choose other representations,
except where Content Providers specifically prohibit
this.
to alert the user to the fact
The BPWG recognizes that it has transformed content and to allow access to an
untransformed representation there is
neither a systematic vocabulary for Content Provider Intentions,
nor a systematic means of the content.
Note: A more extensive discussion expression of the
requirements such intentions. There is
scope for these guidelines can be
found further work in "Content Transformation Landscape" [CT Landscape]
. this area (see J Scope for Future Work ).
Alteration of HTTP requests and responses is not prohibited by HTTP other than in the circumstances referred to in [RFC 2616 HTTP] Section 13.5.2 and Section 14.9.5 .
HTTP defines two types of proxy: transparent proxies and non-transparent proxies. As discussed in [RFC 2616 HTTP] Section 1.3, Terminology :
"A transparent proxy is a proxy that does not modify the request or response beyond what is required for proxy authentication and identification. A non-transparent proxy is a proxy that modifies the request or response in order to provide some added service to the user agent, such as group annotation services, media type transformation, protocol reduction, or anonymity filtering. Except where either transparent or non-transparent behavior is explicitly stated, the HTTP proxy requirements apply to both types of proxies."
This document elaborates the behavior of non-transparent proxies, when used for Content Transformation in the context discussed in [CT Landscape] .
Transforming proxies can carry out a wide variety of operations. In this document we categorize these operations as follows:
Alteration of Requests
Transforming proxies process requests in a number of ways,
especially replacement of various request headers header fields
to avoid HTTP 406 Status responses (if a server can not provide
content that is compatible with the original HTTP request
headers) header
fields) and at user request.
Alteration of Responses
There are three classes of operation on responses:
Restructuring content
Restructuring content is a process
whereby the original layout is altered so that content is added or
removed or where the spatial or navigational relationship of parts
of content is altered, e.g. by
linearization (i.e. reordering presentation
elements, especially tables, so that they fit on a narrow display
and can be traversed without horizontal scrolling) or
pagination. pagination (i.e. splitting a document too large to be
stored in or transmitted to the terminal in one piece, so that it
can be nevertheless accessed by browsing through a succession of
smaller interlinked documents). It includes also includes
rewriting of URIs so that subsequent
requests route are routed via the proxy handling this the response.
Recoding content
Recoding content is a process whereby the layout of the content remains the same, but details of its encoding may be altered. Examples include re-encoding HTML as XHTML, correcting invalid markup in HTML, conversion of images between formats (but not, for example, reducing animations to static images).
Optimizing content
Optimizing content includes removing redundant white space, re-compressing images (without loss of fidelity) and compressing for transfer.
The Content Transformation Guidelines specification has
two classes one
class of products:
A Transformation Deployment is the provision of non-transparent components in the path of HTTP requests and responses. Provisions that are applicable to a Transformation Deployment are identified in this document by use of the term "transforming proxy" or "proxy" in the singular or plural.
Normative parts of this document are identified by the use of "(Normative)" following the section name. Informative parts are identified by use of "(Non-Normative)" following the section name.
The key words must , must not , required , shall , shall not , should , should not , recommended , not recommended ,may , and optional in this Recommendation have the meaning defined in [RFC 2119] .
A Content Transformation Deployment conforms to these
guidelines if it follows the statements in 4.2 Server Response to Proxy . 3.5 3.4 Transformation Deployment Conformance
A Transformation Deployment conforms to these
guidelines if it follows the statements in , 4.1 Proxy Forwarding of
Request , 4.3 4.2 Proxy Forwarding of Response to User
Agent and 5
Testing (Normative) .
A Transformation Deployment that wishes to claim conformance must make available a conformance statement B Conformance Statement that specifies the reasons for non-compliance with any clauses containing the key words " should " and " should not ", " recommended " and " not recommended ".
Conformance statements must be sent to public-content-transformation-conformance@w3.org .Public archives of this list may be found at http://lists.w3.org/Archives/Public/public-content-transformation-conformance/ .
Proxies should not intervene in methods other
than GET, POST, HEAD and PUT.
HEAD.
User agents sometimes issue HTTP HEAD requests in order to determine if a resource is of a type and/or size that they are capable of handling. A transforming proxy may convert a HEAD request into a GET request (in order to determine the characteristics of a transformed response that it would return if the user agent subsequently issued a GET request for the same resource).
If the HTTP method is altered from HEAD to GET, proxies should (providing such action is in accordance with normal HTTP caching rules) cache the response so that a second GET request for the same content is not required (see also 4.1.4 Serving Cached Responses ).
Other than to convert between HEAD and GET proxies must not alter request methods.
no-transform
directive in RequestIf the request contains a Cache-Control:
no-transform
directive
directive, proxies must
not forward alter the
request unaltered to the server, other
than to comply with transparent HTTP behavior defined in [RFC 2616 HTTP] sections
section 14.9.5 and
section 13.5.2 and to add header
fields as noted below (see
described in 4.1.6 Additional
HTTP Headers Header Fields ). below.
Note:
An example of the use of Cache-Control:
no-transform
is the issuing of asynchronous HTTP requests,
perhaps by means of XMLHTTPRequest
XMLHttpRequest [XHR] , which may include such a directive in order
to prevent transformation of both the request and the response.
Proxies must act as though a no-transform
directive is present (see 4.1.2 no-transform directive in Request )
unless they are able positively Before
altering aspects of an HTTP request proxies need to
determine that take account of the user
agent fact that HTTP is
a Web browser. The mechanism by which a proxy
recognizes the user agent used as
a Web browser should use evidence from
the transport mechanism for many
applications other than "Traditional Browsing". Alteration of
HTTP request, in particular the User-Agent
and Accept headers. requests for those
applications can cause serious misoperation.
Proxies should follow standard HTTP
Aside from the usual caching procedures
defined in respect of caching and should use cached copies of
resources where this is [RFC 2616 HTTP] , in accordance with those procedures. In some
circumstances, proxies may paginate responses and
where this is the case a request may be for a subsequent page of a
previously requested resource. In this case proxies
may for the sake of consistency of representation
serve stale data but when doing so should notify
the user that this is the case and should must
provide a simple means of retrieving a fresh copy.
Aside from the usual procedures defined
in [RFC 2616
HTTP] proxies
should not modify the values of header fields other than the
User-Agent
,Accept
,Accept-Charset
,Accept-Encoding
,and Accept-Language
header fields and must not delete
header fields. It must be possible
for the server to reconstruct the original User Agent originated
header fields by copying directly from the corresponding X-Device
header field values (see 4.1.5.5 Original Header Fields
).
Other than to comply with transparent HTTP operation, proxies
should not modify any request headers
unless: header fields unless one of the
following applies:
the user would be prohibited from accessing content as a result
of the server responding that the request is "unacceptable" (see
4.3.3 4.2.5 Server Rejection of HTTP Request
);
the user has specifically requested a restructured desktop experience; experience
(see 4.1.5.3 User Selection of
Restructured Experience );
the request is part of a sequence of requests to the same Web site and either it is technically infeasible not to adjust the request because of earlier interaction, or because doing so preserves consistency of user experience.
These circumstances are detailed in the following sections.
Note:
In this section, the concept of "Web site" is used (rather than "origin server") as some origin servers host many different Web sites. Since the concept of "Web site" is not strictly defined, proxies should use heuristics including comparisons of domain name to assess whether resources form part of the same "Web site".
Note:
The URI referred to in the request plays no part in determining whether or not to alter HTTP request header field values. In particular the patterns mentioned in 4.2.9 Proxy Decision to Transform are not material.
The theoretical idempotency of GET requests is not always
respected by servers. In order, as far as possible, to avoid
mis-operation misoperation of such content, proxies
should avoid issuing duplicate requests and
specifically should not issue duplicate requests
for comparison purposes.
A proxy may reissue a request with altered HTTP
header field values if a previous
request with unaltered values resulted in the origin server
rejecting the request as "unacceptable" (see 4.3.3
4.2.5 Server
Rejection of HTTP Request ). A proxy may apply
heuristics of various kinds to assess, in advance of sending
unaltered header field values, whether
the request is likely to cause a "request unacceptable" response.
If it determines that this is likely then it may
alter header field values without
sending unaltered values in advance, providing that it subsequently
assesses the response as described under 4.3.4
4.2.6 Receipt of
Vary HTTP Header Field
below, and is prepared to reissue the request with unaltered
headers, header
fields, and alter its subsequent behavior in respect of the
Web site so that unaltered headers
header fields are sent.
A proxy must not re-issue reissue a
POST/PUT POST request with altered headers header fields
when the response to the unaltered POST/PUT POST request
has HTTP status code 200 (in other words, it may only send the
altered request for a POST/PUT POST request when the unaltered one
resulted in an HTTP 406 response, and not a "request unacceptable"
response).
Proxies must assume that by default users will wish to receive a representation prepared by the Web site.
Proxies may offer
users an option to choose to view a restructured experience even
when a Web site offers a choice of user experience. If a user has
made such a choice then proxies may alter header
field values when requesting resources
in order to reflect that choice, but must , on
receipt of an indication from a Web site that it offers alternative
representations (see 4.2.3.2 H.1.4.2 Indication of Intended Presentation Media
Type of Representation ), inform the user of that and
allow them to select an alternative representation.
Proxies should assume that by
default users will wish to receive a representation prepared by the
Web site. Proxies must assess whether a user's
expressed preference for a restructured representation is still
valid if a Web site changes its choice of representations (see
4.3.4 4.2.6 Receipt of Vary HTTP Header Field ).
When requesting resources that form part
of the representation of a resource are
included resources (e.g. style sheets, images), proxies
should make the request for such resources with
the same headers User-Agent
header
field as the request for the resource from which they are
referenced.
For the purpose of consistency of representation, proxies
may request linked resources (e.g. those
referenced using the a
element) that form part of the
same Web site as a previously requested resource with the same
headers header
fields as the resource from which they are referenced.
When requesting linked resources that do not form part of the
same Web site as the resource from which they are linked, proxies
should not base their choice of headers header fields
on a consistency of presentation premise.
When forwarding an HTTP request with altered HTTP headers header fields, in
addition to complying with the rules of normal HTTP
operation, proxies must include in the
request copies of the unaltered header field values in the form
"X-Device-"<original header name>
. For example,
if the User-Agent
header
field has been altered, an
X-Device-User-Agent
header field must be added with the
value of the received User-Agent
header. header
field.
Specifically the following mapping must be used:
Original | Replacement | Ref |
---|---|---|
User-Agent |
X-Device-User-Agent |
RFC2616 Section 14.43 |
Accept |
X-Device-Accept |
RFC2616 Section 14.1 |
Accept-Charset |
X-Device-Accept-Charset |
RFC2616 Section 14.2 |
Accept-Encoding |
X-Device-Accept-Encoding |
RFC2616 Section 14.3 |
Accept-Language |
X-Device-Accept-Language |
RFC2616 Section 14.4 |
Note:
The X-Device-
prefixed
header names listed in this section have been provisionally
registered with IANA (see
Provisional Message Header Field
Names ).
Note:
The X-Device-
prefix was chosen primarily on the
basis that this is a an already existing convention. It is noted that
the values encoded in such header fields may not ultimately derive from a device,
they are merely received headers.
fields. The treatment of received
X-Device
headers,
header fields, which may happen where
there are multiple transforming proxies, is undefined (see D
J Scope for
Future Work ).
Irrespective of the presence of a no-transform
directive:
proxies should add the IP address of the
initiator of the request to the end of a comma separated list in an
X-Forwarded-For
HTTP header; header
field;
proxies must include a Via
HTTP
header field (see 4.1.6.1 Proxy Treatment of
Via Header Field ).
Via
Header FieldProxies must (in accordance with compliance to RFC 2616) include a Via
HTTP header field indicating their
presence and should indicate their conformance ability to
this Recommendation transform content by including a comment in the
Via
HTTP header field
consisting of the URI "http://www.w3.org/ns/ct".
When forwarding Via
headers header fields,
proxies should not alter them in any way. by removing
comments from them.
Note:
According to [RFC 2616 HTTP] Section
14.45 Via
header field
comments " may be removed by any recipient prior
to forwarding the message". However, the justification for removing
such comments is based on memory limitations of early proxies, most proxies.
Most modern proxies do not suffer such limitations.
Servers In the
following, proxies should
must respond with an HTTP 406 Status (and not an HTTP 200
Status) if a request cannot be satisfied with content that
meets check for the criteria specified by values presence of equivalent
<meta http-equiv>
elements in HTML content, if the
relevant HTTP request headers. header
field is not present.
Servers must include a Cache-Control:
no-transform directive if one is received in the HTTP request.
Servers Proxies should
not include a Cache-Control: no-transform directive if, for
any reason, they wish to inhibit transformation of
intervene in the response. Note: Including a Cache-Control: no-transform
directive can disrupt response if
the behavior of WAP/WML proxies, because it
can inhibit such proxies from converting WML to WMLC.
request method was not HEAD, GET or
POST.
Servers should take account of user agent
capabilities and formulate an appropriate experience according to
those capabilities. Servers Proxies should
must provide a means for users
to select among available representations,
should default to express preferences
for inhibiting content transformation even when content
transformation has been chosen by the last selected representation and should provide a means
of changing user as the
selection. 4.2.3.1 Use of Vary HTTP Header If
a server varies its representation according to examination of
received HTTP headers then it default
behavior. Those preferences must include a Vary HTTP header indicating this to be
the case. If, in addition to, or instead of
HTTP headers, a server varies its representation based
maintained on other factors (e.g. source IP Address) then it must , in
accordance with [RFC 2616 HTTP] , include a Vary header containing the value '*'. user by user and Web site by Web site basis.
Servers Proxies may
must base their actions on knowledge of behavior
solicit re-expression of specific transforming proxies, as identified
preferences in a
Via header, but should not choose an Internet content type for a
response based on an assumption or heuristics about behavior
respect of any
intermediaries. (e.g. a server should
not choose Content-Type: application/vnd.wap.xhtml+xml solely
on if the basis that it suspects that proxies will not transform
content of this type). 4.2.3.2 Indication of Intended Presentation
Media Type of Representation If a server has distinct representations that vary according
starts to the
target presentation media type, it should inhibit transformation of
the response by including a Cache-Control: no-transform directive
(see 4.2.2 Server Origination of Cache-Control: no-transform ). In
HTML content it should indicate the
medium for which the representation is intended by including a link
element identifying in its media attribute the target presentation
media types of this representation and setting the href attribute
to a valid local reference (i.e. use the fragment identifier (see
[RFC 3986] section 3.5 ) added to the URI of the document being
served to point to a valid target within the document). In
addition that it should include link elements identifying the target
presentation media types of other available representations by
setting the media attribute to indicate those representations and
the href attribute to a URI without a fragment identifier. Note:
The presence of link elements which do not contain a valid local
reference does not indicate one way or another whether this
representation is formatted for the presentation media types
listed. Note: offers varying responses
as discussed under Some examples
of the use 4.2.6
Receipt of the link element are
included below in B Example Transformation Interactions
Vary HTTP Header Field .
Cache-Control: no-transform
If the response includes a Cache-Control:
no-transform
directive then proxies must
not alter it other than to comply with transparent HTTP
behavior as described in [RFC 2616 HTTP]
Section 13.5.2 and
Section 14.9.5 and other than as follows.
If a proxy determines that a resource as currently represented
is likely to cause serious mis-operation misoperation of the user agent then it
may advise the user that this is the case and
must provide the option for the user to continue
with unaltered content. content (and may provide other
options too).
Warning: 214
Transformation Applied Cache-Control:
no-transform
If the response includes a Warning: 214
Transformation Applied HTTP header, proxies Proxies must
not may apply use
Cache-Control:
no-transform
to inhibit
transformation by further transformation. proxies.
For compatibility with servers that do not
implement this Recommendation (see 4.2.1 Use of HTTP 406 Status ),
a proxy Proxies
may treat responses with an HTTP 200 Status as
though they were responses with an HTTP 406 Status if it has
determined that the content (e.g. "Your browser is not supported")
is equivalent to a response with an HTTP 406 Status.
Vary
HTTP
Header FieldIf, in response to an HTTP request with
altered headers that was A proxy
may not preceded by an HTTP
be carrying out content tasting as described
under 4.1.5.2 Avoiding "Request
Unacceptable" Responses if
it anticipates receiving a "request unacceptable" response.
However, if it makes a request with unaltered headers, a proxy altered header fields in these circumstances, and
receives a response containing a Vary
header
field referring to one of the altered
headers header
fields then it should request the resource
again with unaltered headers, it
header fields. It
should also update
whatever heuristics it uses so that unaltered headers header fields
are presented first in subsequent requests for this resource and it should resume the behavior described
under 4.1.5.2 Avoiding "Request Unacceptable" Responses to avoid
rejection of subsequent requests. resource.
If the response is an HTML response and it contains a
<link rel="alternate" media="handheld" />
element, the CT-proxy a proxy should request and
process the referenced resource, unless the resource referenced is
the current resource representation .
Note:
In this document
the term current
representation means a "same
document reference" as determined
by defined in [RFC 3986] Section 4.4 ,with
the presence of addition that if a link Varyelements as discussed under 4.2.3.2
Indication HTTP header field was
present on the response then it is the same representation if the
values of Intended Presentation Media
Type the HTTP header fields of
Representation the request have not been altered.
If the content is WML proxies should act in a transparent manner.
Note:
This does not affect the operation of proxies that are also WAP Gateways.
In the absence of a Vary
or
no-transform
directive (or a meta
HTTP-Equiv
element containing Cache-Control:
no-transform
) proxies should apply heuristics to the response to determine whether it
is appropriate to restructure or recode it (in the presence of such
directives, heuristics should not be used.) Examples transform
content matching any of heuristics: the following
rules unless the user has specifically requested
transformation:
The Web site (see note ) has previously
shown that it the content is
contextually aware, even if HTML and contains <link rel="alternate"
media="handheld"/>
with a
reference to the present response does
not indicate this; current
representation ;
a claim the DOCTYPE
of mobileOK
Basic [mobileOK Basic Tests] the
content (if it has one) indicates XHTML-MP, XHTML Basic, WML or
iMode as listed in D DOCTYPEs Associated with Mobile
Content conformance is
indicated; ;
the Content-Type
or other
aspects has a value listed in
C Internet Content Types associated
with Mobile Content .
the URI of the response (such (following redirection
or as indicated by the
DOCTYPE) are known to be specific to the
device or class of device; Examples of mobile specific
DOCTYPEs: Content-Location
HTTP
header field) matches a pattern listed in -//OMA//DTD
XHTML E URI
Patterns Associated with Mobile 1.2//EN Web
Sites ;
the response contains a resource that is referenced as an included resource suitable for "handheld" in a resource that was itself handled transparently;
a claim of mobileOK Basic [mobileOK Basic Tests] conformance is indicated (see [mobileOK Scheme] for how such a claim may be indicated).
Other factors that a proxy may take into account:
The Web site (see note ) has previously shown that it is contextually aware, even if the present response does not indicate this;
the user agent has features (such as
linearization or zoom capabilities or other
features which zoom) that allow
it to present the content unaltered;
the URI of the response (following
redirection or as indicated by the Content-Location HTTP header)
indicates that the resource is intended for mobile use (e.g. the
domain is *.mobi, wap.*, m.*, mobile.* or the leading portion of
the path is /m/ or /mobile/); the response contains
client-side client side scripts that may mis-operate misoperate
if the resource is restructured;
the response is an HTML response and it includes
<link>
elements specifying alternatives
according to presentation media type.
Note:
A proxy should strive for the best
possible user experience that the user agent supports. It should
only alter the format, layout, dimensions etc. to match
Other than as noted in this section the
specific capabilities nature of the user agent.
For example, when resizing images, they should only be reduced
so restructuring that they are suitable for the specific user agent,
is carried out, any character encoding
alterations and what is omitted and what is inserted is, as discussed in 1.3
Scope ,out of scope of
this should not be done on a generic
basis. document.
If a proxy alters the response then:
It must add a Warning 214 Transformation
Applied
HTTP header; header field;
The altered content should validate according
to an appropriate published formal grammar; grammar and if
XML must
be well-formed ;
It should indicate to the user that the content has been transformed for mobile presentation and provide an option to view the original, unmodified content.
Note:
If the response contains links whose
In this document two
URIs have the Same-Origin
if the scheme
component and the
and https hostport
subcomponents,
as defined in [RFC 3986] ,all
match. Section 6 of
[RFC 3986]
discusses URI comparison.
Some proxy deployments have to "rewrite" links in content in order for the User Agent to request the referenced resources through the proxy. In so doing, proxies make unrelated resources appear as though they have the same-origin and hence there is a danger of introducing security vulnerabilities.
Note:
This section (on link rewriting) refers also to insertion of links, frame flattening and any other techniques that introduces the "same-origin" issue.
Note:
Link rewriting is always used by CT Proxies that are accessed as an origin server initially, e.g. which provide mobile adapted web search and navigation to the web pages returned in the search results, or to which the browser is redirected through the CT Proxy for adaptation of a web page. Link rewriting may be used by CT Proxies acting as normal HTTP proxies (e.g. configured or transparent) for the browser, but may not be required since all browser requests flow through the CT Proxy.
Proxies must not only
rewrite them so links when content transformation is
prohibited.
Proxies must preserve
security between requests for domains that they can are not
same-origin in respect
of cookies and scripts.
Note:
For clarity it is emphasized that it is
not possible for a transforming proxy to transform
the content accessed via an HTTPS link without breaking end-to-end
security.
Interception of linked resources, if HTTPS
and the following provision
circumstances in which it might be
permissible is met. not a "mobile" question, as such, but is highly
pertinent to this document. The BPWG is aware that interception of
HTTPS happens in many networks today. Interception of HTTPS is
inherently problematic and may be unsafe. The BPWG would like to
refer to protocol based "two party consent" mechanisms, but such
mechanisms do not exist at the time of writing of this
document.
The practice of intercepting HTTPS links is strongly NOT RECOMMENDED .
If a proxy does rewrite such
rewrites HTTPS links, it
must advise the user of the security implications
of doing so and must provide the option to
avoid decryption bypass it and transformation
of the resources to communicate
with the server directly.
Notwithstanding anything else in this
document, proxies must
not rewrite HTTPS links
refer to. in the
presence of a Cache-Control:
no-transform
directive.
If a proxy re-writes rewrites HTTPS links, replacement links
must have the scheme https
.
When forwarding requests originating from
HTTPS links proxies must include
a Via
header field as discussed under Note: 4.1.6.1 Proxy Treatment of Via
Header Field .
For clarity it is emphasized that it is
not possible for a transforming proxy to transform content accessed
via an HTTPS link without breaking end to end security.
When forwarding responses from servers
proxies must
notify the user of invalid server
certificates.
Operators of transforming
content transformation proxies
should make available interfaces that facilitate testing an interface through which the functions of
Web sites accessed the proxy can be exercised. The operations
possible through them and
this interface should must
make cover those
necessary to settle the outcome of all conformance statements
listed in section B.
The interface must be reachable
from terminals with browsing capabilities connected to the Web via
a conventional Internet access environment at the tester's
premises; accessing the interface may necessitate
adjusting standard Web browsing configuration parameters --
such interfaces as specifying a proxy IP address and port on a desktop
browser, or activating a WAP setting on a mobile browser.
Such access must be granted under fair, reasonable and non-discriminatory conditions. In particular:
it is available through normal Internet to
all, worldwide, whether or not they are W3C Members;
it does not impose any further conditions
or restrictions on the use of any technology, intellectual property
rights, or other restrictions on behaviour of the tester, but may
include reasonable, customary terms relating to operation or
maintenance of the relationship between tester and proxy operator
such as the following: choice of law and dispute resolution,
confidentiality of parameters to access paths. the interface,
disclaimer of liability.
See http://www.w3.org/TR/2009/WD-ct-guidelines-20091006/ics.html
application/vnd.wap.xhtml+xml
text/vnd.wap.wml
application/vnd.wap.wmlc
text/vnd.wap.wml+xml
text/vnd.wap.wmlscript
application/vnd.wap.wmlscriptc
image/vnd.wap.wbmp
application/vnd.wap.wbxml
application/vnd.wap.multipart.mixed
application/vnd.wap.multipart.related
application/vnd.wap.multipart.alternative
application/vnd.wap.multipart.form-data
image/x-up-wpng
image/x-up-bmp
-//OMA//DTD XHTML Mobile 1.2//EN
-//WAPFORUM//DTD XHTML Mobile 1.1//EN
-//WAPFORUM//DTD XHTML Mobile 1.0//EN
-//W3C//DTD XHTML Basic 1.1//EN
-//W3C//DTD XHTML Basic 1.0//EN
-//OPENWAVE//DTD XHTML 1.0//EN
-//OPENWAVE//DTD XHTML Mobile 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/1.0) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/1.1) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.0) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.1) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.2) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.3) 1.0//EN
-//W3C//DTD Compact HTML 1.0 Draft//EN
-//BBSW//DTD Compact HTML 2.0//EN
-//WAPFORUM//DTD WML 1.0//EN
-//WAPFORUM//DTD WML 1.1//EN
-//WAPFORUM//DTD WML 1.2//EN
-//WAPFORUM//DTD WML 1.3//EN
-//WAPFORUM//DTD WML 2.0//EN
-//PHONE.COM//DTD WML 1.1//EN
-//OPENWAVE.COM//DTD WML 1.3//EN
Using the notation defined in [POWDER Resource Grouping]:
<iriset> <includehosts>mobi</includehosts> </iriset>
User expression of preferences is referred to in several sections in this document. Those sections are:
User preferences are also referred to non-normatively under H.1.4 Varying Representations .
Note:
The following examples refer to requests with the GET method.
Request resource with original headers header
fields
If the response is a 406 response:
If the response contains Cache-Control:
no-transform
, forward it
Otherwise re-request
request again with altered headers header
fields
If the response is a 200 response:
If the response contains Vary:
User-Agent
, an appropriate link
element or
header, header
field, or Cache-Control: no-transform
, forward
it
Otherwise assess whether the 200 response is a form of "Request Unacceptable"
If it is not, forward it
If it is, re-request
request again with altered headers header
fields
Proxy receives a request for resource P that it has not encountered before
Proxy forwards this request
Response is 200 OK containing the text "Unsupported browser. Please get a different one or use a CT proxy."
Proxy determines that this equates to a 406 Status
and re-requests requests the resource from the origin server
again with altered headers header fields
(emulating a well known desktop browser)
Response is a desktop oriented representation of the resource
Proxy transforms this response into content that the user agent can display well and forwards it
Proxy receives a further request for the resource P
Based on evidence from the previous interaction
(e.g. that there was no Vary
header, header field,
that the response was not targeted at only the previous user in
that there was no Cache-Control: private
directive)
the CT proxy forwards the request with altered headers header
fields
Response is a desktop oriented representation of the resource
Proxy transforms this response into content that the user agent can display well and forwards it
Proxy receives a request for resource P, that it
has previously encountered as in B.2
G.2 Optimization
based on Previous Server Interaction
Proxy forwards request with altered headers header
fields
Response is 200 OK containing a Vary:
User-Agent
header field
Proxy notices that behavior has changed and
re-issues reissues the request with original headers header
fields
Response is 200 OK and proxy forwards it
Proxy receives a request for resource P
Proxy forwards request with original headers header
fields
Response is 200 OK with Vary:
User-Agent
and <link type="alternate"
media="handheld" href="P#id" />
where id is a document
local reference
Proxy forwards response as designed specifically for the requesting device
Proxy receives a request for resource P
Proxy forwards request with original headers header
fields
Response is 200 OK with <link
type="alternate" media="handheld" href="Q" />
and Q is
not P
Proxy requests Q with original headers header
fields
Response is 200 OK and proxy forwards it
Content providers may wish to follow these procedures in order to improve interoperability.
Servers should consider using an HTTP 406 Status (and not an HTTP 200 Status) if a request cannot be satisfied with content that meets the criteria specified by values of the HTTP request header fields. However, some browsers do not display the content of HTTP 406 Status responses.
Servers should consider using an HTTP 403
Status if concerned that the security of a link assumed to be
private has been compromised (for example this may be inferred by
the presence of a Via
HTTP header field
in an HTTPS request).
Cache-Control: no-transform
Servers should consider including a
Cache-Control:
no-transform
directive if one is
received in the HTTP request, as it may be an indication that the
client does not wish to receive a transformed response.
Include a Cache-Control: no-transform
directive if, for any reason, transformation of the
response is prohibited.
Note:
Including
a Cache-Control:
no-transform
directive can
disrupt the behavior of WAP Gateways, because it can inhibit such
proxies from converting WML to WMLC.
It is good practice to take account of user agent capabilities and formulate an appropriate experience according to those capabilities. It is good practice to provide a means for users to select among available representations, to default to the last selected representation and to provide a means of changing the selection.
Vary
HTTP Header FieldIf a server varies its representation
according to examination of received HTTP header fields then
[RFC 2616 HTTP]
describes how to use the
Vary
header field to indicate this.
Servers that are aware of the presence of
a transforming proxy, as identified by a Via
HTTP Header field
might alter their responses according to their knowledge of
specific proxy behavior. When doing so it is good practice to make
sure that the Internet content type for a response is correct for
the actual content (e.g. a server should not choose
Content-Type:
application/vnd.wap.xhtml+xml
because it suspects that proxies will not transform
content of this type, if its content is not valid
XHTML-MP).
If a server has distinct representations
that vary according to the target presentation media type, it can
inhibit transformation of the response by including a
Cache-Control:
no-transform
directive
(see H.1.3 Server Origination of
Cache-Control: no-transform ).
In addition, in HTML content it can
indicate the medium for which the representation is intended by
including a link
element identifying in its
media
attribute the target presentation media types of this
representation and setting the href
attribute to
"Same-Document Reference" (see [RFC 3986] section 4.4 ) and in
particular an empty href
attribute is a
"Same Document Reference".
In addition it is good practice to
include link
elements identifying the target presentation
media types of other available representations in a similar
manner.
If content for more than one presentation
media type is served from the same URI, it is better not to use a
link element identifying the presentation media types as the URI
will appear to be a "same document reference", indicating to a
client that this representation is suitable for all the named
presentation media types. Instead, use a Vary
HTTP header
field indicating that the response varies according to the
received User-Agent
HTTP header field.
Note:
Some examples of the use of the
link
element are included above in G Example Transformation
Interactions .
There are a number of well-known examples of solutions that seem
to their users as though they are using a browser, but because the
client software communicates with using
proprietary protocols and techniques, it is the combination of the
client and the in-network network component that is regarded as the HTTP
User Agent. The communication between the client and the
in-network network component is therefore out of scope of
this document.
Additionally, where some kind of administrative arrangement exists between a transforming proxy and an origin server for the purposes of transforming content on the origin server's behalf, this is also out of scope of this document.
In both of the above cases, it is recommended that when forwarding requests
good practice to origin servers that proxies adhere to the
provisions of this document in respect of providing information
about the device and the original IP address.
The BPWG believes that POWDER will represent a powerful mechanism by which a server may express transformation preferences. Future work in this area may recommend the use of POWDER to provide a mechanism for origin servers to indicate more precisely which alternatives they have and what transformation they are willing to allow on them, and in addition to provide for Content Transformation proxies to indicate which services they are able to perform.
link
HTTP Header FieldThe BPWG believes that the link
HTTP header
field which was removed from
recent drafts of HTTP, HTTP/1.1, and which is under discussion for
re-introduction, reintroduction, would represent a more general and
flexible mechanism than use of the HTML link
element,
as discussed in this recommendation.
The process of adapting content at the origin server, or transforming it in a proxy is likely to have a dependency on a repository of device descriptions. An origin server's willingness to allow a transforming proxy to transform content may depend on its evaluation of the trustworthiness of device description data that is being used. There is scope for enhancement of the trust relationship by some means of indicating this.
There is scope for further work to define how multiple proxies
may inter-operate. interoperate. A common case of multiple proxies is
where a network provider transforming proxy and a search engine
transforming proxy are both present.
The BPWG believes that amendments to HTTP are needed to improve
the inter operability interoperability of transforming proxies. For
example, HTTP does not provide a way to distinguish between
prohibition of any kind of transformation and the prohibition only
of restructuring (and not recoding or compression).
At present HTTP does not provide a mechanism for communicating
original header values (hence the use of
X-Device- headers as discussed field
values. The scheme based on X-Device prefixed fields
described under 4.1.5
Alteration of HTTP Header Field
Values ). records and clarifies an approach used to achieve this
effect by some content transformation proxies. This scheme relies
upon non-standard HTTP fields, which are identified by their prefix
as experimental according to IETF standards (notably RFC 822 and
RFC 2076), and are not included in the IANA registry of HTTP header
fields. While the mechanism defined in that section, based on
current practice, applies to conforming transformation proxy
deployments, it is possible that in future, in collaboration with
the IETF, this approach will be reconsidered. This implies that the
specified X-Device prefixed fields may, at some time, become
deprecated in favor of new equivalent fields, or that an entirely
different approach will be taken to representing such
values.
A number of mechanisms exist i
in HTTP which might be exploited given more
precise definition of their operation - for example the
OPTIONS
method and the HTTP 300 (Multiple Choices)
Status.
The editor acknowledges contributions of various kinds from members of the Mobile Web Best Practices Working Group and earlier from the Content Transformation Task Force of that group.
The editor acknowledges significant written contributions from: