Copyright
© 2009
© 2010
W3C
®
(
MIT
,
ERCIM
,
Keio
),
All
Rights
Reserved.
W3C
liability
,
trademark
and
document
use
rules
apply.
This document provides guidance to Content Transformation proxies as to whether and how to transform Web content.
Content Transformation proxies alter requests sent by user agents to servers and responses returned by servers so that the appearance, structure or control flow of Web applications are modified. Content Transformation proxies are mostly used to convert Web sites designed for desktop computers to a form suitable for mobile devices.
Based on current practice and standards, this document specifies mechanisms with which Content Transformation proxies should make their presence known to other parties, present the outcome of alterations performed on HTTP traffic, and react to indications set by clients or servers to constrain these alterations.
The objective is to reduce undesirable effects on Web applications, especially mobile-ready ones, and to limit the diversity in the modes of operation of Content Transformation proxies, while at the same time allowing proxies to alter content that would otherwise not display successfully on mobile devices.
Important considerations regarding the impact on security are highlighted.
This document is an editors' copy that has no official standing.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/ .
This document reflects group resolutions on comments received on the previous Last Call Working Draft .
Publication as a Group Working Draft of a proposed normative Recommendation does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document has been produced by the Mobile Web Best Practices Working Group as part of the Mobile Web Initiative . Please send comments on this document to the Working Group's public email list public-bpwg-ct@w3.org , a publicly archived mailing list .
This document was produced under the 5 February 2004 W3C Patent Policy . W3C maintains a public list of patent disclosures made in connection with this document; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification must disclose the information in accordance with section 6 of the W3C Patent Policy .
1
Introduction
(Non-Normative)
1.1
Purpose
1.2
Audience
1.3
Scope
1.4
Principles
1.4.1
IAB
Considerations
1.4.2
Priority
of
Intention
2
Terminology
(Normative)
2.1
Types
of
Proxy
2.2
Types
of
Transformation
2.3
User
Interaction
3
Conformance
(Normative)
3.1
Classes
of
Product
3.2
Normative
and
Informative
Parts
3.3
Normative
Language
for
Conformance
Requirements
3.4
Transformation
Deployment
Conformance
4
Behavior
of
Components
(Normative)
4.1
Proxy
Forwarding
of
Request
4.1.1
Applicable
HTTP
Methods
4.1.2
no-transform
directive
in
Request
4.1.3
Treatment
of
Requesters
that
are
not
Web
browsers
4.1.4
Serving
Cached
Responses
4.1.5
Alteration
of
HTTP
Header
Field
Values
4.1.5.1
Content
Tasting
4.1.5.2
Avoiding
"Request
Unacceptable"
Responses
4.1.5.3
User
Selection
of
Restructured
Experience
4.1.5.4
Sequence
of
Requests
4.1.5.5
Original
Header
Fields
4.1.6
Additional
HTTP
Header
Fields
4.1.6.1
Proxy
Treatment
of
Via
Header
Field
4.2
Proxy
Forwarding
of
Response
to
User
Agent
4.2.1
Applicable
Responses
4.2.2
User
Preferences
4.2.3
4.2.2
Receipt
of
Cache-Control:
no-transform
4.2.4
4.2.3
Use
of
Cache-Control:
no-transform
4.2.5
4.2.4
Server
Rejection
of
HTTP
Request
4.2.6
4.2.5
Receipt
of
Vary
HTTP
Header
Field
4.2.7
4.2.6
Link
to
"handheld"
Representation
4.2.8
4.2.7
WML
Content
4.2.9
4.2.8
Proxy
Decision
to
Transform
4.2.9.1
4.2.8.1
Alteration
of
Response
4.2.9.2
4.2.8.2
Link
Rewriting
4.2.9.3
4.2.8.3
HTTPS
Link
Rewriting
5
Testing
(Normative)
A
References
B
Conformance
Statement
C
Internet
Content
Types
associated
with
Mobile
Content
D
Internet
Content
Types
associated
with
Data
Content
E
DOCTYPEs
Associated
with
Mobile
Content
E
F
URI
Patterns
Associated
with
Mobile
Web
Sites
F
G
Summary
of
User
Preference
Handling
G
H
Example
Transformation
Interactions
(Non-Normative)
G.1
H.1
Basic
Content
Tasting
by
Proxy
G.2
H.2
Optimization
based
on
Previous
Server
Interaction
G.3
H.3
Optimization
based
on
Previous
Server
Interaction,
Server
has
Changed
its
Operation
G.4
H.4
Server
Response
Indicating
that
this
Representation
is
Intended
for
the
Target
Device
G.5
H.5
Server
Response
Indicating
that
another
Representation
is
Intended
for
the
Target
Device
H
I
Informative
Guidance
for
Origin
Servers
(Non-Normative)
H.1
I.1
Server
Response
to
Proxy
H.1.1
I.1.1
Use
of
HTTP
406
Status
H.1.2
I.1.2
Use
of
HTTP
403
Status
H.1.3
I.1.3
Server
Origination
of
Cache-Control:
no-transform
H.1.4
I.1.4
Varying
Representations
H.1.4.1
I.1.4.1
Use
of
Vary
HTTP
Header
Field
H.1.4.2
I.1.4.2
Indication
of
Intended
Presentation
Media
Type
of
Representation
I
J
Applicability
to
Transforming
Solutions
which
are
Out
of
Scope
(Non-Normative)
J
K
Scope
for
Future
Work
(Non-Normative)
J.1
K.1
POWDER
J.2
K.2
link
HTTP
Header
Field
J.3
K.3
Sources
of
Device
Information
J.4
K.4
Inter
Proxy
Communication
J.5
K.5
Explicit
Consent
K.6
Amendment
to
and
Refinement
of
HTTP
K
L
Acknowledgments
(Non-Normative)
Within this document Content Transformation refers to the manipulation of requests to, and responses from, an origin server. This manipulation is carried out by proxies in order to provide a better user experience of content that would otherwise result in an unsatisfactory experience on the device making the request.
The W3C Mobile Web Best Practices Working Group neither approves nor disapproves of Content Transformation, but recognizes that is being deployed widely across mobile data access networks. The deployments are widely divergent to each other, with many non-standard HTTP implications, and no well-understood means either of identifying the presence of such transforming proxies, nor of controlling their actions. This document establishes a framework to allow that to happen.
The overall objective of this document is to provide a means, as far as is practical, for users to be provided with at least a "functional user experience" [Device Independence Glossary] of the Web, when mobile, taking into account the fact that an increasing number of content providers create experiences specially tailored to the mobile context which they do not wish to be altered by third parties. Equally it takes into account the fact that there remain a very large number of Web sites that do not provide a functional user experience when perceived on many mobile devices.
It is stressed that this document is unlikely to be the last word on this topic. As noted below ( 1.3 Scope ) it is out of scope to provide a thoroughgoing solution to control of transforming proxies, though that would appear to be needed. It is an attempt to improve a situation at a point in time where there appears to be disregard of the provisions of HTTP - and is primarily a reminder and an encouragement to follow those provisions more closely.
The audience for this document is creators of Content Transformation proxies and purchasers and operators of such proxies. The document also contains non-normative guidance for content providers whose services may be accessed by means of such proxies.
The recommendations in this document refer only to "Web browsing" - i.e. access by user agents that are intended primarily for interaction by users with HTML Web pages (Web browsers) using HTTP. Clients that interact with proxies using mechanisms other than HTTP (and that typically involve the download of a special client) are out of scope, and are considered to be a distributed user agent. Proxies which are operated in the control of or under the direction of the operator of an origin server are similarly considered to be a distributed origin server and hence out of scope.
The W3C Mobile Web Best Practices Working Group (BPWG) is not chartered to create new technology - its role is to advise on best practice for use of existing technology. In satisfying Content Transformation requirements, existing HTTP header fields, directives and behaviors must be respected, and as far as is practical, no extensions to [RFC 2616 HTTP] are to be used.
The recommendations in this document refer to interactions of a proxy and do not refer to any presumed aspects of the internal operation of the proxy. For this reason, the document does not discuss use of "allow" and "disallow" lists (though it does discuss behavior that is induced by the implementation of such lists). In addition it does not discuss details of how transformation is carried out except if this is reflected in interoperability. For this reason, it does not discuss the insertion or insertion of headers and footers or any other specific behaviors (though it does discuss the need for essential user interaction of some form).
Moral, legal and other similar questions are not in scope of this document. The BPWG does not have authority or expertise to comment one way or another about setting precedent or authorising any particular behavior or its absence.
The BPWG made reference to Internet Architecture Board (IAB) work on "Open Pluggable Edge Services" [RFC 3238 OPES] for various principles that underlie behavior of proxies. In this work the IAB expressed its concerns about privacy, control, monitoring, and accountability of such services.
The Web allows users considerable flexibility in respect of the representation of content. At the same time, Content Providers may have a preferred manner in which they wish their content to be represented. Content Transformation must reconcile these contrasting factors. In creating this Recommendation the BPWG has determined that Content Transformation proxies should respect Content Providers intentions, where they are expressed, but may allow users to choose other representations, except where Content Providers specifically prohibit this.
The
BPWG
recognizes
that
there
is
neither
a
systematic
vocabulary
for
Content
Provider
Intentions,
nor
a
systematic
means
of
expression
of
such
intentions.
There
is
scope
for
further
work
in
this
area
(see
J
K
Scope
for
Future
Work
).
Alteration of HTTP requests and responses is not prohibited by HTTP other than in the circumstances referred to in [RFC 2616 HTTP] Section 13.5.2 and Section 14.9.5 .
HTTP defines two types of proxy: transparent proxies and non-transparent proxies. As discussed in [RFC 2616 HTTP] Section 1.3, Terminology :
"A transparent proxy is a proxy that does not modify the request or response beyond what is required for proxy authentication and identification. A non-transparent proxy is a proxy that modifies the request or response in order to provide some added service to the user agent, such as group annotation services, media type transformation, protocol reduction, or anonymity filtering. Except where either transparent or non-transparent behavior is explicitly stated, the HTTP proxy requirements apply to both types of proxies."
This document elaborates the behavior of non-transparent proxies, when used for Content Transformation in the context discussed in [CT Landscape] .
Transforming
proxies
can
carry
out
a
wide
variety
of
operations.
In
this
document
we
categorize
these
operations
as
follows:
follows
(noting
that
these
are
general
concepts
that
we
do
not
formalize
further):
Alteration of Requests
Transforming proxies process requests in a number of ways, especially replacement of various request header fields to avoid HTTP 406 Status responses (if a server can not provide content that is compatible with the original HTTP request header fields) and at user request.
Alteration of Responses
There are three classes of operation on responses:
Restructuring content
Restructuring content is a process whereby the original layout is altered so that content is added or removed or where the spatial or navigational relationship of parts of content is altered, e.g. linearization (i.e. reordering presentation elements, especially tables, so that they fit on a narrow display and can be traversed without horizontal scrolling) or pagination (i.e. splitting a document too large to be stored in or transmitted to the terminal in one piece, so that it can be nevertheless accessed by browsing through a succession of smaller interlinked documents). It also includes rewriting URIs so that subsequent requests are routed via the proxy handling the response.
Recoding content
Recoding content is a process whereby the layout of the content remains the same, but details of its encoding may be altered. Examples include re-encoding HTML as XHTML, correcting invalid markup in HTML, conversion of images between formats (but not, for example, reducing animations to static images).
Optimizing content
Optimizing content includes removing redundant white space, re-compressing images (without loss of fidelity) and compressing for transfer.
At various points in this document there is reference to "notifying the user", "informing the user" - in general making the user aware that a situation exists or interacting with the user to solicit a choice of options. The expectation is that such user interaction is conducted in a way that allows the user to perceive and interact with such information or choices in the same way as they interact with the Web sites that they are visiting.
The Content Transformation Guidelines specification has one class of products:
A Transformation Deployment is the provision of non-transparent components in the path of HTTP requests and responses. Provisions that are applicable to a Transformation Deployment are identified in this document by use of the term "transforming proxy" or "proxy" in the singular or plural.
Normative parts of this document are identified by the use of "(Normative)" following the section name. Informative parts are identified by use of "(Non-Normative)" following the section name.
The key words must , must not , required , shall , shall not , should , should not , recommended , not recommended , may , and optional in this Recommendation have the meaning defined in [RFC 2119] .
A Transformation Deployment conforms to these guidelines if it follows the statements in 3.4 Transformation Deployment Conformance , 4.1 Proxy Forwarding of Request , 4.2 Proxy Forwarding of Response to User Agent and 5 Testing (Normative) .
A Transformation Deployment that wishes to claim conformance must make available a conformance statement B Conformance Statement that specifies the reasons for non-compliance with any clauses containing the key words " should " and " should not ", " recommended " and " not recommended ".
Conformance statements must be sent to public-content-transformation-conformance@w3.org . Public archives of this list may be found at http://lists.w3.org/Archives/Public/public-content-transformation-conformance/ .
User agents sometimes issue HTTP HEAD requests in order to determine if a resource is of a type and/or size that they are capable of handling. A transforming proxy may convert a HEAD request into a GET request (in order to determine the characteristics of a transformed response that it would return if the user agent subsequently issued a GET request for the same resource).
If the HTTP method is altered from HEAD to GET, proxies should (providing such action is in accordance with normal HTTP caching rules) cache the response so that a second GET request for the same content is not required (see also 4.1.4 Serving Cached Responses ).
Other than to convert between HEAD and GET proxies must not alter request methods.
no-transform
directive
in
Request
If
the
request
contains
a
Cache-Control:
no-transform
directive,
proxies
must
not
alter
the
request
other
than
to
comply
with
transparent
HTTP
behavior
defined
in
[RFC
2616
HTTP]
sections
section
14.9.5
and
section
13.5.2
and
to
add
header
fields
as
described
in
4.1.6
Additional
HTTP
Header
Fields
below.
Note:
An
example
of
the
use
of
Cache-Control:
no-transform
is
the
issuing
of
asynchronous
HTTP
requests,
perhaps
by
means
of
XMLHttpRequest
[XHR]
,
which
may
include
such
a
directive
in
order
to
prevent
transformation
of
both
the
request
and
the
response.
Before
altering
aspects
of
an
HTTP
request
requests
and
responses
proxies
need
to
take
account
of
the
fact
that
HTTP
is
used
as
a
transport
mechanism
for
many
applications
other
than
"Traditional
Browsing".
Alteration
of
HTTP
requests
for
those
Increasingly
browser
based
applications
can
involve
exchanges
of
data
using
XmlHttpRequest
(see
4.2.8
Proxy
Decision
to
Transform
)
and
alteration
of
such
exchanges
is
likely
to
cause
serious
misoperation.
Aside from the usual caching procedures defined in [RFC 2616 HTTP] , in some circumstances, proxies may paginate responses and where this is the case a request may be for a subsequent page of a previously requested resource. In this case proxies may for the sake of consistency of representation serve stale data but when doing so should notify the user that this is the case and must provide a simple means of retrieving a fresh copy.
Aside
from
Other
than
the
usual
procedures
defined
in
modifications
required
by
[RFC
2616
HTTP]
proxies
should
not
modify
the
values
of
header
fields
other
than
the
User-Agent
,
Accept
,
Accept-Charset
,
Accept-Encoding
,
and
Accept-Language
header
fields
and
must
not
delete
header
fields.
It
must
be
possible
for
the
server
to
reconstruct
the
original
User
Agent
user
agent
originated
header
fields
by
copying
directly
from
the
corresponding
X-Device
header
field
values
(see
4.1.5.5
Original
Header
Fields
).
Other than to comply with transparent HTTP operation, proxies should not modify any request header fields unless one of the following applies:
the
user
would
be
prohibited
from
accessing
content
as
a
result
of
the
server
responding
that
the
request
is
"unacceptable"
(see
4.2.5
4.2.4
Server
Rejection
of
HTTP
Request
);
the user has specifically requested a restructured desktop experience (see 4.1.5.3 User Selection of Restructured Experience );
the
request
is
part
of
a
sequence
of
requests
to
comprising
either
included
resources
or
linked
resources
on
the
same
Web
site
and
either
it
is
technically
infeasible
not
to
adjust
the
request
because
of
earlier
interaction,
or
because
doing
so
preserves
consistency
(see
4.1.5.4
Sequence
of
user
experience.
Requests
).
These circumstances are detailed in the following sections.
Note:
It
is
emphasized
that
requests
must
not
be
altered
in
the
presence
of
Cache-Control:
no-transform
as
described
under
4.1.2
no-transform
directive
in
Request
.
Note:
In this section, the concept of "Web site" is used (rather than "origin server") as some origin servers host many different Web sites. Since the concept of "Web site" is not strictly defined, proxies should use heuristics including comparisons of domain name to assess whether resources form part of the same "Web site".
Note:
The
URI
referred
to
in
the
request
plays
no
part
in
determining
whether
or
not
to
alter
HTTP
request
header
field
values.
In
particular
the
patterns
mentioned
in
4.2.9
4.2.8
Proxy
Decision
to
Transform
are
not
material.
While
complying
with
this
section
The
theoretical
idempotency
4.1.5
Alteration
of
GET
requests
is
not
always
respected
by
servers.
In
order,
as
far
as
possible,
to
avoid
misoperation
HTTP
Header
Field
Values
and
4.2.5
Receipt
of
such
content,
Vary
HTTP
Header
Field
proxies
should
avoid
issuing
duplicate
requests
and
specifically
should
not
issue
duplicate
making
repeated
requests
for
comparison
purposes.
the
same
resource.
Note:
While HTTP does not prohibit repetition of GET requests, repeated requests place an unnecessary load on the network and server.
A
proxy
may
reissue
a
request
with
altered
HTTP
header
field
values
if
a
previous
request
with
unaltered
values
resulted
in
the
origin
server
rejecting
the
request
as
"unacceptable"
(see
4.2.5
4.2.4
Server
Rejection
of
HTTP
Request
).
A
proxy
may
apply
heuristics
of
various
kinds
to
assess,
in
advance
of
sending
unaltered
header
field
values,
whether
the
request
is
likely
to
cause
a
"request
unacceptable"
response.
If
it
determines
that
this
is
likely
then
it
may
alter
header
field
values
without
sending
unaltered
values
in
advance,
providing
that
it
subsequently
assesses
the
response
as
described
under
4.2.6
4.2.5
Receipt
of
Vary
HTTP
Header
Field
below,
and
is
prepared
to
reissue
the
request
with
unaltered
header
fields,
and
alter
its
subsequent
behavior
in
respect
of
the
Web
site
so
that
unaltered
header
fields
are
sent.
A
proxy
must
not
reissue
a
POST
request
with
altered
header
fields
when
the
response
to
the
unaltered
POST
request
has
HTTP
status
code
200
(in
other
words,
as
it
may
only
send
the
altered
request
for
a
POST/PUT
request
when
the
unaltered
one
resulted
in
an
HTTP
406
response,
and
not
a
"request
unacceptable"
response).
is
unsafe
(see
[RFC
2616
HTTP]
Section
9.1.1
).
Proxies must assume that by default users will wish to receive a representation prepared by the Web site.
Proxies
may
offer
users
an
option
to
choose
to
view
a
restructured
experience
even
when
a
Web
site
offers
a
choice
of
user
experience.
If
a
user
has
made
such
a
choice
then
proxies
may
alter
header
field
values
when
requesting
resources
in
order
to
reflect
that
choice,
but
must
,
on
receipt
of
an
indication
from
a
Web
site
that
it
offers
alternative
representations
(see
H.1.4.2
I.1.4.2
Indication
of
Intended
Presentation
Media
Type
of
Representation
),
inform
the
user
of
that
and
allow
them
to
select
an
alternative
representation.
Proxies
must
assess
whether
a
user's
expressed
preference
for
a
restructured
representation
is
still
valid
if
a
Web
site
changes
its
choice
of
representations
(see
4.2.6
4.2.5
Receipt
of
Vary
HTTP
Header
Field
).
When
requesting
resources
that
are
included
resources
(e.g.
style
sheets,
images),
proxies
should
make
the
request
for
such
resources
with
the
same
User-Agent
header
field
as
the
request
for
the
resource
from
which
they
are
referenced.
For
the
purpose
of
consistency
of
representation,
proxies
may
request
linked
resources
(e.g.
those
referenced
using
the
a
element)
that
form
part
of
the
same
Web
site
as
a
previously
requested
resource
with
the
same
header
fields
as
the
resource
from
which
they
are
referenced.
When requesting linked resources that do not form part of the same Web site as the resource from which they are linked, proxies should not base their choice of header fields on a consistency of presentation premise.
When
forwarding
an
HTTP
request
with
altered
HTTP
header
fields,
in
addition
to
complying
with
the
rules
of
normal
HTTP
operation,
proxies
must
include
in
the
request
copies
of
the
unaltered
header
field
values
in
the
form
"X-Device-"<original
header
name>
.
For
example,
if
the
User-Agent
header
field
has
been
altered,
an
X-Device-User-Agent
header
field
must
be
added
with
the
value
of
the
received
User-Agent
header
field.
Specifically the following mapping must be used:
Original | Replacement | Ref |
---|---|---|
User-Agent
|
X-Device-User-Agent
|
RFC2616 Section 14.43 |
Accept
|
X-Device-Accept
|
RFC2616 Section 14.1 |
Accept-Charset
|
X-Device-Accept-Charset
|
RFC2616 Section 14.2 |
Accept-Encoding
|
X-Device-Accept-Encoding
|
RFC2616 Section 14.3 |
Accept-Language
|
X-Device-Accept-Language
|
RFC2616 Section 14.4 |
Note:
The
X-Device-
prefixed
header
names
listed
in
this
section
have
been
provisionally
registered
with
IANA
(see
Provisional
Message
Header
Field
Names
).
Note:
The
X-Device-
prefix
was
chosen
primarily
on
the
basis
that
this
is
a
an
already
existing
convention.
It
is
noted
that
the
values
encoded
in
such
header
fields
may
not
ultimately
derive
from
a
device,
they
are
merely
received
fields.
The
treatment
of
received
X-Device
header
fields,
which
may
happen
where
there
are
multiple
transforming
proxies,
is
undefined
(see
J
K
Scope
for
Future
Work
).
Irrespective
of
the
presence
of
a
no-transform
directive:
proxies
should
add
the
IP
address
of
the
initiator
of
the
request
to
the
end
of
a
comma
separated
list
in
an
X-Forwarded-For
HTTP
header
field;
proxies
must
include
a
Via
HTTP
header
field
(see
4.1.6.1
Proxy
Treatment
of
Via
Header
Field
).
Via
Header
Field
Proxies
must
(in
accordance
with
RFC
2616)
include
a
Via
HTTP
header
field
indicating
their
presence
and
should
indicate
their
ability
to
transform
content
by
including
a
comment
in
the
Via
HTTP
header
field
consisting
of
the
URI
"http://www.w3.org/ns/ct".
When
forwarding
Via
header
fields,
proxies
should
not
alter
them
by
removing
comments
from
them.
Note:
According
to
[RFC
2616
HTTP]
Section
14.45
Via
header
field
comments
"
may
be
removed
by
any
recipient
prior
to
forwarding
the
message".
However,
the
justification
for
removing
such
comments
is
based
on
memory
limitations
of
early
proxies.
Most
modern
proxies
do
not
suffer
such
limitations.
In
the
following,
proxies
must
check
for
the
presence
of
equivalent
<meta
http-equiv>
elements
in
HTML
content,
if
the
relevant
HTTP
header
field
is
not
present.
Proxies
must
provide
a
means
for
users
to
express
preferences
for
inhibiting
content
transformation
even
when
this
content
transformation
has
been
chosen
by
the
user
as
the
default
behavior.
Those
preferences
must
be
maintained
on
a
user
by
user
and
Web
site
by
Web
site
basis.
Proxies
must
solicit
re-expression
of
preferences
in
respect
of
a
server
if
the
server
starts
to
indicate
that
it
offers
varying
responses
as
discussed
under
4.2.6
4.2.5
Receipt
of
Vary
HTTP
Header
Field
.
Cache-Control:
no-transform
If
the
response
includes
a
Cache-Control:
no-transform
directive
then
proxies
must
not
alter
it
other
than
to
comply
with
transparent
HTTP
behavior
as
described
in
[RFC
2616
HTTP]
Section
13.5.2
and
Section
14.9.5
and
other
than
as
follows.
If
a
proxy
determines
that
a
resource
as
currently
represented
is
likely
to
cause
serious
misoperation
of
the
user
agent
then
it
may
advise
the
user
that
this
is
the
case
and
must
provide
the
option
for
the
user
to
continue
with
unaltered
content
(and
may
provide
other
options
too).
.
Cache-Control:
no-transform
Proxies
may
use
Cache-Control:
no-transform
to
inhibit
transformation
by
further
proxies.
Proxies may treat responses with an HTTP 200 Status as though they were responses with an HTTP 406 Status if it has determined that the content (e.g. "Your browser is not supported") is equivalent to a response with an HTTP 406 Status.
Vary
HTTP
Header
Field
A
proxy
may
not
be
carrying
out
content
tasting
as
described
under
4.1.5.2
Avoiding
"Request
Unacceptable"
Responses
if
it
anticipates
receiving
a
"request
unacceptable"
response.
However,
if
it
makes
a
request
with
altered
header
fields
in
these
circumstances,
and
receives
a
response
containing
a
Vary
header
field
referring
to
one
of
the
altered
header
fields
then
it
should
request
the
resource
again
with
unaltered
header
fields.
It
should
also
update
whatever
heuristics
it
uses
so
that
unaltered
header
fields
are
presented
first
in
subsequent
requests
for
this
resource.
If
the
response
is
an
HTML
response
and
it
contains
a
<link
rel="alternate"
media="handheld"
/>
element,
element
(and
the
user
agent
is
determined
as
being
"handheld"),
a
proxy
should
request
and
process
the
referenced
resource,
unless
the
resource
referenced
is
the
current
representation
.
Note:
In
this
document
the
term
current
representation
means
a
"same
document
reference"
as
defined
in
[RFC
3986]
Section
4.4
,
with
the
addition
that
if
a
Vary
HTTP
header
field
was
present
on
the
response
then
it
is
the
same
representation
if
the
values
of
the
HTTP
header
fields
of
the
request
have
not
been
altered.
If the content is WML proxies should act in a transparent manner.
Note:
This does not affect the operation of proxies that are also WAP Gateways.
In
the
absence
of
a
Vary
or
no-transform
directive
(or
a
meta
HTTP-Equiv
element
containing
Cache-Control:
no-transform
)
proxies
should
not
transform
content
matching
any
of
the
following
rules
unless
the
user
has
specifically
requested
transformation:
the
content
is
HTML
and
contains
<link
rel="alternate"
media="handheld"/>
with
a
reference
to
the
current
representation
;
the
DOCTYPE
of
the
content
(if
it
has
one)
indicates
XHTML-MP,
XHTML
Basic,
WML
or
iMode
as
listed
in
D
E
DOCTYPEs
Associated
with
Mobile
Content
;
the
Content-Type
has
a
value
listed
in
C
Internet
Content
Types
associated
with
Mobile
Content
.
the
URI
of
the
response
(following
redirection
or
as
indicated
by
the
Content-Location
HTTP
header
field)
matches
a
pattern
listed
in
E
F
URI
Patterns
Associated
with
Mobile
Web
Sites
;
the response contains a resource that is referenced as an included resource suitable for "handheld" in a resource that was itself handled transparently;
the
Content-Type
indicates
that
the
content
is
"data"
-
some
values
are
listed
in
D
Internet
Content
Types
associated
with
Data
Content
;
a claim of mobileOK Basic [mobileOK Basic Tests] conformance is indicated (see [mobileOK Scheme] for how such a claim may be indicated).
Other factors that a proxy may take into account:
The Web site (see note ) has previously shown that it is contextually aware, even if the present response does not indicate this;
the
user
agent
has
features
(such
as
linearization
or
zoom)
zoom,
or
is
a
desktop
device
using
a
mobile
network
for
access)
that
allow
it
to
present
the
content
unaltered;
the response contains client side scripts that may misoperate if the resource is restructured;
the
response
is
an
HTML
response
and
it
includes
<link>
elements
specifying
alternatives
according
to
presentation
media
type.
Note:
Other than as noted in this section the nature of restructuring that is carried out, any character encoding alterations and what is omitted and what is inserted is, as discussed in 1.3 Scope , out of scope of this document.
If a proxy alters the response then:
It
must
add
a
Warning
214
Transformation
Applied
HTTP
header
field;
The altered content should validate according to an appropriate published formal grammar and if XML must be well-formed ;
It should indicate to the user that the content has been transformed for mobile presentation and provide an option to view the original, unmodified content.
Note:
In
this
document
two
URIs
have
the
Same-Origin
if
the
scheme
component
and
the
host
and
port
subcomponents,
as
defined
in
[RFC
3986]
,
all
match..
match.
Section
6
of
[RFC
3986]
discusses
URI
comparison.
Some
proxy
deployments
have
to
"rewrite"
links
in
content
in
order
for
the
User
Agent
user
agent
to
request
the
referenced
resources
through
the
proxy.
In
so
doing,
proxies
make
unrelated
resources
appear
as
though
they
have
the
same-origin
and
hence
there
is
a
danger
of
introducing
security
vulnerabilities.
Note:
This section (on link rewriting) refers also to insertion of links, frame flattening and any other techniques that introduces the "same-origin" issue.
Note:
Link rewriting is always used by CT Proxies that are accessed as an origin server initially, e.g. which provide mobile adapted web search and navigation to the web pages returned in the search results, or to which the browser is redirected through the CT Proxy for adaptation of a web page. Link rewriting may be used by CT Proxies acting as normal HTTP proxies (e.g. configured or transparent) for the browser, but may not be required since all browser requests flow through the CT Proxy.
Proxies must not rewrite links when content transformation is prohibited.
Proxies must preserve security between requests for domains that are not same-origin in respect of cookies and scripts.
Note:
For clarity it is emphasized that it is not possible for a transforming proxy to transform content accessed via an HTTPS link without breaking end-to-end security.
Interception of HTTPS and the circumstances in which it might be permissible is not a "mobile" question, as such, but is highly pertinent to this document. The BPWG is aware that interception of HTTPS happens in many networks today. Interception of HTTPS is inherently problematic and may be unsafe. The BPWG would like to refer to protocol based "two party consent" mechanisms, but such mechanisms do not exist at the time of writing of this document.
The practice of intercepting HTTPS links is strongly NOT RECOMMENDED .
If a proxy rewrites HTTPS links, it must advise the user of the security implications of doing so and must provide the option to bypass it and to communicate with the server directly.
Notwithstanding
anything
else
in
this
document,
proxies
must
not
rewrite
HTTPS
links
in
the
presence
of
a
Cache-Control:
no-transform
directive.
If
a
proxy
rewrites
HTTPS
links,
replacement
links
must
have
the
scheme
https
.
When
forwarding
requests
originating
from
HTTPS
links
proxies
must
include
a
Via
header
field
as
discussed
under
4.1.6.1
Proxy
Treatment
of
Via
Header
Field
.
When forwarding responses from servers proxies must notify the user of invalid server certificates.
Operators of content transformation proxies should make available an interface through which the functions of the proxy can be exercised. The operations possible through this interface must cover those necessary to settle the outcome of all conformance statements listed in section B.
The interface must be reachable from terminals with browsing capabilities connected to the Web via a conventional Internet access environment at the tester's premises; accessing the interface may necessitate adjusting standard Web browsing configuration parameters -- such as specifying a proxy IP address and port on a desktop browser, or activating a WAP setting on a mobile browser.
Such access must be granted under fair, reasonable and non-discriminatory conditions. In particular:
it is available to all, worldwide, whether or not they are W3C Members;
it does not impose any further conditions or restrictions on the use of any technology, intellectual property rights, or other restrictions on behaviour of the tester, but may include reasonable, customary terms relating to operation or maintenance of the relationship between tester and proxy operator such as the following: choice of law and dispute resolution, confidentiality of parameters to access the interface, disclaimer of liability.
Editorial Note: Update to final location
See
http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/ics-100125
http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/CT/editors-drafts/Guidelines/ics-090923
This list is not exhaustive and is likely to change.
application/vnd.wap.xhtml+xml
text/vnd.wap.wml
application/vnd.wap.wmlc
text/vnd.wap.wml+xml
text/vnd.wap.wmlscript
application/vnd.wap.wmlscriptc
image/vnd.wap.wbmp
application/vnd.wap.wbxml
application/vnd.wap.multipart.mixed
application/vnd.wap.multipart.related
application/vnd.wap.multipart.alternative
application/vnd.wap.multipart.form-data
image/x-up-wpng
image/x-up-bmp
This list is not exhaustive and is likely to change.
application/json
application/soap+xml
application/soap+fastinfoset
application/fastsoap
application/fastinfoset
This list is not exhaustive and is likely to change.
-//OMA//DTD XHTML Mobile 1.2//EN
-//WAPFORUM//DTD XHTML Mobile 1.1//EN
-//WAPFORUM//DTD XHTML Mobile 1.0//EN
-//W3C//DTD XHTML Basic 1.1//EN
-//W3C//DTD XHTML Basic 1.0//EN
-//OPENWAVE//DTD XHTML 1.0//EN
-//OPENWAVE//DTD XHTML Mobile 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/1.0) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/1.1) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.0) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.1) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.2) 1.0//EN
-//i-mode group (ja)//DTD XHTML i-XHTML (Locale/Ver.=ja/2.3) 1.0//EN
-//W3C//DTD Compact HTML 1.0 Draft//EN
-//BBSW//DTD Compact HTML 2.0//EN
-//WAPFORUM//DTD WML 1.0//EN
-//WAPFORUM//DTD WML 1.1//EN
-//WAPFORUM//DTD WML 1.2//EN
-//WAPFORUM//DTD WML 1.3//EN
-//WAPFORUM//DTD WML 2.0//EN
-//PHONE.COM//DTD WML 1.1//EN
-//OPENWAVE.COM//DTD WML 1.3//EN
Using the notation defined in [POWDER Resource Grouping] :
<iriset> <includehosts>mobi</includehosts> </iriset>
User expression of preferences is referred to in several sections in this document. Those sections are:
User
preferences
are
also
referred
to
non-normatively
under
H.1.4
I.1.4
Varying
Representations
.
Note:
The following examples refer to requests with the GET method.
Request resource with original header fields
If the response is a 406 response:
If
the
response
contains
Cache-Control:
no-transform
,
forward
it
Otherwise request again with altered header fields
If the response is a 200 response:
If
the
response
contains
Vary:
User-Agent
,
an
appropriate
link
element
or
header
field,
or
Cache-Control:
no-transform
,
forward
it
Otherwise assess whether the 200 response is a form of "Request Unacceptable"
If it is not, forward it
If it is, request again with altered header fields
Proxy receives a request for resource P that it has not encountered before
Proxy forwards this request
Response is 200 OK containing the text "Unsupported browser. Please get a different one or use a CT proxy."
Proxy determines that this equates to a 406 Status and requests the resource from the origin server again with altered header fields (emulating a well known desktop browser)
Response is a desktop oriented representation of the resource
Proxy transforms this response into content that the user agent can display well and forwards it
Proxy receives a further request for the resource P
Based
on
evidence
from
the
previous
interaction
(e.g.
that
there
was
no
Vary
header
field,
that
the
response
was
not
targeted
at
only
the
previous
user
in
that
there
was
no
Cache-Control:
private
directive)
the
CT
proxy
forwards
the
request
with
altered
header
fields
Response is a desktop oriented representation of the resource
Proxy transforms this response into content that the user agent can display well and forwards it
Proxy
receives
a
request
for
resource
P,
that
it
has
previously
encountered
as
in
G.2
H.2
Optimization
based
on
Previous
Server
Interaction
Proxy forwards request with altered header fields
Response
is
200
OK
containing
a
Vary:
User-Agent
header
field
Proxy notices that behavior has changed and reissues the request with original header fields
Response is 200 OK and proxy forwards it
Proxy receives a request for resource P
Proxy forwards request with original header fields
Response
is
200
OK
with
Vary:
User-Agent
and
<link
type="alternate"
media="handheld"
href="P#id"
/>
where
id
is
a
document
local
reference
Proxy forwards response as designed specifically for the requesting device
Proxy receives a request for resource P
Proxy forwards request with original header fields
Response
is
200
OK
with
<link
type="alternate"
media="handheld"
href="Q"
/>
and
Q
is
not
P
Proxy requests Q with original header fields
Response is 200 OK and proxy forwards it
Content providers may wish to follow these procedures in order to improve interoperability.
Servers should consider using an HTTP 406 Status (and not an HTTP 200 Status) if a request cannot be satisfied with content that meets the criteria specified by values of the HTTP request header fields. However, some browsers do not display the content of HTTP 406 Status responses.
Servers
should
consider
using
an
HTTP
403
Status
if
concerned
that
the
security
of
a
link
assumed
to
be
private
has
been
compromised
(for
example
this
may
be
inferred
by
the
presence
of
a
Via
HTTP
header
field
in
an
HTTPS
request).
Cache-Control:
no-transform
Servers
should
consider
including
a
Cache-Control:
no-transform
directive
if
one
is
received
in
the
HTTP
request,
as
it
may
be
an
indication
that
the
client
does
not
wish
to
receive
a
transformed
response.
Include
a
Cache-Control:
no-transform
directive
if,
for
any
reason,
transformation
of
the
response
is
prohibited.
Note:
Including
a
Cache-Control:
no-transform
directive
can
disrupt
the
behavior
of
WAP
Gateways,
because
it
can
inhibit
such
proxies
from
converting
WML
to
WMLC.
It is good practice to take account of user agent capabilities and formulate an appropriate experience according to those capabilities. It is good practice to provide a means for users to select among available representations, to default to the last selected representation and to provide a means of changing the selection.
Vary
HTTP
Header
Field
If
a
server
varies
its
representation
according
to
examination
of
received
HTTP
header
fields
then
[RFC
2616
HTTP]
describes
how
to
use
the
Vary
header
field
to
indicate
this.
Servers
that
are
aware
of
the
presence
of
a
transforming
proxy,
as
identified
by
a
Via
HTTP
Header
field
might
alter
their
responses
according
to
their
knowledge
of
specific
proxy
behavior.
When
doing
so
it
is
good
practice
to
make
sure
that
the
Internet
content
type
for
a
response
is
correct
for
the
actual
content
(e.g.
a
server
should
not
choose
Content-Type:
application/vnd.wap.xhtml+xml
because
it
suspects
that
proxies
will
not
transform
content
of
this
type,
if
its
content
is
not
valid
XHTML-MP).
If
a
server
has
distinct
representations
that
vary
according
to
the
target
presentation
media
type,
it
can
inhibit
transformation
of
the
response
by
including
a
Cache-Control:
no-transform
directive
(see
H.1.3
I.1.3
Server
Origination
of
Cache-Control:
no-transform
).
In
addition,
in
HTML
content
it
can
indicate
the
medium
for
which
the
representation
is
intended
by
including
a
link
element
identifying
in
its
media
attribute
the
target
presentation
media
types
of
this
representation
and
setting
the
href
attribute
to
"Same-Document
Reference"
(see
[RFC
3986]
section
4.4
)
and
in
particular
an
empty
href
attribute
is
a
"Same
Document
Reference".
In
addition
it
is
good
practice
to
include
link
elements
identifying
the
target
presentation
media
types
of
other
available
representations
in
a
similar
manner.
If
content
for
more
than
one
presentation
media
type
is
served
from
the
same
URI,
it
is
better
not
to
use
a
link
element
identifying
the
presentation
media
types
as
the
URI
will
appear
to
be
a
"same
document
reference",
indicating
to
a
client
that
this
representation
is
suitable
for
all
the
named
presentation
media
types.
Instead,
use
a
Vary
HTTP
header
field
indicating
that
the
response
varies
according
to
the
received
User-Agent
HTTP
header
field.
Note:
Some
examples
of
the
use
of
the
link
element
are
included
above
in
G
H
Example
Transformation
Interactions
.
There
are
a
number
of
well-known
examples
of
solutions
that
seem
to
their
users
as
though
they
are
using
a
browser,
but
because
the
client
software
communicates
with
using
proprietary
protocols
and
techniques,
it
is
the
combination
of
the
client
and
the
network
component
that
is
regarded
as
the
HTTP
User
Agent.
The
communication
between
the
client
and
the
network
component
is
therefore
out
of
scope
of
this
document.
Additionally, where some kind of administrative arrangement exists between a transforming proxy and an origin server for the purposes of transforming content on the origin server's behalf, this is also out of scope of this document.
In both of the above cases, it is good practice to adhere to the provisions of this document in respect of providing information about the device and the original IP address.
The BPWG believes that POWDER will represent a powerful mechanism by which a server may express transformation preferences. Future work in this area may recommend the use of POWDER to provide a mechanism for origin servers to indicate more precisely which alternatives they have and what transformation they are willing to allow on them, and in addition to provide for Content Transformation proxies to indicate which services they are able to perform.
link
HTTP
Header
Field
The
BPWG
believes
that
the
link
HTTP
header
field
which
was
removed
from
HTTP/1.1,
and
which
is
under
discussion
for
reintroduction,
would
represent
a
more
general
and
flexible
mechanism
than
use
of
the
HTML
link
element,
as
discussed
in
this
recommendation.
The process of adapting content at the origin server, or transforming it in a proxy is likely to have a dependency on a repository of device descriptions. An origin server's willingness to allow a transforming proxy to transform content may depend on its evaluation of the trustworthiness of device description data that is being used. There is scope for enhancement of the trust relationship by some means of indicating this.
There is scope for further work to define how multiple proxies may interoperate. A common case of multiple proxies is where a network provider transforming proxy and a search engine transforming proxy are both present.
Robust mechanisms are needed for indicating consent to or prohibition of transformation operations of various kinds, especially HTTPS link rewriting (see 4.2.8.3 HTTPS Link Rewriting ).
The BPWG believes that amendments to HTTP are needed to improve the interoperability of transforming proxies. For example, HTTP does not provide a way to distinguish between prohibition of any kind of transformation and the prohibition only of restructuring (and not recoding or compression).
At present HTTP does not provide a mechanism for communicating original header field values. The scheme based on X-Device prefixed fields described under 4.1.5 Alteration of HTTP Header Field Values records and clarifies an approach used to achieve this effect by some content transformation proxies. This scheme relies upon non-standard HTTP fields, which are identified by their prefix as experimental according to IETF standards (notably RFC 822 and RFC 2076), and are not included in the IANA registry of HTTP header fields. While the mechanism defined in that section, based on current practice, applies to conforming transformation proxy deployments, it is possible that in future, in collaboration with the IETF, this approach will be reconsidered. This implies that the specified X-Device prefixed fields may, at some time, become deprecated in favor of new equivalent fields, or that an entirely different approach will be taken to representing such values.
A
number
of
mechanisms
exist
in
HTTP
which
might
be
exploited
given
more
precise
definition
of
their
operation
-
for
example
the
OPTIONS
method
and
the
HTTP
300
(Multiple
Choices)
Status.
The editor acknowledges contributions of various kinds from members of the Mobile Web Best Practices Working Group and earlier from the Content Transformation Task Force of that group.
The editor acknowledges significant written contributions from: