Copyright © 2009 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
This document identifies the issues surrounding use of transforming proxies in the delivery of Web content. It does not comment on the techniques that cause these issues, it merely identifies them in order to inform the requirements of the Content Transformation Guidelines document. That document is to offer recommendations as to how components of the delivery context can cooperate to achieve, at a minimum, a functional user experience.
In this revision the document is largely historical - the Content Transformation Guidelines document, published under the title "Guidelines for Web Content Transformation Proxies" was initially inspired by the set of requirements identified in this document, but has subsequently evolved based on available technologies and feedback.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document was produced by the Content Transformation Task Force of the Mobile Web Best Practices Working Group as part of the Mobile Web Initiative. It is the Problem Statement referred to in the Charter of the W3C Mobile Web Initiative Best Practices Working Group Content Transformation Task Force, and served as a basis for the work on the Guidelines for Web Content Transformation Proxies.
Publication as a Working Group Note does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
Changes since publication of the previous Working Draft (published on 25 October 2007) are an update of the Abstract section to clarify the historical status of this work and refer to the normative Guidelines for Web Content Transformation Proxies document that was initially inspired by this document. The Working Group does not anticipate any further change on this document.
Comments on this document may be sent to the Working Group's public email list public-bpwg-comments@w3.org, a publicly archived mailing list. The Working Group rather expects comments to be made against the normative Guidelines for Web Content Transformation Proxies document.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
1 Introduction
1.1 Purpose
1.2 Scope
1.3 Structure of the Document
2 Landscape
2.1 Accessing Web Content from Mobile Devices
2.2 Content Transformation Proxies
2.2.1
Advantages of Content Transformation
2.2.2 Issues with Content Transformation
2.2.2.1 Web Presentation
2.2.2.2 Non Web Applications
2.2.2.3 Legal, Moral and Commercial Issues
2.2.2.4 Security Issues
3 Constraints
A Terminology (Non-Normative)
B References (Non-Normative)
C Acknowledgments (Non-Normative)
Mobile Web Best Practices [BestPractices] encourages content providers to produce "made for mobile" experiences of their Web sites. While the number of such Web sites continues to increase, there are still many Web sites that are unaware of mobile presentation. Those Web sites, when accessed from mobile devices, do not present a satisfactory user experience or may indeed cause failure of the user's device. In order to mitigate this unsatisfactory experience, mobile network operators and others use proxies to transform the content of these sites.
At the same time there are an increasing number of highly capable mobile devices that offer enhanced browsing experiences designed to assist users of mobile devices with their typically small screen and limited input capabilities to navigate sites designed with larger displays, pointing devices and full keyboards in mind.
The W3C MWI Best Practices Working Group recognizes that on the one hand transforming proxies can diminish the value of sites that have been designed specifically for mobile presentation. On the other hand transforming proxies can enhance the mobile experience of sites unaware of mobile presentation. Yet again, transforming proxies can diminish the value of such sites when presented on devices that are capable of simulating a desktop experience while mobile.
This document identifies specific issues that can result from content transformation and identifies requirements that must be fulfilled by the Content Transformation Guidelines document. That document identifies techniques that are designed to help components in the Delivery Context cooperate in the presentation of Web sites in order that users experience, at a minimum, a functional user experience and that authors' intentions are fulfilled to whatever degree possible.
While the Working Group's charter refers specifically to mobile, the Working Group recognizes that the issues here relate to any non-desktop presentation of content, or indeed presentation of any content that has alternative representations. Hence, in the following, requirements are presented in a more general form.
This document does not offer comment on the rights and wrongs of any particular issues it mentions. It does not condone nor deprecate any particular technique. They are mentioned in order to clarify the requirements of the Guidelines document that consequently addresses the issues mentioned.
In 2 Landscape the document identifies specific benefits and issues relating to content transformation. Where requirements can be inferred from the discussion they are identified in the text, thus:
This is a requirement.
Various technologies and techniques that must be taken into account when satisfying the requirements are identified in 3 Constraints.
While an increasing number of Web sites [Definition: mobile aware] are being designed specifically to tailor their presentation to be suitable for users in a mobile context (see [BestPractices]), many Web sites [Definition: mobile unaware] have been (and continue to be) designed with only the features of desktop browsers in mind. From a mobile perspective, typical issues with the resulting Web pages are:
page layouts assume a screen larger than the device supports;
pages require more memory than the device supports;
the design of interaction with the page assumes that a full qwerty keyboard and mouse are available;
pages use content types that the device does not support.
As a result accessing such a Web page from a mobile device often results in a poor or unusable experience (The differences between desktop and mobile presentation are discussed in more detail in [BestPractices] - 2 Requirements).
Some Web sites [Definition: mobile blocking], aware that access is not from the expected desktop context, send either an HTTP error status code or an HTTP success code accompanied by text indicating that they cannot present an acceptable experience for the user - thus preventing access from mobile users.
Mobile operators, search engines and others recognize that while the number of mobile friendly Web sites is growing, there will remain a considerable number of mobile unaware and mobile blocking sites. One approach to providing a more satisfactory mobile user experience of mobile unaware and blocking sites is to insert an intermediary in the communications path between the user agent and the origin server. These intermediaries, known as [Definition: content transformation proxies] (see A Terminology) adjust HTML pages designed for desktop presentation to provide an enhanced user experience when accessed from a mobile device.
In order to avoid blocking behavior and in order to achieve a consistent presentation from Web sites that vary their experience according to the browser type, proxies typically work by masquerading as a specific desktop browser, fetching content and then modifying it before returning it to mobile devices. These transformations include:
character encoding corrections,
image reformatting and resizing,
layout modifications and page segmentation,
multi step JavaScript transaction emulation.
Content transformation proxies can also add content to, or remove it from a page.
1. It must be possible to make a distinction between different types of transformation - e.g. correction of malformed markup as opposed to pagination and addition of headers and footers.
The principal advantage of Content Transformation is that it provides the ability for mobile users to enjoy a Web experience even though a large number of Web sites that are mobile unaware or mobile blocking.
2. Transforming proxies must be able to overcome the behavior of "blocking" sites.
Content transformation also offers opportunities to site designers, allowing their sites to be served to a wide range or devices while freeing them from having to detect and keep up to date with the capabilities of new devices.
3. Transforming proxies must be able to identify their presence and advertise their capabilities.
4. Origin servers must be able to selectively enable or disable features of transforming proxies.
By presenting requests as though they came from a different device, Content Transformation prevents mobile aware sites from detecting the actual mobile devices behind the request. As a result, mobile aware sites whose purpose is to provide mobile compatible pages or mobile compatible content like ring-tones or Java applications are unable to operate correctly. Such sites require information about the actual device.
5. Origin servers and proxies must be able to identify the actual identity of components of the delivery context, including (other) proxies and browsers.
When mobile aware sites send content to devices, they often choose to deliver very specific variations on their content tailored exactly for the device. Any attempt by a content transformation proxy to change the content may very well destroy or in any case decrease the value of the content. For example, a site which delivers ring tones or wallpaper which it tailors exactly to devices' characteristics may be rendered useless if the audio format is changed, or less valuable if the color depth of an image is reduced to something below that which the device is capable of rendering.
6. Origin servers must be able to prohibit any kind of transformation of its content.
7. Origin servers must be able to advertise author preferences.
Some Web sites cater for both desktop browsers and mobile devices, especially when the site recognizes that the device has a browser that is capable of providing a 'full screen' presentation mode. In these cases the user of the mobile device may be prevented from exercising their choice of presentation and may be unable to access either the site's mobile or desktop presentation in its original form.
8. Browsers must be able to identify and advertise user interface modalities.
9. Components must be able to solicit and advertise user choice.
10. Origin servers must be able to identify and advertise different representations they may have available.
In the mobile Internet, just like the Internet as a whole, communication between a client and a server is not confined to Web applications. In the mobile world, however, use of HTTP as a bearer for other applications is, if anything, even more prevalent. Services are often designed this way as a result of there being no access to TCP sockets on clients, or fears of non-transparency of gateways. Examples include:
J2ME client applications
native applications (Symbian for Symbian devices, Microsoft for Microsoft devices, etc.)
Adobe Flash applications
SVGT documents
Ajax applications embedded in HTML
There are many other examples, but these are sufficient to illustrate a point: not all clients using HTTP as a communication protocol with an origin server are interested in "content transformation". In fact, transformation may break the semantics of the client-server communication and render the application completely useless.
See Requirement 6 above.
A further issue concerns the limits of acceptable content transformation and to what degree content providers can reasonably expect to remain in control of how their content is presented.
Some content providers may feel that, for whatever reason, their content should not be fragmented, or their images removed or transformed. In a medical application, for example, it may be essential that an image is presented in its original format.
See Requirement 6 above.
An example of the commercial issues is removing advertising banners from the origin server and perhaps even replacing them with new advertisements adapted for the specific user agent. This is obviously controversial because it hi-jacks an advertising revenue stream from the origin server.
11. It must be possible for origin servers selectively to indicate that content of various types must not be removed, replaced or inserted by transforming proxies.
The most commonly used security mechanisms require a direct connection between the origin server and the ultimate client. Intermediaries of any kind, including content transformation proxies, may break this security model and cause exposure to theft of sensitive information.
12. It must be possible to establish a secure end to end connection.
The BPWG, and hence the Content Transformation Task Force, is not chartered to create "new technology", rather it is chartered to advise on the use of existing technology. Hence the techniques should operate within existing technology. That said, the Task Force is free to suggest new mechanisms and enhancements to existing mechanisms to other W3C groups that are chartered with the development of such technology.
Examples of technologies and techniques that will be further investigated include, but are not limited to:
The HTTP protocol [HTTP], which provides several mechanisms that are likely to be useful. In particular to promote awareness of the "Cache-Control: no-transform" HTTP directive, or other mechanism for allowing content authors to prohibit transformation of a resource by intermediaries.
The POWDER protocol, which provide mechanisms for describing resources using RDF and OWL. More generally, ascertaining that a representation of a resource is suitable for the user's delivery context either because the resource has been so described or from the nature of the representation of the resource.
mobileOK (see [mobileOKBasic10Tests]) which is a claim that can be represented using (POWDER) Web Description Resources, that a resource achieves certain minimum interoperability requirements for mobile delivery.
The work of the W3C MWI DDWG relating to definition of device capabilities.
In [HTTP], 1.3 Terminology various types of intermediary are described:
A proxy is a forwarding agent, receiving requests for a URI in its absolute form, rewriting all or part of the message, and forwarding the reformatted request toward the server identified by the URI.
A gateway is a receiving agent, acting as a layer above some other server(s) and, if necessary, translating the requests to the underlying server's protocol.
A tunnel acts as a relay point between two connections without changing the messages; tunnels are used when the communication needs to pass through an intermediary (such as a firewall) even when the intermediary cannot understand the contents of the messages.
The term "content transformation proxy" has become popular for a special type of gateway: one that transforms content as described above. The term "content transformation gateway" would perhaps have been a better choice or maybe even "content transformation proxy gateway", since the relevant server technology actually is both a proxy and a gateway.
The editors acknowledge contributions of various kinds from members of the MWI BPWG Content Transformation Task Force.
The editors acknowledge significant written contributions from: