Main Page

Prior discussion: https://etherpad.mozilla.org/responsive-assets

W3 Responsive Images Community Group: http://www.w3.org/community/respimg

Unofficial <picture> draft by Mat Marquis: https://gist.github.com/2159117

Problem Statement

The web needs a declarative way to define resolution-independent bitmap images.

• Declarative, through markup, because it provides hooks for implementers and browsers to interact with the sizes.
• Resolution-independent, because we want to be able to create a single document and deliver it to the ever-growing plethora of web-enabled devices.
• Bitmap, because vector (SVG) is wonderful for limited types of artwork, but bitmap is an efficient and commonly used delivery format for a wide variety of artwork.

Existing Solutions, Limitations

TODO: A summary of currently implemented solutions and their limitations should go here. Feel free to add it you good looking person, you.

Editing @src via JavaScript

An implementer writes in detail about the limitations of this technique here: http://www.alistapart.com/articles/responsive-images-how-they-almost-worked-and-what-we-need/

Limitations:

• 2 HTTP requests. JavaScript can not change the @src attribute of an image before the browser prefetches the original @src. This results in an additional download. For example, Apple.com's own retina image script exhibits this behavior: it first loads the standard resource and then loads a high-resolution image additionally when high-res-device is calling for it.
• A JavaScript @src-based solution would not provide declarative hooks for other agents to access the various image sizes.

Write <img> with JS, <noscript> fallback

To work around the prefetching issue, an alternative method has been proposed using the <noscript> tag as a fallback:

<img data-src="imagefile.jpg" data-width="320" data-height="240" class="fs-img">
<noscript>
    <img src="imagefile.jpg">
</noscript>

The @src is written dynamically with JavaScript, and a fallback provided for UAs without scripting capability.

Limitations:

• Questionable use of the <img> tag -- if it has no @src, is it valid content?
• No declarative hooks for image sizes.

Examples in the wild: Foresight.js. Adam Bradley released this nice javascript for using / providing responsive images today. This script looks ahead what resolution is needed, then looks what's the network speed the user has and then it delivers the right image to the user. This is a nice technique that works and is future-proof because it uses the img-tag with the HTML5 data-attributes and a noscript-fallback. No new elements etc introduced here. And it works in IE8 and all other modern browsers. The script also has a huge documentation which explains the code, the thoughts.

Hybrid: server-side + script and cookie

In this approach, a script sets a cookie on the client side. When images are requested, the cookie is sent along with the image request. The server-side is expected to to look for this cookie and change the image resource sent based on the cookie value.

Example: http://adaptive-images.com/details.htm#cookie-issue

Limitations:

• Cookies are not guaranteed to be set before images are requested.
• URLs for images can no longer be considered URIs, since the image served changed depending on cookie value. This can cause problems with late-caching tools like Varnish.
• Requires server-side configuration.

CSS Image Replacement

Use CSS media queries, backgrounds and image replacement to serve different graphics.

Limitations:

• Does not provide delarative hooks for other agents to access various image sizes. Content images are content, but this method moves them into the presentation layer.
• Difficult to do with existing CMS publishing systems.

"Upgrading" images via diff

Philip Ingrey has suggested a clever workaround to the issue of browser prefetching: allow the lowest resolution of the image to load normally. For browsing agents that require a higher resolution, send a .diff file describing how to "upgrade" the image to a higher resolution. A client-side <canvas>-based script pieces together the final image, setting it as a data URL. Though this still incurs 2 HTTP requests, the second request is smaller than loading the full high-res image. Source for the prototype solution: https://github.com/pci/deltaimg.

Limitations:

• 2 HTTP Requests
• If several image sizes are required, diffing becomes more complex
• Special script or toolchain required to generate diffs. Developers are not used to diffing images.
• Browser must support data URLs.
• Does not provide declarative hooks for other agents.

Prior Art and Discussion

TODO: Please organize me and add summaries to links

WebKit has just integrated a patch from Apple, landing experimental support for variants of a CSS image based on the device scale factor: http://trac.webkit.org/changeset/111637. It doesn't address the markup-based use case, but should probably inform it.

• Includes W3 mailing list discussions for Responsive Images: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-February/
• http://www.w3.org/Search/Mail/Public/search?type-index=public-html&index-type=t&keywords=picture+element
• http://lists.w3.org/Archives/Public/public-html/2007Jun/1057.html
• http://lists.w3.org/Archives/Public/public-html/2011May/0386.html

Proposed Solutions

TODO: A summary of the picture element. Bullet points: how does it solve these problems?

Other proposals

A new image format + HTTP Range headers has been suggested by Le Roux. Questions:

• Could patents be a problem here?
• Has such an image format been demonstrated?
• This would require the adoption of a new image type, meaning current authoring tools (like Photoshop) would have to change.
• HTTP headers are outside the skillset of most front-end developers.

No-gos

Solutions that seem feasible, but are no-gos (pull in content from http://www.w3.org/community/respimg/common-questions-and-concerns/)

Device headers: We’ve seen standardized solutions that require a server-side component in the past with appcache, where setting MIME types was once required. This has since been removed from the spec due to overwhelming feedback from developers [1]: the server-side requirement served as a major barrier to adoption for several reasons, including developers lacking direct access to servers, difficulty of implementation, and dependency on a server-side scripting language solely to take advantage of a feature inherent to HTML5. Media queries exist to keep exactly this sort of logic on the front-end, and a markup-based approach follows the lead of other media elements that already solve their respective alternate-source delivery problems well: the <video> and <audio> tags.

Pure-CSS solution: It may one day be possible to use CSS to swap an image’s sources based on values set in data attributes, using existing proposals in draft specs [2][3][4]. However, no proposed CSS-based solution accounts for avoiding the downloading of multiple assets at larger screen widths. Additionally, a CSS-only solution would not provide declarative hooks for other agents to access the various image sizes.

Modifying the behavior of <img>: we spent a great deal of time looking for ways to work with/around the img tag (and its alias in many browsers, “image”)— unfortunately, after speaking to several browser representatives, it was established early on that it was nigh impossible to modify the behavior of img to “look ahead” for multiple sources. Another major caveat is that any modification to the img tag stands to break things in terms of backwards-compatibility. Some of that discussion is documented in the conversation history on https://etherpad.mozilla.org/responsive-assets.

 <img src="mobile-size.png">
   <source src media />
   <source src media />
 </img>

• In DOM, Chrome/Firefox lose child relationships of source and img relative to image.
• Confirmed by Paul Irish: at least in Chrome, “looking ahead” for sources on an <img>/<image> tag isn’t viable.
• Hixie said this is a lost cause. This cannot be backwards compatible because no browser can realistically update <img> parsing.
• Responsive features in legacy browser dependent upon a polyfill that would replace the initial img's src with an appropriate URL from the immediately following <source> tags depending on their media query.

Fit with existing spec

TODO: What is the impact and usefulness of @media on <source> elements for other elements in the spec?

• It could be used with <video> to deliver videos of different sizes
• Future media queries could allow <audio> to deliver different bitrates for different bandwidths. Research @media discussion for bandwidth queries.

Working Demos

Demos implemented with JavaScript, as a preview of how things could work.

• Scott Jehl's Picturefill: Scott Jehl created a polyfill for the picture-element (non-existent but proposed) that actually makes responsive images work in most browsers including IE9. This polyfill uses a current proposal of the picture element but surely isn't the final solution.
  • Blog post: http://www.w3.org/community/respimg/2012/03/15/polyfilling-picture-without-the-overhead/.
  • Source code: https://github.com/scottjehl/picturefill
• Gbrander: I heard about a <video> tag hack that uses poster images. [anselmh] Found this comment (http://www.alistapart.com/comments/responsive-images-how-they-almost-worked-and-what-we-need/P60/#69) on ALA with this solution (not sure if it works):

  <video>
      <source src=“high-res.jpg” media=“min-width:800px” type=“image/jpeg”>
      <source src=“low-res.jpg” type=“image/jpeg”>
      poster.jpg
  </video>

Solution Evaluation Matrix

This matrix is attempting to make sense of all the various recommendations found in posts and comments, and to minimize the number of descriptions of why a solution will or will not work. Feel free to modify the evaluation data to ensure it best describes each solution's advantages and disadvantages, and to add solutions not yet listed.

Please also add in any other requirements which may be missing, even if it is a "nice-to-have". Additionally, please feel free to add any new solutions just dreamed up, but ensure it has link to its post so we can read more about it. Even though the picture element is the current proposal, its presence should not stop discussions with the common goal of developing an improved solution.