Prior discussion: https://etherpad.mozilla.org/responsive-assets
W3 Responsive Images Community Group: http://www.w3.org/community/respimg
Unofficial <picture> draft by Mat Marquis: https://gist.github.com/2159117
The web needs a declarative way to define resolution-independent bitmap images.
- Declarative, through markup, because it provides hooks for implementers and browsers to interact with the sizes.
- Resolution-independent, because we want to be able to create a single document and deliver it to the ever-growing plethora of web-enabled devices.
- Bitmap, because vector (SVG) is wonderful for limited types of artwork, but bitmap is an efficient and commonly used delivery format for a wide variety of artwork.
Existing Solutions, Limitations
TODO: A summary of currently implemented solutions and their limitations should go here. Feel free to add it you good looking person, you.
An implementer writes in detail about the limitations of this technique here: http://www.alistapart.com/articles/responsive-images-how-they-almost-worked-and-what-we-need/
Write <img> with JS, <noscript> fallback
To work around the prefetching issue, an alternative method has been proposed using the <noscript> tag as a fallback:
<img data-src="imagefile.jpg" data-width="320" data-height="240" class="fs-img"> <noscript> <img src="imagefile.jpg"> </noscript>
- Questionable use of the <img> tag -- if it has no @src, is it valid content?
- No declarative hooks for image sizes.
In this approach, a script sets a cookie on the client side. When images are requested, the cookie is sent along with the image request. The server-side is expected to to look for this cookie and change the image resource sent based on the cookie value.
- Cookies are not guaranteed to be set before images are requested.
- URLs for images can no longer be considered URIs, since the image served changed depending on cookie value. This can cause problems with late-caching tools like Varnish.
- Requires server-side configuration.
CSS Image Replacement
Use CSS media queries, backgrounds and image replacement to serve different graphics.
- Does not provide delarative hooks for other agents to access various image sizes. Content images are content, but this method moves them into the presentation layer.
- Difficult to do with existing CMS publishing systems.
"Upgrading" images via diff
Philip Ingrey has suggested a clever workaround to the issue of browser prefetching: allow the lowest resolution of the image to load normally. For browsing agents that require a higher resolution, send a .diff file describing how to "upgrade" the image to a higher resolution. A client-side <canvas>-based script pieces together the final image, setting it as a data URL. Though this still incurs 2 HTTP requests, the second request is smaller than loading the full high-res image. Source for the prototype solution: https://github.com/pci/deltaimg.
- 2 HTTP Requests
- If several image sizes are required, diffing becomes more complex
- Special script or toolchain required to generate diffs. Developers are not used to diffing images.
- Browser must support data URLs.
- Does not provide declarative hooks for other agents.
Prior Art and Discussion
TODO: Please organize me and add summaries to links
WebKit has just integrated a patch from Apple, landing experimental support for variants of a CSS image based on the device scale factor: http://trac.webkit.org/changeset/111637. It doesn't address the markup-based use case, but should probably inform it.
- Includes W3 mailing list discussions for Responsive Images: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-February/
TODO: A summary of the picture element. Bullet points: how does it solve these problems?
- Could patents be a problem here?
- Has such an image format been demonstrated?
- This would require the adoption of a new image type, meaning current authoring tools (like Photoshop) would have to change.
- HTTP headers are outside the skillset of most front-end developers.
Solutions that seem feasible, but are no-gos (pull in content from http://www.w3.org/community/respimg/common-questions-and-concerns/)
Device headers: We’ve seen standardized solutions that require a server-side component in the past with appcache, where setting MIME types was once required. This has since been removed from the spec due to overwhelming feedback from developers : the server-side requirement served as a major barrier to adoption for several reasons, including developers lacking direct access to servers, difficulty of implementation, and dependency on a server-side scripting language solely to take advantage of a feature inherent to HTML5. Media queries exist to keep exactly this sort of logic on the front-end, and a markup-based approach follows the lead of other media elements that already solve their respective alternate-source delivery problems well: the <video> and <audio> tags.
Pure-CSS solution: It may one day be possible to use CSS to swap an image’s sources based on values set in data attributes, using existing proposals in draft specs . However, no proposed CSS-based solution accounts for avoiding the downloading of multiple assets at larger screen widths. Additionally, a CSS-only solution would not provide declarative hooks for other agents to access the various image sizes.
Modifying the behavior of <img>: we spent a great deal of time looking for ways to work with/around the img tag (and its alias in many browsers, “image”)— unfortunately, after speaking to several browser representatives, it was established early on that it was nigh impossible to modify the behavior of img to “look ahead” for multiple sources. Another major caveat is that any modification to the img tag stands to break things in terms of backwards-compatibility. Some of that discussion is documented in the conversation history on https://etherpad.mozilla.org/responsive-assets.
<img src="mobile-size.png"> <source src media /> <source src media /> </img>
- In DOM, Chrome/Firefox lose child relationships of source and img relative to image.
- Confirmed by Paul Irish: at least in Chrome, “looking ahead” for sources on an <img>/<image> tag isn’t viable.
- Hixie said this is a lost cause. This cannot be backwards compatible because no browser can realistically update <img> parsing.
- Responsive features in legacy browser dependent upon a polyfill that would replace the initial img's src with an appropriate URL from the immediately following <source> tags depending on their media query.
Fit with existing spec
TODO: What is the impact and usefulness of @media on <source> elements for other elements in the spec?
- It could be used with <video> to deliver videos of different sizes
- Future media queries could allow <audio> to deliver different bitrates for different bandwidths. Research @media discussion for bandwidth queries.
- Scott Jehl's Picturefill: Scott Jehl created a polyfill for the picture-element (non-existent but proposed) that actually makes responsive images work in most browsers including IE9. This polyfill uses a current proposal of the picture element but surely isn't the final solution.
- Gbrander: I heard about a <video> tag hack that uses poster images. [anselmh] Found this comment (http://www.alistapart.com/comments/responsive-images-how-they-almost-worked-and-what-we-need/P60/#69) on ALA with this solution (not sure if it works):
<video> <source src=“high-res.jpg” media=“min-width:800px” type=“image/jpeg”> <source src=“low-res.jpg” type=“image/jpeg”> poster.jpg </video>
Solution Evaluation Matrix
This matrix is attempting to make sense of all the various recommendations found in posts and comments, and to minimize the number of descriptions of why a solution will or will not work. Feel free to modify the evaluation data to ensure it best describes each solution's advantages and disadvantages, and to add solutions not yet listed.
Please also add in any other requirements which may be missing, even if it is a "nice-to-have". Additionally, please feel free to add any new solutions just dreamed up, but ensure it has link to its post so we can read more about it. Even though the picture element is the current proposal, its presence should not stop discussions with the common goal of developing an improved solution.
|Evaluation||Picture Element||image-set CSS/uri templates||Spacer background-image||Request headers|
|Bandwidth efficiency||Yes, requests only what it needs||Yes, requests only what it needs||Yes, requests only what it needs||Yes, requests only what it needs|
|Single HTTP request||Yes||Yes||Yes||Yes|
|Server-side config not required||Client based||Client based||Client based||Server-config required|
|Structure Simplicity (HTML)||Difficult for beginners||(structure not decided, only focuses on CSS)||Existing elements||No changes|
|Presentation Simplicity (CSS)||Yes||Difficult for beginners, but consistent image-set for backgrounds||Heavy use of media queries||No changes|
|Easy of maintenance||Responsive scenarios within structure||Control image variants with CSS||Control image variants with CSS||Server-side config|
|Separation of concerns||Presentation scenarios within structure||Control image variants with CSS||Control image variants with CSS||Depends on implementation|
|Declarative? (e.g provides semantic hooks for sizes)||Yes||This is only CSS, so it allows the markup to be declarative too. Additionally, it provides the browser with request formats and presentation scenarios which can be reused sitewide.||No||No|
|Centralized image variants||Located within structure||Variants formats within image-set() urls||Variants within media queries||Depends on implementation|
|Request static images||Yes||Yes||Yes||Yes|
|Request dynamically resized images||Manually code dimensions within urls||Request exact dimension needed||Manually code dimensions within urls||Yes|
|High-resolution when the device is capable||Yes, from media queries||Yes, from image-set()||Yes, from media queries||Yes, but may be maintained by user agent|
|Adjusts to device bandwidth||Yes (if bandwidth added to media queries)||Yes (if bandwidth added to image-set)||Yes (if bandwidth added to media queries)||No|
|Fallback for browsers without support||Yes, img within noscript||(no structure to evaluate)||Yes||Yes|
|Polyfill||Yes||Yes||Not needed||Not needed|
|Minimal structure updates for browser support||Create a new picture element||(no structure to evaluate)||Not needed||Heavy restructuring|
|Minimal presentation updates for browser support||No new CSS required||image-set being evaluated by webkit for background images||Not needed||No new CSS required|
|Image within content||Yes||Yes||No||Yes|
|No new file formats||Works with existing files||Works with existing files||Works with existing files||Works with existing files|