Main Page

From Responsive Images Community Group
Revision as of 19:37, 4 May 2012 by Pingrey (Talk | contribs)

Jump to: navigation, search

Prior discussion: https://etherpad.mozilla.org/responsive-assets

W3 Responsive Images Community Group: http://www.w3.org/community/respimg

Unofficial <picture> draft by Mat Marquis: https://gist.github.com/2159117

Problem Statement

The web needs a declarative way to define resolution-independent bitmap images.

  • Declarative, through markup, because it provides hooks for implementers and browsers to interact with the sizes.
  • Resolution-independent, because we want to be able to create a single document and deliver it to the ever-growing plethora of web-enabled devices.
  • Bitmap, because vector (SVG) is wonderful for limited types of artwork, but bitmap is an efficient and commonly used delivery format for a wide variety of artwork.

Existing Solutions, Limitations

TODO: A summary of currently implemented solutions and their limitations should go here. Feel free to add it you good looking person, you.

Editing @src via JavaScript

An implementer writes in detail about the limitations of this technique here: http://www.alistapart.com/articles/responsive-images-how-they-almost-worked-and-what-we-need/

Limitations:

  • 2 HTTP requests. JavaScript can not change the @src attribute of an image before the browser prefetches the original @src. This results in an additional download. For example, Apple.com's own retina image script exhibits this behavior: it first loads the standard resource and then loads a high-resolution image additionally when high-res-device is calling for it.
  • A JavaScript @src-based solution would not provide declarative hooks for other agents to access the various image sizes.

Write <img> with JS, <noscript> fallback

To work around the prefetching issue, an alternative method has been proposed using the <noscript> tag as a fallback:

<img data-src="imagefile.jpg" data-width="320" data-height="240" class="fs-img">
<noscript>
    <img src="imagefile.jpg">
</noscript>

The @src is written dynamically with JavaScript, and a fallback provided for UAs without scripting capability.

Limitations:

  • Questionable use of the <img> tag -- if it has no @src, is it valid content?
  • No declarative hooks for image sizes.

Examples in the wild: Foresight.js. Adam Bradley released this nice javascript for using / providing responsive images today. This script looks ahead what resolution is needed, then looks what's the network speed the user has and then it delivers the right image to the user. This is a nice technique that works and is future-proof because it uses the img-tag with the HTML5 data-attributes and a noscript-fallback. No new elements etc introduced here. And it works in IE8 and all other modern browsers. The script also has a huge documentation which explains the code, the thoughts.


Hybrid: server-side + script and cookie

In this approach, a script sets a cookie on the client side. When images are requested, the cookie is sent along with the image request. The server-side is expected to to look for this cookie and change the image resource sent based on the cookie value.

Example: http://adaptive-images.com/details.htm#cookie-issue

Limitations:

  • Cookies are not guaranteed to be set before images are requested.
  • URLs for images can no longer be considered URIs, since the image served changed depending on cookie value. This can cause problems with late-caching tools like Varnish.
  • Requires server-side configuration.

CSS Image Replacement

Use CSS media queries, backgrounds and image replacement to serve different graphics.

Limitations:

  • Does not provide delarative hooks for other agents to access various image sizes. Content images are content, but this method moves them into the presentation layer.
  • Difficult to do with existing CMS publishing systems.

"Upgrading" images via diff

Philip Ingrey has suggested a clever workaround to the issue of browser prefetching: allow the lowest resolution of the image to load normally. For browsing agents that require a higher resolution, send a .diff file describing how to "upgrade" the image to a higher resolution. A client-side <canvas>-based script pieces together the final image, setting it as a data URL. Though this still incurs 2 HTTP requests, the second request is smaller than loading the full high-res image. Source for the prototype solution: https://github.com/pci/deltaimg.

Limitations:

  • 2 HTTP Requests
  • If several image sizes are required, diffing becomes more complex
  • Special script or toolchain required to generate diffs. Developers are not used to diffing images.
  • Browser must support data URLs.
  • Does not provide declarative hooks for other agents.

Prior Art and Discussion

TODO: Please organize me and add summaries to links

WebKit has just integrated a patch from Apple, landing experimental support for variants of a CSS image based on the device scale factor: http://trac.webkit.org/changeset/111637. It doesn't address the markup-based use case, but should probably inform it.

Proposed Solutions

TODO: A summary of the picture element. Bullet points: how does it solve these problems?

Other proposals

A new image format + HTTP Range headers has been suggested by Le Roux. Questions:

  • Could patents be a problem here?
  • Has such an image format been demonstrated?
  • This would require the adoption of a new image type, meaning current authoring tools (like Photoshop) would have to change.
  • HTTP headers are outside the skillset of most front-end developers.

No-gos

Solutions that seem feasible, but are no-gos (pull in content from http://www.w3.org/community/respimg/common-questions-and-concerns/)

Device headers: We’ve seen standardized solutions that require a server-side component in the past with appcache, where setting MIME types was once required. This has since been removed from the spec due to overwhelming feedback from developers [1]: the server-side requirement served as a major barrier to adoption for several reasons, including developers lacking direct access to servers, difficulty of implementation, and dependency on a server-side scripting language solely to take advantage of a feature inherent to HTML5. Media queries exist to keep exactly this sort of logic on the front-end, and a markup-based approach follows the lead of other media elements that already solve their respective alternate-source delivery problems well: the <video> and <audio> tags.

Pure-CSS solution: It may one day be possible to use CSS to swap an image’s sources based on values set in data attributes, using existing proposals in draft specs [2][3][4]. However, no proposed CSS-based solution accounts for avoiding the downloading of multiple assets at larger screen widths. Additionally, a CSS-only solution would not provide declarative hooks for other agents to access the various image sizes.

Modifying the behavior of <img>: we spent a great deal of time looking for ways to work with/around the img tag (and its alias in many browsers, “image”)— unfortunately, after speaking to several browser representatives, it was established early on that it was nigh impossible to modify the behavior of img to “look ahead” for multiple sources. Another major caveat is that any modification to the img tag stands to break things in terms of backwards-compatibility. Some of that discussion is documented in the conversation history on https://etherpad.mozilla.org/responsive-assets.

 <img src="mobile-size.png">

Invalid language.

You need to specify a language like this: <source lang="html4strict">...</source>

Supported languages for syntax highlighting:

4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, algol68, apache, applescript, apt_sources, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, c_loadrunner, c_mac, caddcl, cadlisp, cfdg, cfm, chaiscript, cil, clojure, cmake, cobol, coffeescript, cpp, cpp-qt, csharp, css, cuesheet, d, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, j, java, java5, javascript, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nsis, oberon2, objc, objeck, ocaml, ocaml-brief, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, php-brief, pic16, pike, pixelbender, pli, plsql, postgresql, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, rails, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, sas, scala, scheme, scilab, sdlbasic, smalltalk, smarty, spark, sparql, sql, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xorg_conf, xpp, yaml, z80, zxbasic


    <source src media>
  </img>

* In DOM, Chrome/Firefox lose child relationships of source and img relative to image.
* Confirmed by Paul Irish: at least in Chrome, “looking ahead” for sources on an <img>/<image> tag isn’t viable.
* Hixie said this is a lost cause. This cannot be backwards compatible because no browser can realistically update <img> parsing.
* Responsive features in legacy browser dependent upon a polyfill that would replace the initial img's src with an appropriate URL from the immediately following <source> tags depending on their media query.

== Fit with existing spec ==

''TODO: What is the impact and usefulness of @media on <source> elements for other elements in the spec?''

* It could be used with <video> to deliver videos of different sizes
* Future media queries could allow <audio> to deliver different bitrates for different bandwidths. ''Research @media discussion for bandwidth queries''.

== Working Demos ==

Demos implemented with JavaScript, as a preview of how things could work.

* '''Scott Jehl's Picturefill''': Scott Jehl created a polyfill for the picture-element (non-existent but proposed) that actually makes responsive images work in most browsers including IE9. This polyfill uses a current proposal of the picture element but surely isn't the final solution.
** Blog post: http://www.w3.org/community/respimg/2012/03/15/polyfilling-picture-without-the-overhead/.
** Source code: https://github.com/scottjehl/picturefill
* Gbrander: I heard about a <video> tag hack that uses poster images. [anselmh] Found this comment (http://www.alistapart.com/comments/responsive-images-how-they-almost-worked-and-what-we-need/P60/#69) on ALA with this solution (not sure if it works): 

   <video>
       <source src=“high-res.jpg” media=“min-width:800px” type=“image/jpeg”>
       <source src=“low-res.jpg” type=“image/jpeg”>
       poster.jpg
   </video>


== Solution Evaluation Matrix ==

This matrix is attempting to make sense of all the various recommendations found in posts and comments, and to minimize the number of descriptions of why a solution will or will not work. Feel free to modify the evaluation data to ensure it best describes each solution's advantages and disadvantages, and to add solutions not yet listed.

Please also add in any other requirements which may be missing, even if it is a "nice-to-have". Additionally, please feel free to add any new solutions just dreamed up, but ensure it has link to its post so we can read more about it. Even though the picture element is the current proposal, its presence should not stop discussions with the common goal of developing an improved solution.


{| class="wikitable"  style="border-collapse: collapse; border-width: 1px; border-style: solid; border-color: #000"
! style="border:1px solid black"                       | Evaluation
! style="border:1px solid black"                       | [http://www.w3.org/community/respimg/2012/03/15/polyfilling-picture-without-the-overhead/ Picture Element]
! style="border:1px solid black"                       | [http://www.w3.org/community/respimg/2012/04/08/using-css-to-control-image-variants/ image-set CSS/uri templates]
! style="border:1px solid black"                       | [http://www.w3.org/community/respimg/2012/04/06/responsive-content-images-using-a-spacer-png-and-background-image/ Spacer background-image]
! style="border:1px solid black"                       | [http://www.w3.org/community/respimg/2012/02/20/whatwg-discussion/ Request headers]
|-
| style="border:1px solid black"                       | Bandwidth efficiency
| style="border:1px solid black; background:#90FF90"   | Yes, requests only what it needs <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Yes, requests only what it needs <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Yes, requests only what it needs <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Yes, requests only what it needs <!-- Request headers -->
|-
| style="border:1px solid black"                       | Single HTTP request
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Request headers -->
|-
| style="border:1px solid black"                       | Server-side config not required
| style="border:1px solid black; background:#90FF90"   | Client based <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Client based <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Client based <!-- Spacer background-image -->
| style="border:1px solid black; background:#FF9090"   | Server-config required <!-- Request headers -->
|-
| style="border:1px solid black"                       | Structure Simplicity (HTML)
| style="border:1px solid black; background:#FFFF99"   | Difficult for beginners <!-- Picture Element -->
| style="border:1px solid black; background:#DDDDDD"   | (structure not decided, only focuses on CSS) <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Existing elements <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | No changes <!-- Request headers -->
|-
| style="border:1px solid black"                       | Presentation Simplicity (CSS)
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Picture Element -->
| style="border:1px solid black; background:#FFFF99"   | Difficult for beginners, but consistent image-set for backgrounds <!-- image-set() CSS -->
| style="border:1px solid black; background:#FFFF99"   | Heavy use of media queries <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | No changes <!-- Request headers -->
|-
| style="border:1px solid black"                       | Easy of maintenance
| style="border:1px solid black; background:#FF9090"   | Responsive scenarios within structure <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Control image variants with CSS <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Control image variants with CSS <!-- Spacer background-image -->
| style="border:1px solid black; background:#FF9090"   | Server-side config <!-- Request headers -->
|-
| style="border:1px solid black"                       | Separation of concerns
| style="border:1px solid black; background:#FF9090"   | Presentation scenarios within structure <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Control image variants with CSS <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Control image variants with CSS <!-- Spacer background-image -->
| style="border:1px solid black; background:#FFFF99"   | Depends on implementation <!-- Request headers -->
|-
| style="border:1px solid black;"                      | Declarative? (e.g provides semantic hooks for sizes)
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | This is only CSS, so it allows the markup to be declarative too. Additionally,  it provides the browser with request formats and presentation scenarios which can be reused sitewide. <!-- image-set() CSS -->
| style="border:1px solid black; background:#FF9090"   | No <!-- Spacer background-image -->
| style="border:1px solid black; background:#FF9090"   | No <!-- Request headers -->
|-
| style="border:1px solid black"                       | Centralized image variants
| style="border:1px solid black; background:#FFFF99"   | Located within structure <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Variants formats within image-set() urls <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Variants within media queries <!-- Spacer background-image -->
| style="border:1px solid black; background:#FFFF99"   | Depends on implementation <!-- Request headers -->
|-
| style="border:1px solid black"                       | Request static images
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Request headers -->
|-
| style="border:1px solid black"                       | Request dynamically resized images
| style="border:1px solid black; background:#FFFF99"   | Manually code dimensions within urls <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Request exact dimension needed <!-- image-set() CSS -->
| style="border:1px solid black; background:#FFFF99"   | Manually code dimensions within urls <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Request headers -->
|-
| style="border:1px solid black"                       | High-resolution when the device is capable
| style="border:1px solid black; background:#90FF90"   | Yes, from media queries <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Yes, from image-set() <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Yes, from media queries <!-- Spacer background-image -->
| style="border:1px solid black; background:#FFFF99"   | Yes, but may be maintained by user agent <!-- Request headers -->
|-
| style="border:1px solid black"                       | Adjusts to device bandwidth
| style="border:1px solid black; background:#FFFF99"   | Yes (if bandwidth added to media queries) <!-- Picture Element -->
| style="border:1px solid black; background:#FFFF99"   | Yes (if bandwidth added to image-set) <!-- image-set() CSS -->
| style="border:1px solid black; background:#FFFF99"   | Yes (if bandwidth added to media queries) <!-- Spacer background-image -->
| style="border:1px solid black; background:#FF9090"   | No <!-- Request headers -->
|-
| style="border:1px solid black"                       | Fallback for browsers without support
| style="border:1px solid black; background:#90FF90"   | Yes, img within noscript <!-- Picture Element -->
| style="border:1px solid black; background:#DDDDDD"   | (no structure to evaluate) <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Request headers -->
|-
| style="border:1px solid black"                       | Printable images
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- image-set() CSS -->
| style="border:1px solid black; background:#FF9090"   | No <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Request headers -->
|-
| style="border:1px solid black"                       | Polyfill
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Yes  <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Not needed <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Not needed <!-- Request headers -->
|-
| style="border:1px solid black"                       | Minimal structure updates for browser support
| style="border:1px solid black; background:#FFFF99"   | Create a new picture element <!-- Picture Element -->
| style="border:1px solid black; background:#DDDDDD"   | (no structure to evaluate) <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Not needed <!-- Spacer background-image -->
| style="border:1px solid black; background:#FF9090"   | Heavy restructuring <!-- Request headers -->
|-
| style="border:1px solid black"                       | Minimal presentation updates for browser support
| style="border:1px solid black; background:#90FF90"   | No new CSS required <!-- Picture Element -->
| style="border:1px solid black; background:#FFFF99"   | image-set being evaluated by webkit for background images <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Not needed <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | No new CSS required <!-- Request headers -->
|-
| style="border:1px solid black"                       | Image within content
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- image-set() CSS -->
| style="border:1px solid black; background:#FF9090"   | No <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Yes <!-- Request headers -->
|-
| style="border:1px solid black"                       | No new file formats
| style="border:1px solid black; background:#90FF90"   | Works with existing files <!-- Picture Element -->
| style="border:1px solid black; background:#90FF90"   | Works with existing files <!-- image-set() CSS -->
| style="border:1px solid black; background:#90FF90"   | Works with existing files <!-- Spacer background-image -->
| style="border:1px solid black; background:#90FF90"   | Works with existing files <!-- Request headers -->
|}