Picture Element Proposal

From Responsive Images Community Group
Revision as of 20:20, 20 July 2012 by Mmarquis (Talk | contribs)

Jump to: navigation, search

Rationale

This proposal has been submitted to the HTML WG for consideration as part of HTML.next and the WHATWG for their ongoing consideration.

This proposal addresses the need to optimize the users’ experience of images across an increasingly diverse range of displays. It addresses this issues through a markup-based means of delivering alternate content image sources, based on device capabilities.

This proposal is based on Florian Rivoal’s original proposal to the WHATWG.

Thank you for your consideration.

Mat Marquis Chair, Responsive Images Community Group

Summary

Serving images within a flexible layout, or a layout dictated by media queries, requires a source image with an inherent size equal to the largest possible display size which is then scaled with CSS. On smaller displays such as phones and tablets — where bandwidth is often at a premium — this can not only mean an exceptionally wasteful request, but an image poorly-suited to the user’s unique context: aspect ratio, color depth, viewport size, etc.

A single image source is unlikely to be appropriate for a wide range of displays, in terms of composition. Take, for example, the following photograph of President Obama speaking at a Chrysler Plant:

Obama-500.jpg

The background adds additional context at larger sizes. When the image is resized for smaller displays, the original subject matter is nearly lost:

Obama-100.jpg

Ideally, rather than simply resizing the image, an alternate crop of the same subject could be introduced on smaller displays to better convey the focus of the image:

Obama-100-art.jpg

There is a need to enable authors to provide different sources for images at different sizes not based on resolution or network speed, but based on the judgment of the designer for what is the best image source at a particular breakpoint.

Proposed Solution

To satisfy the requirements below, we propose a new picture element that provides the following:

  • A container for disparate source elements, all of which represent the same subject matter.
  • A consistent and predictable pattern for delivering alternate media sources based on client context based on media queries, within attributes on source elements.
  • The ability to optionally specify resolution-appropriate sources on each source element, using the srcset attribute in place of src.
  • A proven, reliable fallback pattern for non-supporting browsers.

The proposed element is based on the markup pattern established by the HTML video element.

Details of Proposed Solution

Any combination of existing media queries could be used in conjunction with srcset attributes to determine the appropriate image source based on a combination of display size, orientation, and resolution. This is identical to the specified behavior of media attributes on the video’s source elements, as outlined in the spec. Any implementation of the picture tag should allow for the inclusion of fallback markup that is ignored by any UA that supports picture, and is only displayed in browsers that do not recognize the new tag. UA's may decide to ignore HD content by default and expose to the user a means of defaulting to HD, or the option to make that decision “on the fly,” based on available bandwidth.

Note that older browsers can be polyfilled with behavior similar to a native implementation ( see #Functional_Polyfill ).

Sample Markup Pattern

<picture>
    <source srcset="small-1.jpg 1x, small-2.jpg 2x">
    <source media="(min-width: 18em)" srcset="med-1.jpg 1x, med-2.jpg 2x">
    <source media="(min-width: 45em)" srcset="large-1.jpg 1x, large-2.jpg 2x">
    <img src="small-1.jpg">
</picture>

The chain of events followed by the above markup pattern are:

  1. If the picture element is unsupported, the img contained therein is shown as fallback markup.
  2. Uses media attributes to determine which source element best suits the user’s viewport.
  3. Once an appropriate source element has been selected, the srcset attribute determines which image source is best suited to the user’s context in terms of pixel density. Theoretically, this can be overridden based on settings within the UA — bandwidth detection, or a user preference. If only a single resolution is necessary, the src attribute will function as expected.

Functional Polyfill

The logic proposed above has been implemented in JavaScript as a theoretical polyfill:

Demo

http://wil.to/picturefill/

Source

https://github.com/Wilto/picturefill-proposal

Relative Units

A common practice in creating flexible layouts is to specify the value of one’s media queries in relative units: em, rem, vw/vh etc. This is most commonly seen with ems in order to reflow layouts based on users’ zoom preferences, or to resize elements through JavaScript by dynamically altering a font-size value.

Through the use of media queries this layout flexibility would extend to images as well, ensuring that a source always remains appropriate to the size of its containing element.

Use Cases

Mobile-First Development

A common approach in sites that cater to a wide range of devices using a single codebase is a “mobile-first” development pattern — starting with a simple, linear layout and building complexity through larger screen sizes using media queries.

Developer Pattern

<picture alt="">
    <!-- Matches by default: -->
    <source src="mobile.jpg">
    <source src="medium.jpg" media="min-width: 600px">
    <source src="fullsize.jpg" media="min-width: 900px">
    <img src="mobile.jpg">
</picture>

Assuming a 960px wide window at the time the page is requested and the sample markup pattern in section 3.1, the UA should make a single request for “fullsize.jpg.” Any window/screen smaller than 600px is served “mobile.jpg”, which—as a completely alternate source—could be cropped as well as resized in order to preserve the focus of the image at smaller sizes.

This example could be made more specific through the orientation, device-width, and device-height media queries.

Desktop-First Development

Following the reverse of the logic above, authors may want to ensure that the largest possible images are served by default. For example: a site intended for browsing across a range of “smart televisions,” for example, where one assumes a large viewport but assets could be optimized for smaller displays in the event that media queries are supported.

Developer Pattern

<picture alt="">
    <source src="default-fullsize.jpg">
    <source src="medium.jpg" media="max-width: 1400px">
    <source src="smaller.jpg" media="max-width: 1280px">
    <img src="default-fullsize.jpg">
</picture>

This use case could be further tailored through the aspect-ratio and device-aspect-ratio media queries.

Alternate Print Sources

Where media queries allow conditional logic based on whether the page is being rendered for display on screen or for print, one could extend this same logic to provide a print source other than the one rendered on the screen. For example: a photo sharing site could provide a bandwidth-optimized image for display on screen, but a high resolution image for print.

Developer Pattern

<picture alt="">x
    <source src="standard-res.jpg" media="screen" />
    <source src="high-res.jpg" media="print" />
    <img src="standard-res.jpg" />
</picture>

Requirements

  • The appropriate asset MUST be fetched by way of a single request. A change in window size causing the media attribute to match an alternate source SHOULD trigger a request for said source (to be retrieved from the browser cache, if possible).
  • As with the video and audio tags, this solution MUST NOT require any client-side scripting, server-side technologies, or headers to reliably deliver content tailored for the end user’s context.
  • Similar to the video tag, fallback markup MUST be rendered in any browser that does not recognize the picture element. The example in 3.1 uses the “mobile”-sized image as the fallback content, which is the recommended approach: barring the use of a polyfill, the smaller/low-res image should be provided as a fallback to prevent incurring a costly download in contexts that may see no benefit.
  • The specification MUST provide at least the same level of accessibility as img, with an alt attribute readily accessible to assistive technology.