News

Add new post

Thoughts on adding a type attribute

Matt Wilcox brought this idea up a few months ago, but the issue seemed to fall off the table and was lost in other discussions. It’s something I’d like to see get some more feedback and general discussion from other developers so I’m bringing it back up. Below are some thoughts I’ve put together for adding a type attribute to source tags in the picture element.

  • A type attribute would allow for a native HTML way to use new image formats, which could help decrease the load of image assets, increase site performance, i.e. WebP, JPEG-XR.
  • The type attribute could also be a starting point for responsive image formats. As an example, a user could specify a new responsive image format which streams its data per however that magic would work and if the new responsive image format is not supported in the browser, it could fallback to the next image in the source stack, depending on which media query is active in the source list.
  • The idea of fallback images is something that is being worked on in CSS: http://dev.w3.org/csswg/css4-images/#image-notation
  • Since the picture element markup is very similar to the HTML5 video element, it seems to just make sense to also include type attributes just like the video element.
  • I’m curious if type could be an optional attribute and only needed if you are planning to use newer image formats that would need a fallback pattern. This would be ideal so developers don’t have to specify type=”image/jpg” or type=”image/png” on each source tag
  • Adding a type attribute seems like a very future friendly and forward thinking action.

Here is a gist of this concept: https://gist.github.com/3410238

Adding the type attribute to source tags in the picture element seems to be beneficial with no apparent drawbacks. I’m sure there are some issues so feel free to let me know what your concerns and issues/drawbacks are of adding a type attribute.

`Picture` in the HTML5 Spec

Paul Cotton, one of the HTML WG Chairs, has arranged for me to work with several members of the HTMLWG on a final draft of the `picture` specification, as we discussed at the HTML WG weekly teleconference on July 26th.

picture is tentatively slated for inclusion under the HTML5 umbrella, rather than being pushed to HTML.next. The status of the proposal can be tracked via the W3C Bugtracker. I’ll be working with Adrian Bateman and Frank Oliver from Microsoft on a formal specification in the coming weeks, and will have an update on our progress by August 16th (per https://www.w3.org/html/wg/tracker/actions/220 ).

Laura Carlson has also been working to gather feedback from the HTML WG Accessibility Task Force on the proposed markup, in addition to being tremendously helpful in navigating the standards process. Welcome to the RICG, Laura!

`Picture` in Drupal 8

The Drupal team is currently discussing the inclusion of the picture markup pattern in the Drupal 8 core. They seem to be leaning towards including the proposed element itself, rather than an equivalent pattern using Picturefill. I’ve let them know that doing so may be a bit preemptive at this time. If nothing else, this further proves the developer community’s interest in this solution.

I’ve asked Jesse Beach to act as the Responsive Image Community Group’s primary contact with the Drupal team to help field implementation questions and feedback.

The Drupal team working on this issue includes (among many) John Albin Wilkins, Jeff Burnz, Lewis Nyman, Tim Plunkett, Peter Droogmans, Théodore Biadala, Rob Wierzbowski, Wim Leers, medden, Gábor Hojtsy, Sam Richard, Alex Bronstein, Marc Drummond, Bojhan, and Angie Byron.

I’m really looking forward to hashing out some real-world issues, especially on such a large scale project.

Discussion about the first draft of

This page is to discuss the first-draft of W3C’s <picture>-element. Discussed and voted changes will be submitted to Wiki = draft.

 

1) Wouldn’t we drop alt attribute from picture to make use of the full alternative content inside the <picture>-wrapper? Thought we discussed this in several threads already… I personally think this makes sense.

2) Do we really need <picture> as element-name? WHATWG mailing list and its contributors seem to have spoken that img could also work…? If it would, it would be a much smarter solution.

3) Templating ways are still missing. We never discussed this to the end…

4) Does it really make sense to provide srcset along with media attribute? I think it is much better to either use srcset and let the UA detect which one to choose or provide media and fixed source to serve as the developer wants to.

 

First Draft of Picture Element Proposal published by Responsive Images Community Group

On 2012-07-02 the Responsive Images Community Group published the first draft of the following specification:

Participants contribute material to this specification under the W3C Community Contributor License Agreement (CLA).

If you have any questions, please contact the group on their public list: public-respimg@w3.org. Learn more about the Responsive Images Community Group.

The Picturefill named Div

So I was messing with my own site when this spark started (also the same time Scott was updating the repos) and I’d like to get some feedback from outside entities. To give you an idea of what I’m talking about let me show you this little snippet (pasted below) and mind you this is just for the stupid little social icons in my footer. I’ve implemented the most recent commits from Picturefill that I’d like to personally go ahead and coin as “Divfill.”

[side note : This Divfill artist formerly known as Picturefill now appends an img element within div[data-picture]. As you hopefully will see this markup pattern can get unruly very quickly if used on a larger scale.]

I've also created a new branch with my own fork of Picturefill where I'm putting together a real world example using this technique of Scott's on a larger scale –with a Flickr–like layout. I hope we can use this live example and build upon it as a way to identify verboseness and at the same time indicate what's working. I've done this only because a real world example is crucial to understand what we really need. This isn't to blow the wind out of the Divfill sail. The two example images in use are just not cutting it. As they say, the proof is in the puddin' so here comes the spoon.

Markup using div polyfill

The following is the snippet I used within my footer. I also set my fallbacks for Retina

 
<article id="footer-socials">
<h3>Follow Along</h3> 
<ul class="socials-listing"> 
<li class="twitter"> 
<a href="https://twitter.com"> 
<div data-picture data-alt=""> 
<div data-src="img/twitterBird.png" data-media="screen"></div> 
<div data-src="img/twitterBird@2x.png" data-media="(-webkit-min-device-pixel-ratio: 2),(min-device-pixel-ratio: 2)"></div> 
<!-- Fallback content for non-JS browsers. Same img src as the initial, unqualified source element. --> 
<noscript><img src="img/twitterBird.png" alt=""></noscript> 
</div> 
</a> 
</li> 

<li class="dribbble">
<a href="http://dribbble.com"> 
<div data-picture data-alt="Social Follow Buttons"> 
<div data-src="img/dribbbleLogo.png" data-media="screen"></div> 
<div data-src="img/dribbbleLogo@2x.png" data-media="(-webkit-min-device-pixel-ratio: 2), (min-device-pixel-ratio: 2)"></div> 
<!-- Fallback content for non-JS browsers. Same img src as the initial, unqualified source element. --> 
<noscript><img src="img/dribbbleLogo.png" alt=""></noscript> 
</div> 
</a> 
</li> 

<li class="facebook"> 
<a href="http://www.facebook.com"> 
<div data-picture data-alt="Social Follow Buttons">
<div data-src="img/facebookLike.png" data-media="screen"></div> 
<div data-src="img/facebookLike@2x.png" data-media="(-webkit-min-device-pixel-ratio: 2), (min-device-pixel-ratio: 2)"></div> 
<!-- Fallback content for non-JS browsers. Same img src as the initial, unqualified source element. --> 
<noscript><img src="img/facebookLike.png" alt=""></noscript> 
</div> 
</a> 
</li> 

<li class="github"> 
<a href="https://github.com"> 
<div data-picture data-alt="Social Follow Buttons"> 
<div data-src="img/githubOctocat.png" data-media="screen"></div> 
<div data-src="img/githubOctocat@2x.png" data-media="(-webkit-min-device-pixel-ratio: 2), (min-device-pixel-ratio: 2)"></div> 
<!-- Fallback content for non-JS browsers. Same img src as the initial, unqualified source element. --> 
<noscript><img src="img/githubOctocat.png" alt=""></noscript>> 
</div> 
</a> 
</li> 
</ul> 
</article>

Florian’s Compromise

I’ve been fairly quiet in this group for a while now. Part of that was wanting to let the dust settle on all the chaos that was surrounding this topic a few short weeks ago, and part of that was giving myself some time to digest the bits of signal that came through amidst all the “responsive images” noise.

More and more it seems that it’s a waste of effort trying to retrofit the original srcset proposal to cover all the use cases of the original picture proposal. As we attempt to do so, the srcset microsyntax grows and more and more confusing, and shares an increasing amount of overlap with media queries. To those ends, I asked Florian Rivoal — editor of the media query spec — to join the discussion on the WHATWG mailing list and offer his perspective.

Florian joined the list by posting a brilliantly thought-out compromise between the two syntax patterns. I’d like to share my thoughts on this proposal here, as I feel it combines the strengths of srcset and picture in a practical and logical way.

Let’s begin by taking a look at the proposed syntax.

Sample Markup Pattern


<picture alt="Description of image subject.">
  <source srcset="small.jpg 1x, small-highres.jpg 2x">
  <source media="(min-width: 18em)" srcset="med.jpg 1x, med-highres.jpg 2x">
  <source media="(min-width: 45em)" srcset="large.jpg 1x, large-highres.jpg 2x">
  <img src="small.jpg" alt="Description of image subject.">
</picture>

The chain of events followed by the above markup pattern are:

  1. If the picture element is unsupported, the img contained therein is shown as fallback markup.
  2. If picture is supported, use media attributes to determine which source element best suits the user’s viewport.
  3. Once an appropriate source element has been selected, the srcset attribute determines which image source is best suited to the user’s screen resolution. If only a single resolution is necessary, the src attribute will function as expected.

In terms of selecting a source element, this markup leverages all the strengths of media queries — the syntax created for this very purpose — to handle the “art direction” use case that Jason Grigsby has illustrated so eloquently.

However, as has been detailed at length on the WHATWG mailing list and elsewhere, device-pixel-ratio media queries are poorly suited towards these decisions. As an author, using vendor-prefixed min-device-pixel-ratio media queries in the example above would involve a massive amount of text and twice as many source elements. This could get unwieldy for authors very quickly, a concern voiced numerous times in these ongoing discussions. Further, implementation of MQ-based resolution switching is far more difficult on the UA side: a very real concern.

Once we’ve used media queries to determine the most appropriate source element, srcset’s originally intended usage becomes absolutely ideal for our purposes: simply determining the appropriate image source for a user’s resolution.

It’s worth noting that this example is, in fact, the most convoluted this element can ever be. This pattern in no way precludes the use of srcset on an img tag for simply preforming resolution switching, nor does it preclude the use of picture as originally proposed, with src in source elements rather than srcset.

Bandwidth

The dark cloud hanging over all these discussions is the concept of “bandwidth detection.” We cannot reliably make assumptions about bandwidth based on client capabilities — a MacBook Pro with a Retina display may be tethered to a 3G phone; a high-resolution mobile device is as likely to be connected to WiFi as it is an EDGE connection.

It would assume a great deal if authors were to make this decision for the users. It would add a point of failure: we would be taking the bandwidth information afforded us by the browser, and selectively applying that information. Some of us may do it wrong; some of us may find ourselves forced to make a decision as to whether we account for users with limited bandwidth or not. To not account for it would be, in my opinion, untenable — I’ve expressed that elsewhere, in no uncertain terms.

I feel that bandwidth decisions are best left to the browser. The decision to download high vs. standard resolution images should be made by user agents, depending on the bandwidth available — and further, I believe there should be a user settable preference for “always use standard resolution images,” “always use high resolution images,” ”download high resolution as bandwidth permits,” and so on. This is the responsibility of browser implementors, and they largely seem to be in agreement on this.

In discussing the final markup pattern, we have to consider the above. Somewhere, that markup is going to contain a suggestion, rather than an imperative. srcset affords us that opportunity: a new syntax _designed_ to be treated as such. I wouldn’t want to introduce that sort of variance to the media query spec — a syntax long established as a set of absolutes.

It seems srcset won’t be going anywhere, and that’s not an indictment. There is a time and a place for srcset. I feel that place is resolution switching, as it was originally intended. Our best efforts to bring srcset closer in-line with the originally proposed picture element only stand to leave us with a siloed microsyntax that inconsistently serves the purpose of media queries. With that comes further opportunity for errors by implementors and authors alike — countless new potential points of failure.

An Updated Polyfill

In order to better wrap my head around this pattern, I’ve updated Scott Jehl’s Picturefill to make use of the proposed syntax. The source code is available on GitHub, and I’ve posted a demo as well.

Next Steps

I’ve been discussing the implementation details of this pattern with several vendors recently, and the feedback has been extremely promising.

I’d love to hear everyone’s thoughts on this compromise, and through your feedback put together a set of formal proposals: a change proposal returning srcset to its original resolution-only syntax, and a proposal for picture that focuses on the “art direction” use case and optimization for client displays through media queries, excluding resolution.

The Constant Caveat

It seems I always end my posts in much the same way, but it always seems to ring true: this solution is not the ultimate solution to every problem in the “selectively loading assets” arena — nor does it have to be, right now.

There will always be limitless room for improvement when it comes to markup — a better way to handle source management for rich media, for example. But we can’t solve everything now — we can’t fall into the trap of seeking the perfect solution all at once.

Our goal is a laser-focused solution with the potential to fall in-line with other rich media elements, as we solve the greater issues one by one — issues of bandwidth detection; issues of organization. Our goal is to solve a very real and increasingly urgent problem, in a way that serves as a canvas for future enhancements. I’m confident that this syntax affords us that opportunity.

We can’t predict the future. We can only strive to be future friendly, while solving the problems of today. I’m confident this proposal does so.

A new proposal in the WHATWG Mailing List featuring both advantages

Kornel Lesinski just published a new proposal on the WHATWG mailing list this morning. He proposes a new element in HTML called <pic> which is shorter to write than <picture> would be.

This solution combines the advantages of our proposal of the picture-element and the proposal of the resolution based srcset approach by the WHATWG published recently.

The syntax also can be written in a short way and in a long way. This is good for developers because normal developers can write it the common HTML-like way while advanced developers are provided with a short-syntax.
Also the syntax is able to use @media-queries as well as resolutions.

In short syntax the code could look:

<pic src="small.jpg (max-width:320px), medium.jpg (max-width:768px), large.jpg">alt text</pic>

<pic src="portrait.jpg (orientation:portrait), landscape.jpg">alt text</pic>

<pic src="small.jpg 0.5x, medium.jpg 1x, large.jpg 2x" style="width:100%">alt text</pic>

While the long syntax could be like that:

<pic>
       <source src="large.jpg" media="(min-width:1024px)" width="1024" height"="300">
       <source src="medium.jpg" media="(min-width:768px)" width="768" height="200">
       <img src="small.jpg" width="320" height="100">
       alt text
</pic>

This of course is a first proposal to which I already replied with several improvements but it looks like we finally get in the right direction where we will get the best approach we can find together. And I am super happy that this has been proposed.

Here is the full description and here the link to the thread.

Respondu: responsive implementation framework (alpha) – implements picture and srcset

Update: After being inspired by the suggestions of Florian Rivoal (editor of the CSS Media Queries spec) and Markus Ernst over on the whatwg mailinglist Respondu now supports a new hybrid format that combines picture and srcset in a way that reduces markup and increases flexibility for the best of both worlds.

References:

Hi all,

I’ve developed a basic framework that allows us to implement a responsive technique (e.g. picture, srcset or any new/improved ideas), it’s called Respondu https://github.com/davidmarkclements/Respondu

(if anyone has a better name, let me know :D).

Examples can be found at http://respondu.davidmarkclements.com/

One of the main features of this framework is that it defers the loading of assets (e.g. src’s etc) until it’s finished processing (this includes script tags in the body). It achieves this using the noscript technique (discovered by filament group (?)) – except instead of wrapping img’s individually we wrap the whole body and then use a cheeky hack (dynamically wrap it in a style tag) to pull the noscript contents out (for browsers that wipe noscript contents on render).

Benefits,

  1. supports non-js clients (e.g. search bots, screen readers etc.)
  2. no wasted HTTP-connections or bandwidth
  3. it’s relatively unintrusiveness, you simply wrap it round all your body code
  4. extendible, you can easily create you own implementations

Once the final spec is implemented by vendors, the technique can be used as a fallback for
browsers without support for responsive, whilst also in turn falling back again for browsers without Javascript.

Currently, Respondu has picture and srcset implementations available, for instance here’s srcset:

<!DOCTYPE HTML>
<html>
<head>
<style>img {width:100%} /* use fluid layouts */</style>
<script src=js/R.js></script>
</head>
<body>

<script>window['#R']('srcset');</script>
<noscript>

<img src="images/photo.jpg" srcset="images/photo.small.jpg 320w, images/photo.small@2x.jpg 320w 2x, images/photo.medium.jpg 640w, images/photo.medium@2x.jpg 640w 2x, images/photo.large.jpg 1280w, images/photo.large@2x.jpg 1280w 2x, images/photo.xlarge.jpg 20000w, images/photo.xlarge@2x.jpg 20000w x2">

</noscript></style>

</body>
</html>

So far, I’ve tested it in Chrome, IE8, iOS Safari, and Firefox.

If anyone would like to test in other browsers (ie7, ie9, OSX Safari, Opera) please be my guest, you beautiful person.

Respondu is currently alpha as there’s a few things to do for smooth script integration (getting it to “just-work” with domready stuff), I’m totally sure the code could be tidier, and more optimized – any contributions to the code base or posted issues are invited, and warmly welcomed.

See the github page for more details!

https://github.com/davidmarkclements/Respondu

An update on responsive images

Hey everyone,

I guess a lot of you guys aren’t signed up to the whatwg mailing list so I thought it’d be good to give an update on what’s going on. I don’t claim to be an authority, in fact I’m sure my knowledge is flawed; Also things are moving so fast that any information here might be plain wrong by the time you read it! (please leave a comment if you see an error)

How srcset really works (-ish!)
When an image tag reads <img src=”mobile.jpg” alt=”” srcset=”medium.jpg 600w, desktop.jpg 1000w”> what this means is: default to mobile.jpg, unless the viewport width is greater* than 600px then the browser is free to choose mobile.jpg or medium.jpg as it sees fit. Once the viewport width is greater than 1000px the browser is free to choose any of the three images.

For a much better explanation read Bruce Lawson’s great html5doctors post

Tab’s suggestion
On the mailing list Tab Atkins (& others) suggested the following CSS like notation might be more helpful:
<img src=”mobile.jpg” alt=”” srcset=”medium.jpg min-width:600px, desktop.jpg min-width:1000px”>
And the browser would be free to choose any image where it’s corresponding media query is satisfied (this is different to <picture> where the browser gets no choice). Also only a subset of media queries would be allowed (min-width, max-width and maybe min-pixel-density). This is more friendly to developers that know CSS, it’s more readable and allows for units other that px to be used. On the downside it’s slightly more verbose.

Going forward
I’d suggest you all join up to the mailing list, but if you don’t want to receive dozens of emails a day, feel free to leave comments/thoughts below.

* there is discussion whether “600w” should reference the maximum viewport or minimum viewport, however for the sake of brevity I’m going to just to talk about minimum.