Our goal is a markup-based means of delivering alternate image sources based on device capabilities, to prevent wasted bandwidth and optimize display for both screen and print.
Note: When the group expanded its scope in November 2014, it changed the name from "Responsive Images" to "Responsive Issues."
Group's public email, repo and wiki activity over time
Note: Community Groups are proposed and run by the community. Although W3C hosts these conversations, the groups do not necessarily represent the views of the W3C Membership or staff.
One of the challenges faced when working with responsive images has been managing each of the image variants, and deciding which image file to request depending pixel ratio and network connection.
To generalize an example, a paragraph element containing text provides the structure, and its presentation comes from CSS. If the paragraph renders to 3 lines of text, and the browser changes size to where the paragraph now renders to 5 lines of text, the content is still the same, but only its presentation has changed.
I think content images should be seen in a similar manner. The img element provides the structure and the src attribute is the content. However, depending on browser conditions, the presentation of the image should change, not the content. Whether an image of your cat is viewed on a high-res or low-res display, the image is still the same “content” of a cat; its just that they’re different variants of the same image.
The img element, or whatever next-gen element comes around for displaying images, should only worry about providing the content of the image, and let the CSS control presentation, just like any other element. And because image filename examples, such as milo.jpg and milo_2x.jpg, are actually providing the same content, just a variant in quality, the CSS should be given the task of deciding which image variant to show.
Image-set() CSS is a great format for the browser to decide which image variant to show depending on browser conditions. For example, if the browser has a device pixel ratio of 2 and high-bandwidth, then the browser would know to show image file X, otherwise show image file Y. I also agree that using image-set() should be preferred over media queries, because (as the article referenced above mentions), “Using @media pushes the two asset references apart from one another, whereas such a function keeps related asset references together. It also helps keep selectors DRY.”
However, the direct image references may still cross the line by putting specific content image filenames in the CSS (yes background-image url() has specific image filenames, but I see background images as presentation, not content). Instead of having CSS specify exactly which image file to show, the CSS should be used to specify which image variant to show.
I put together foresight.js more as a proof-of-concept than anything else. It’s a tool to make responsive content images work in today’s browsers, and judges which image variant to show depending on network connection speed and device pixel ratio. This project is not attempting to be an end-all answer for responsive content images, but rather a practical workaround for today’s browsers.
The original version of foresight.js used a lot of element attributes and javascript configurations, which was in violation of my points above. But since the web community has showed an interest in foresight.js, I decided to take it another step further and made it configurable using CSS.
Foresight.js uses a number of tricks to pull off responsive images, none of which are groundbreaking, but rather very common workarounds. However, the CSS is applied to the responsive content images and follows the concepts that I described above. The javascript of foresight.js is doing the work I believe the browser should do, which is to decide which image variant to show depending on browser conditions. But the javascript itself is not what I would like to present, but rather how CSS could be used to control image variants of the same image, without hardcoding image filenames.
Again, foresight.js is a proof-of-concept for using image-set() CSS, along with formatting image requests using URI templates and URI find/replace methods. Hopefully this can help spark further discussions to eventually shed some light for handling responsive content images: https://github.com/adamdbradley/foresight.js
I’ve spoken to a few people on Twitter about this, but it occurred me to that the simplest way of dealing with content images (as img tags) in a responsive design and for retina screens was to use a spacer PNG (generally smaller than an GIF) and background images.
All it requires is a blank PNG or GIF the same size as the intended image, and then setting that image’s background to whatever image needs to be served along with background-size: contain.
It works because the spacer PNG is transparent, which allows the image’s background image to be visible. Because the background is set with CSS, it can then be changed with media queries as required to serve different images to different viewport sizes or resolutions.
And yes, it even works with serving "retina images" to the new iPad.
Pros
All users, regardless of device width/resolution/whatever get the image that’s targeted for them with minimal overhead.
Requires no additional markup – just the img tag
Works with existing technologies and markup.
No dependencies on JavaScript or other polyfills.
Images will still retain any accessibility features from the alt text.
Replacement images can easily be targeted with the [src] attribute selector if a consistent naming convention is used.
Cons
An extra HTTP request to first load the spacer PNG.
A unique PNG is required for every different dimension of content image that needs to be made responsive.
Replacement images defined in CSS, not in markup or on the image itself.
Several posts are being passed around regarding the 3rd generation iPad’s Retina graphics holding a direct light to how compressed our images are on the web.
I thought I’d start a discussion on possible approaches (notice I didn’t use the word ‘solutions’ because I think it’ll be an iterative process), and the potential pitfalls this might
bring in future responsive designs. So, it’s GO time.
WebP is awesome, far better compression than JPG or PNG etc. Modern browsers support it. But, we can’t use it at all until everyone uses a browser that supports it. Because there’s no way in HTML to do fallbacks for non-supported image formats.
Remember when we wanted to use Alpha PNGs but it took years and years because we couldn’t rely on browsers implementing it?
This is an opportunity for us to define a method whereby <picture> could take a selection of resources to use depending on browser capabilty. I.e., if WebP is supported request that file, if not then fall back to a JPG or PNG, as set in the tag.
This would perhaps speed up adoption of WebP, and more importantly allow for future formats to be developed and become usable without the wait for every browser to implement whatever new formats appear and then for the public to all use such browsers.
What do you think? Should we also try to tackle this “short-sighted” file-format behaviour? I would like to be able to save 20%+ on all my images by using WebP sometime before 2020…
UPDATE: Read this post if you want to know why we don’t need different content. Matt explains this in comments.
I recently saw this post and was happy to see what this man had done: http://css-tricks.com/lark-queries/. He changes content when viewing on different devices / media.
I think we need different resolutions and different content on different devices. Not every text is proper for a smartphone. This has also been addressed in several blog posts out there.
So why not covering this problem with our new syntax?
As Matt already proposed he’d like to have a universal indicator to reference the responsive-assets. Now we have the <picture>-element but why don’t we just use e.g. <responsive> as name so we can have any content in there?
<responsive> should then be a inline-element as we could have this inside a h1-h(n),p,span or other elements, I think. Or we need to identify the content by providing a type-attribute?
I know this is a request that isn’t simple and easy to figure out how to do it but I think it can be important to think about it because we’re now building the future of responsive assets for the next years! And I want a good solution 🙂
I was surprised to see this method hadn’t been attempted before so I put together a demo (full explanatory post is here), here’s the tl;dr :
A resolution and bandwidth responsive image system is shown. All img tags start pointing to the low res image versions, then javascript downloads (in the background) difference files to upgrade the low resolution images into a high resolution images, and as such doesn’t waste http requests or data… oh and there are some fun kitten demos as well. All files can be found at github.com/pci/deltaimg.
I realise it probably only has niche usage, but it’d love some feedback.
I want to keep up on the conversations in this group, but find it very difficult to remember to come back to the site and open every thread to see if there are any new comments.
Am I missing some magically way to keep track of the conversations?
Also, I wonder if we would be better off using the mailing list associated with this group (which until recently I didn’t know existed) as a way to keep conversations going and allow people to control the volume of updates they receive (e.g., digest mode).
If no one else is having this problem, I’ll suck it up and get in the habit of checking the site more regularly. Or if I’m missing some way to keep up to date on activity in the group, please let me know.
In an early post, I offered up a sample picture element implementation in JavaScript (a polyfill) that we could use to play with and plan how the element should behave when/if implemented. The script did “work” in that it used the proposed picture element markup to deliver the appropriate image asset based on source elements with media queries, but it came with the common drawback that the fallback image request could not be prevented, incurring unnecessary overhead in larger-screen environments. Not ideal for production use…
However, with a small addition to the boilerplate picture markup and a tweak to the JavaScript, we can feasibly kiss that overhead issue goodbye without sacrificing accessibility in non-JS environments, giving us all the benefits of the picture element right now, provided we’re okay with the addition of a noscript wrapper (*) around our fallback image. Those newiPad images already begin to sound a lot less scary…
So, is it responsible to suggest that developers might use something like this today, given that it provides a workable solution to this very real problem, even if the markup is not yet standard or supported natively anywhere? I’m certainly attracted to the idea, given how hard it is to do this through other means. Perhaps it’d even encourage browser implementations more if the pattern gained traction like this, I’m not sure.
A native implementation would still of course allow image sources to be fetched immediately when the document is parsed, but as a fallback polyfill for today, I think I’m comfortable with the advantages that this slightly ugly workaround brings.
Feedback welcome!
Thanks
Scott Jehl
* The noscript idea here isn’t entirely new… Head London cleverly proposed it a while back. For some reason, it feels a little more purposeful to me when the noscript is used as fallback content in a picture element.
It can be frustrating seeing this Community Group go quiet, especially following a surge in traffic that very literally brought the Community Group site down. We have to understand—and I’m speaking in no small part to myself, here—that most standards folks have a massive backlog of issues and suggestions to work through. With that in mind: even as public-facing as this Community Group is, it’s a lot to ask of members of the standards bodies or browser teams to parse though several pages of prose and leaf through comments for the meat of this thing.
For that reason I’ll be making a first pass at a draft spec myself. Not with any intent to see it codified word for word, mind you, but to break down the core details of this new responsive images element in the most efficient and easily parsed way. All signal, no noise. This will be a big departure for me, since—as many of you know—I’m especially noise inclined.
We’ve done a lot of work reasoning this thing out and adapting it to community feedback to get it to this point: a point where it feels like it may be ready for sample implementation. I find this particularly exciting—like many of you, I fell into this website-making gig because it afforded me a chance to solve new and exciting problems. There’s not much I enjoy more than seeing a solution to a tricky issue come to life, and I’m certain that’s a sentiment shared by many browser developers.
If you’re a similarly-inclined browser developer, I’d love to work with you on a sample implementation of the <picture> element. I wouldn’t expect anyone to barrel ahead with this based on the content of this group or an email exchange with me, but rather once we have a pared-down and easily-distributed example spec to reference along the way. Of course, it goes without saying that I’m more than happy to work alongside vendors in whatever capacity I can.
If this is something you think that you or your team might be interested in exploring, I’d love to hear from you while the draft spec is coming together.
Our goal is a markup-based means of delivering alternate image sources based on device capabilities, to prevent wasted bandwidth and optimize display for both screen and print. From an accessibility standpoint, it should at the very least have something equaling img’s alt tag. For non-supporting browsers, fallback content should be displayed.
This Community Group is my effort to bring a growing number of developers together with some like-minded browser and standards folk, so we can iron out any issues we may not have considered and keep things moving forward.
The idea is to use the video tag’s markup pattern as the inspiration, as it’s specced to allow the use of media queries within attributes on its source elements and reliably displays the markup inside the tag in any browser that doesn’t recognize it. As with the video and audio tags, this solution shouldn’t require any custom scripting, HTTP/SPDY headers, or server-side technologies to reliably deliver content tailored for the end user’s context. Through use of media attributes we would not only be able to reduce wasteful image requests for the sake of users with smaller displays, but we could tailor our images’ resolutions for users with high-res displays, or for printing.
<picture alt="Alt tag should accurately describe the image represented by all sources, though cropping and zooming may differ.">
<source src="mobile.jpg" /> <!-- Matches by default. -->
<source src="high-res.jpg" media="min-width: 800px" /> <!-- Overrides the previous source over 800px before any assets are fetched, resulting in a single request. -->
<img src="mobile.jpg" /> <!-- Fallback content, in the event the <picture> tag is completely unsupported by the user’s browser. -->
</picture>
It turns out we reached much the same conclusion as many others have previously—in some cases, right down to the semantics:
We posted all this information to the WHATWG mailing list recently, but ended up covering a lot of the same “what if we”/“how about” ground. While a little disjointed, you can mine through said thread at http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-February/ by doing a search for “responsive images”.
I’m looking forward to some focused, productive discussion here. Let’s get this thing done.
ResponsiveImagesCG/
