<picture> respecting conventions.

After checking out the etherpad with brainstorming, asking some questions in this group and seeing what Scott has done with the proposed markup using <picture> , I like the way we go. Now I clearly see the reasons why we can not re-use <img> . I see why some other proposed solutions don’t deliver what we need and how <picture> solution tries to solve some of those concerns.

But today, after the Matt’s post I realized that there is something more we should keep in mind. There are certain ways people used to interact with the images on the web. There are certain conventions. Saving images to a user’s hard-drive, maybe referencing an image in a visual editor’s “Add Image” drawer to name a couple.

I gave another though to the <picture> element on how we can deliver the responsiveness and, at the same time, let the conventions work still. Here is what I came up with:

<picture>
    <source query="(min-width:420px) and (max-width: 767px)">
        <img src="http://dev.null/small.jpg" alt="Alternative text for capable browsers wider than 420 but narrower than 768px" />
    </source>
    <source query="(min-width:768px)">
        <img src="http://dev.null/large.jpg" alt="Alternative text for capable browsers wider than 768px" />
    </source>
    <img src="http://dev.null" alt="Alternative text for browsers not understanding CSS media queries or screens smaller than 420px" />
</picture>

So, as in the original proposal, we have <img> fallback for the browsers not supporting CSS media queries. The same <img> fallback is used for capable browsers in case none of the media queries is true. We have a set of <source> elements to reference different versions of the image. The differece is that I intend <source> to contain real <img> tag. Any browser knows how to deal with it and it is clear for any user what to expect from it. Additional advantage is the flexibility we get in supplying more information than just an image for different media queries. For example a caption for an image or alike (though it starts smelling like <figure> already).

What I like about this solution.

  • This can be re-used for anything, not images only. If we decide to let wider usage of this pattern, we might just easily drop <picture> and leave those <source> elements with query attributes standalone. Imagine using this for different navigational elements depending on the screen size, supplying a screenshot instead of full-size video on the smaller screens and so on.
  • No need to duplicate browsers’ functionality reserved for <img> tag.
  • Not breaking conventions users got used to when dealing with images on the web.

What I don’t like about this solution.

  • Duplication of the alt attribute for the images. Though it might be an advantage. You might need to supply completely different image for smaller screens (consider complex chart on a wide screen that is significantly simplified for the smaller ones) where alt text should be different for different screens.
  • Mixing presentation with markup by using CSS media queries. Though, pretty much any solution we came up with until now had this flaw.
  • Use of the new element ( <picture> ). But, as mentioned above, we can ditch it and leave standalone <source> elements instead. <source> is an existing tag. Though query attribute has to be implemented yet.
  • Not very elegant markup.
  • Maybe a little bit too much flexibility — some people can go nuts supplying different content for different resolutions. This can easily break usability of the sites.
  • UPDATE I don’t like that this doesn’t work in the browsers just now even if one tries to polyfill it.
    Browsers should implement a mechanism to not download an <img> if it is within <source>
  • UPDATE This pattern is new and does not correspond to the existing usage of <source>

What do you think?

18 Responses to <picture> respecting conventions.

  1. Ben Callahan says:

    Please forgive me if this is obvious or if I’ve missed the discussion on it already, but could we add an element which is intended solely for the purpose of executing a media query (since we all seem to be OK with having media queries in our content)? Obviously, this could be used in many more instances, but for images we might do something like this:

    I suppose, after considering more closely, that there isn’t a good fallback…

    [posting the comment anyway, in hopes that it spurs other (more useful) ideas]

    • Denys Mishunov says:

      Hi Ben,

      Thanks for your comment. I think the code block you were intended to have in your reply got wiped out. Could you please re-post it?

      Concerning a media query element. Yeah, that might be a way to go. In my example, SOURCE element works like that. I could use some new element for this of course, but since this is an existing element, I thought it’s good to re-use it even though with a new attribute. So, as mentioned in the post, we could get rid of the surrounding PICTURE element and live with just SOURCE defining media queries.

      • Chris Krycho says:

        The major downside to this is that it’s very quickly approaching (if not already crossing the line into) mixing presentation with content. The nice bit about a <picture> wrapper around everything with sources specified (i.e. making this analogous to the video or audio specs) is that it allows presentation to be controlled independently of content delivery. I think that’s something we should stay far away from. Let the content be the content; change how it’s presented, possibly even change which asset you deliver with server-side machinations/templating/etc., but don’t start putting in media queries in the actual content.

        Because from media queries in the content, we can only be one step away from <marquis>, and you all know it in your heart of hearts.

        Even granting that it might not be as bad as all that, the point remains: we should continue to try to keep presentation separate from content.

        • Denys Mishunov says:

          Hi Chris,

          The current “go” solution for <picture> already has the new media attribute that uses the CSS media queries syntax. If we could come up with something that doesn’t require us to mix this bit of CSS into HTML I would be all for that. But til now all such solutions were tied to some sort of attributes like s, xs, l and so on. These are pretty inflexible in my opinion. Moreover there has to be a separate mechanism that decides how those correspond to the actual screen size or bandwidth or any other criteria.

          So, for now, I don’t think we have any way to avoid mixing CSS into HTML. Even though it smells like the experiences we have had before.

          • GaryJ says:

            > If we could come up with something that doesn’t require us to mix this bit of CSS into HTML

            Instead of repeating the same media query values on each occurrence of picture within a page, could they be set for the document once, in a meta element?

            e.g.

            <head>
            <meta name=”media” content=”large=screen and (min-device-width:1024px)” />
            <meta name=”media” content=”medium=screen and (min-device-width:640px and max-device-width:320px)” />
            <meta name=”media” content=”tiny=screen and (max-device-width:320px)” />
            <meta name=”media” content=”foo=print” />
            </head>

            <body>
            <!– whatever the decided content markup structure is, but referencing the meta values –>
            <picture alt=”…”>
            <source src=”foobar-150×150.png” media=”tiny” />
            <source src=”foobar-300×300.png” media=”medium” />
            <source src=”foobar-600×600.png” media=”large” />
            <source src=”foobar-black-white.png” media=”foo” />
            <img src=”foobar.png” />
            </picture>
            </body>

            Code repeated at https://gist.github.com/1901483 for convenience.

            There could be as many query conditions / break points as needed, the name could be anything (so not English-specific), the CSS is kept out of the actual content, would be consistent across the page, and could be made consistent across a site (and more importantly, easily changed across a site) via CMS or other templates etc.

            Meta tags already set things like viewport width, so it wouldn’t be a huge leap to defining media query values there too.

            In addition, the sizes would also be available to video, audio or other such elements that need multiple sources, without needing to define any new elements. Any improvements to media queries that later support bandwidth or other factors can be handled via the meta tag, keeping it abstracted away from the content elements that can use them.

  2. Ben Callahan says:

    Lost the HTML—trying again…


    <query media="max-width: 500px">
    <img src="small.jpg" alt="small image">
    </query>
    <query media="min-width: 501px">
    <img src="large.jpg" alt="large image">
    </query>

  3. Denys Mishunov says:

    Tried to polyfill this idea, but apparently it doesn’t work because browsers download all images right away. There has to be something preventing browsers from doing so if an image is within SOURCE or, maybe PICTURE

  4. Mathew Marquis says:

    Unfortunately, this solution would require heavily modifying the behavior of source tags, which would then have two very different markup patterns for the sake of picture vs. audio/video. This not only means a steeper learning curve for the developers, but would introduce a major barrier to adoption with standards bodies and browser vendors. We would also be breaking from the alternate-source-delivery scheme already in use on audio/video. This would also mean a great deal more markup (as well as modifying already specced elements) to achieve, effectively, the same thing as the original proposal.

    It sounds like your primary concern is that the user would be unable to interact with a picture element the way they currently do with standard img tags, yeah?

    It may not productive to try to retrofit a new element to suit the desired behavior, though. The benefit of the standards process is that we developers, in conjunction with browser vendors and standards bodies, can codify the behavior we want. It’s a pretty radical change in thinking: as developers, we’re all used to working around “the way things are.”

    This is our chance to decide “how things are.” Let’s work towards making the picture tag exactly what we want it to be.

  5. Scott Jehl says:

    Agreed with Mat. This idea is a nice suggestion, but I don’t agree with the premise that there’s a major problem with the simpler picture element proposal, and browsers will certainly be free to implement it in ways that seem most useful to people.

    If a picture element has many source children, a user agent might choose to add behavior to allow the user to specify alternates to view or download, perhaps via a context click menu. Developers have typically had control over these sorts of things in the past, in regards to what defaults are applied in a certain setting, and beyond that, the browser could certainly expose other means of interacting with the element and exploring all the media it references.

    There’s a parallel here to video too, I’d think. It’s also similar to alternate stylesheets – which browsers often expose to the user alongside zooming and other view settings.

  6. Denys Mishunov says:

    Hi Mathew and Scott.

    Thanks a lot for your comments.
    I really like the idea of letting a user agent do some fancy stuff like selectable source downloading for an image. Would be great to have that. But I would like to ask and note some things. Please don’t get me wrong here. I am new to this process and don’t really know how it should work and how this bubbles up to the browser vendors. I am just trying to think of a mechanism that would make my life as a developer easier when working with responsive content.

    * I totally agree with Mat in “It may not be productive to try to retrofit a new element to suit the desired behavior”. But there is a certain expectation users have for the images in their web browser no matter what. They should not have any idea of what is under-the-hood of this image. If browser provides new fancy feature of alternative source downloading, fantastic! I would love it. But if it looks like an image, it should be an image. New features should come in addition to the core, not instead of it in this particular case. And, in my opinion, we should have really-really good reasons to break this expectation. It’s not about retrofit, but about backwards-compatibility with an expectation that is here since the early days of web. New elements worked for <video> and <audio> just because those were really new in my opinion. Until their introduction, the only expectation users had about those objects on the web were — they require a plugin or non-semantic hack-arounds in the markup. So new semantic elements, brought a gulp of fresh air. The situation with images is completely different in my opinion.

    * The original proposal, in contrast to the suggestion above, has an advantage of re-using established patters of <source> in <video>/<audio>. This is great and I like it. Even though we expect <source> to implement a new attribute `media`. But replace that <source> tag in my example with, let’s say, <media> (to match `query` attribute and form analogy to CSS media query to set certain expectations for developers) or whatever you can come up with as a new tag that has no set usage patterns yet, but with the same syntax as in the example above. I would like to discuss what is wrong with a solution like this in general even if it doesn’t belong to this particular group.

    * Probably this is not the task for this group to provide a generic wrapper for anything-responsive. But if we start mixing CSS into HTML anyway, why to artificially limit it’s use by saying, “We let only media types to be responsive”? Why not to give more flexibility to the developers? As in my example in the post — there might be a need to deliver different content instead of an image for a certain media query. Responsive adds anyone? How do we address that?

    * Referencing an image in visual editor. Not a big deal. But for a person coming from CMS world, where the CMS drives thousands of sites running in production, this point bothers me a little. You can’t expect visual editors to change their behavior to support both <img> and <picture> at the same time all of a sudden. And now we leave it up to the end users, who might be not very technical, to realize that the image they want to reference in their visual editor is not a real image even though looks like that. And there are quite a few tools and utilities that give a user ability to reference an online image. If we say, “We don’t care about this usecase and we let the editors’ authors take care of this whether they want or not”, fine by me. But we should be aware of this implication.

    * The original proposal doesn’t provide a way for dragging an image to my Desktop, does it?We can make browsers treat it as an image. But what do we do with OS’es when one tries to drag an image to Finder, for example, to save the image that is not really an image tag? Will it still work? Is it just me being spoiled by things being draggable or other people do that as well? :)

    * A user agent could still implement some alternative source downloading mechanism for images wrapped with <picture>, containing different <media> elements in it. I know this is not as neat as it could be with the original proposal, but there is a possibility for that in here anyway.

    • Anselm Hannemann says:

      * Probably this is not the task for this group to provide a generic wrapper for anything-responsive. But if we start mixing CSS into HTML anyway, why to artificially limit it’s use by saying, “We let only media types to be responsive”? Why not to give more flexibility to the developers? As in my example in the post — there might be a need to deliver different content instead of an image for a certain media query. Responsive adds anyone? How do we address that?

      We already talked about responsive ads. This shouldn’t be a problem in case they’re an image. But yes, you’re right talking about responsiveness everywhere. Why not adding responsiveness to video-tag, too? Why not for every element? Means you have this media-attribute on every block-element.
      But that would be another topic. Where we can focus here is a new element. If we name it how is its behavior? What if you have HTML content in there? I think it’s not good to have section elements etc in there which might be necessary for texts. So we could do a media element which could have media in it which could be responsive. Not more not less it should be. :)
      So let’s think a bit more abstract how to handle this within our new element.

      Referencing an image in visual editor. Not a big deal. But for a person coming from CMS world, where the CMS drives thousands of sites running in production, this point bothers me a little. You can’t expect visual editors to change their behavior to support both and at the same time all of a sudden.

      Don’t think that’s a problem because of course the editors have to adapt its code if they decide to use the new elements and syntax. If they don’t use no problem at all, right? It’s how HTML5 works. If you want to use something new, it needs to be implemented. Otherwise you have to use the old way. Nothing a user has to know if he uses WYSIWYG editors.

      * The original proposal doesn’t provide a way for dragging an image to my Desktop, does it?We can make browsers treat it as an image. But what do we do with OS’es when one tries to drag an image to Finder, for example, to save the image that is not really an image tag? Will it still work? Is it just me being spoiled by things being draggable or other people do that as well?

      This has been addressed in another topic here in the group. Look for “save image…” article…

  7. Isaac Lewis says:

    Denys, thanks for your insightful post — very interesting.

    I am new to this group, but have been part of the discussion on Etherpad, and some other twitter convos over the past month. I was wondering if you could expound on why the tag won’t be feasible? Is there another article somewhere that gives some concrete reasons?

    Thanks again,
    Isaac

    • Isaac Lewis says:

      Bugger… this strips out HTML tag references.

      Sentence “…expound on why the tag…” should read “…expound on why the img tag…”

      srry.

    • Denys Mishunov says:

      Hi Isaac,

      I am glad you find this idea interesting.

      Concerning the <img> tag, check out the very first proposal in the Etherpad. It highlights the main problems with it — a) pre-fetching of the images in Chrome (and Firefox, I think) and b) lack of backwards compatibility. Messing up with <img> is not going to be accepted if it's not backwards-compatible. And I see no way fo making it such.

      Hope this answers your question.

  8. Kevin Suttle says:

    I posted my thoughts that are relevant to this discussion. Summary: we need a semantic element, and ‘picture’ isn’t it.

    http://www.w3.org/community/respimg/2012/02/20/14/#comment-134

  9. Pingback: Адаптивные изображения в HTML5: конец первого раунда

  10. Pingback: brvty++ ? | Christian Heilmann

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Before you comment here, note that this forum is moderated and your IP address is sent to Akismet, the plugin we use to mitigate spam comments.