Responsive Images breakout session TPAC 2016 -- 21 Sep 2016

<zcorpan> Simon Pieters, Opera

<gregwhitworth> gregwhitworth, Microsoft

<yoav> Yoav Weiss, Akamai

<nigel> scribenick: nigel

<dsinger> …is David Singer (Apple)

<cyril> Cyril Concolato, Telecom ParisTech & GPAC

<jihye> Jihye Hong, LG elec.

nigel: Nigel Megitt, BBC

yoav: Responsive Images is something we worked on and implemented - it's awesome.
... but we still see people advocating fixed width and height in the HTML because the browser
... does not know the image's aspect ratio and height for performing layout unless the authors
... perform hacks to get the aspect ratio in place. So this is a problem that needs solving that we want
... to discuss. We have some emails this morning on the responsive images mailing lists proposing
... various declarative markup ways to do that; there are also proposals for a CSS based solution to replace
... the existing hacks to specify aspect ratio for images and other elements. Discuss.

<zcorpan> example in the spec for how to solve this issue with css today, https://html.spec.whatwg.org/multipage/embedded-content.html#introduction-3:art-direction-3

zcorpan: This works - it's annoying to have to use CSS I guess, and you have to specify both width and height
... for each break point in the CSS.

gregwhitworth: I would prefer not to propagate layout based thing inside HTML, primarily because there
... are other reasons to specify aspect ratio than images.
... I would like to see it in CSS.
... I loved seeing a spec for this and also that you guys are tackling sizes.
... To me defining another aspect ratio is an additional constraint. I would prefer to set a width+height or
... a width or a height, and set aspect ratio = true|false and use max/min constraints to allow flex
... I'm not a huge fan of specifying an aspect ratio specifically.

jkt: The reason for adding sizes to the CSS spec was so that you could clearly define either size being absent

<gregwhitworth> https://jonathankingston.github.io/logical-sizing-properties/#propdef-aspect-ratio

jkt: e.g. missing height or width and that would be calculated from the aspect ratio

gregwhitworth: So you could technically still have a missing height or width without having the sizes shorthand

yoav: The point is that people are recommending fixed width and height to avoid reflows while the page is loading.
... They're not using image also so that's a particular case. I want people to be able to define images
... with aspect ratios so that they won't reflow when the image reloads - bouncing around is awful on mobile.
... You could say that css aspect ratio is enough, but still people advocate html width and height because
... it is easy to verify. I'm in favour of a CSS based proposal but we may need an equivalent in markup
... as an alternative to width and height.

gregwhitworth: That feels redundant.

yoav: It is in a way

dbaron: I wonder if we want something that does what HTML height and width were perhaps originally intended
... to do, which is indicate the expected image size but not override if the height and width are actually wrong.
... Also if you set one of the height and width properties then you only scale in one dimension rather than
... in both according to the aspect ratio.

<zcorpan> inside the srcset microsyntax

<dbaron> saying that perhaps we could have something (maybe inside the srcset microsyntax, ugh), that could describe the intrinsice width/height/aspect-ratio of the resource being pointed to, but only be used while the image is loading, and not override the resource nor set any CSS properties

yoav: Currently it only sets width...

zcorpan: ... but it has no effect until the image is loaded.

yoav: I guess with CSS it would also be applicable to non-image elements, which would be good enough from a loading perspective.

gregwhitworth: Setting dimensions of a box is usually CSS's job, that's why I want to do it there.
... Then we also get the benefits of doing it on other elements.
... It would be worth seeing a proposal on the html side also.
... The main use case is to see where these are being used.
... I know there are more use cases to understand not just in mobile.
... I'd like to set some actions.

<Zakim> dsinger, you wanted to talk about measurements, and about resolution

dsinger: It's rare to want non isotropic scaling of images so it does seem to make sense to be able to specify
... either height or width but not both.
... I would dispute that a pixel count is a visual size. All screens are not 72dpi anymore.
... I'm not comfortable with the idea that pixel counts indicate actual size.

yoav: The srcset sizes syntax in html tackles that and allows the browser to use the density of the entity as
... well as the dimensions and take both into account.
... The aspect ratio part is not necessarily tightly related to intrinsic dimensions.

gregwhitworth: That said if it does have an intrinsic aspect ratio then it should be used.
... you need to decide whether to care about that and then meet height or width constraints.
... We already have this problem with video and players.

cyril: I agree that it would be good to preserve aspect ratio of divs and other things not just images. But
... I understood the responsive images solutions were in HTML so that CSS is not needed to make requests,
... so how does it work if you solve it in CSS.

yoav: The responsive images syntax allows the width of the resource to be defined then you don't need the

<jkt> https://jonathankingston.github.io/logical-sizing-properties/demo/ << is a demo of the above proposal, if anyone has not seen that (it uses an image but I didn't initially)

yoav: height from the aspect ratio until layout actually happens. So you can start your request just based on
... the width info and maybe in the future height as well. We reserved this part for v2 of the spec.
... You should be able to load the resource based on one of those dimensions.
... I'm assuming all the resources in the srcset have the same aspect ratio.

zcorpan: We're trying to solve the problem of jumpy layout during page load. When the page starts to render
... at that point the stylesheets are loaded so you could make the request before layout.
... You can load the image knowing the layout before having the image. I still think it would be nice to have
... an attribute or something in HTML because when a CMS drops an image into a document they may
... not want to change the markup.

yoav: It's easier to verify and therefore some performance projects are using HTML.

dsinger: MPEG recently standardised a new image file format that may be related. It's codec neutral.
... The concern was about expressing how to package an image collection together and specifying which
... to use for printing, or non destructive edits etc. This is called the High Efficiency Image File Format.
... This could be related to the same use case.

<zcorpan> https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format

yoav: That's extremely interesting. Does it say in the headers the different image resolutions?

<cyril> https://github.com/nokiatech/heif

dsinger: Yes.

gregwhitworth: That's awesome

dsinger: Images are stored mp4 style with a structure at the beginning with byte range offsets etc. and you

<cyril> you can inspect examle images with http://download.tsi.telecom-paristech.fr/gpac/mp4box.js/filereader.html

dsinger: could put mp4 tracks in the same file if you wanted. You would want to profile it down for some scenarios.

yoav: Can you have low and higher resolutions compounded with scalable coding? And cropping?

dsinger: Yes to both.

cyril: I have examples of scalable in art directions.

gregwhitworth: Have you spoken with browser vendors and about OS support etc?

dsinger: Not yet but I'm happy to do tutorials and chat more with people.

<scribe> ACTION: gregwhitworth Get use case data on basic elements that would need to preserve aspect ratio [recorded in http://www.w3.org/2016/09/21-respimg-minutes.html#action01]

<trackbot> Sorry, but no Tracker is associated with this channel.

<scribe> ACTION: yoav Get the loading use case [recorded in http://www.w3.org/2016/09/21-respimg-minutes.html#action02]

<trackbot> Sorry, but no Tracker is associated with this channel.

<scribe> scribenick: gregwhitworth

h descriptor

<scribe> scribenick: gregwhitworth

yoav: currently you can't state the height via height restrictions
... the current syntax doesn't allow this
... do we want to discuss the various options we had

zcorpan: I think the syntax is not the important part, but we need to figure out what people are trying to do with height based images
... we need usecases
... once we understand that we can get a syntax

<scribe> ACTION: zcorpan with the getting height restricted use cases [recorded in http://www.w3.org/2016/09/21-respimg-minutes.html#action03]

<trackbot> Sorry, but no Tracker is associated with this channel.

nigel: you mentioned earlier about the video player
... the player can have one size, and the video that's playing might have a different aspect ratio
... so you need to know the reference area
... specifiying a box with an AR and then positioning a box inside of that
... you all remember 4:3 tv and you are watching a 16:9 video, do you scale or do you center cut
... one you're preserving the AR but you're cropping it

zcorpan: there is a property in CSS called object-fit where you can control this
... the default for video is object-fit: contain
... and images are set to preserve AR
... however there is no way of knowing how big the rendered image is if you preserve AR
... nah, you could...
... there are properties to get the width/height of the resource once it's loaded
... you do know the size of it
... you can render things on top of it, but you would need to use other things than CSS to accomplish it

nigel: that's interesting. That's the point of it, we need to work out what area you have. Sure, I guess we can do that in JS, but it may be worth looking into whether we want to somehow solve that before loading

zcorpan: anything else

yoav: what level of effort do you want
... to help out

jkt: I'm willing to look into getting use cases as well
... other than that, I can work with the syntax but that's not something we need to do early on

yoav: cool

<zcorpan> 👍

cyril: what's the responsiveness in the video element

zcorpan: it's in there, but implementers aren't jumping for it, and haven't gotten a lot of feedback from devs for it either
... you can put a high res in for a poster

yoav: do you see a use case for an art directed poster

cyril: could you put picture inside as the poster?
... any reason why the video element is not able to play images

yoav: I would ask the opposite question, why can't you play the video in the image?

<yoav> An early prototype I wrote down in 2013 for something like HEIF: https://www.smashingmagazine.com/2013/09/responsive-image-container/

zcorpan: on a theoretical level, it's possible but on an architectural level it's hard

<yoav> ^^ dsinger cyril

zcorpan: there is an interest I believe

yoav: animated gifs are a good use case, we should get rid of this

???: one thing that struck me is that this may be a problem for a11y if you put a video inside of image

???/kirkwood

- DRAFT -

Responsive Images breakout session TPAC 2016

21 Sep 2016

Attendees

Contents

h descriptor

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output