<scribe> scribenick: zcorpan

[presentation]

cyril: container format used heavily for video, now proposed for images
... HEIF
... lots of images on a web page, on average 30 per page
... small in resolution
... for icons
... constraints cna be very different compared to video
... images need to be decoded very fast
... footprint is a problem
... small footprint for video decoder
... gpu for video
... goal is to try to understand the consequences for browsers, content providers, users
... overlap and differences
... not everyone is familiar, want to give more details
... ISOBMFF
... "boxes"
... MSE-type video "moov" "moof"
... for images "meta" box
... freely available
... link to spec
... more recent spec defines hwo to use the "meta" box to store props for th eimage
... new type of track, "pict" for animations
... how to store thumbnails
... something channels
... codec agnostic
... annexes how to store HEVC, AVC, JPEG, but can use any codec

<markw> ISO BMFF: https://standards.iso.org/ittf/PubliclyAvailableStandards/c068960_ISO_IEC_14496-12_2015.zip

pal: freely available? Rf?

cyril: the spec is freely available.
... in terms of royalties, the spec has been available for 20 years...

pal: ISO patent database...

cyril: doesn't mean naything

<markw> MPEG Image File Format: https://standards.iso.org/ittf/PubliclyAvailableStandards/c066067_ISO_IEC_23008-12_2017.zip

cyril: if someone impl it, and has been sued, that's relevant
... large spec with lots of tech, not all of it is useful
... MIAF
... meta box is required
... for specifying base image
... even if you have a sequence of images
... e.g. middle image is representative, used for thumbnail
... media profiles
... brands, special profiles
... progressively loaded
... burst of images
... spec is not freely downloadable
... can pay to get it
... AVIF
... hosted on github
... freely available
... is based on MIAF
... uses AV1 codec
... transparency
... two media profiles
... baseline, advanced
... main and high
... reference software
... C impl, portable
... encode and decode
... including alpha
... the support, HEIF-based images, in browsers is ... currently limited
... does apple support HEIF?

OS supports it but not web-exposed

JakeA: you can encode but not decode in Safari macOS

cyril: other browsers, dunno?

mstange: firefox doesn't support

cyril: chrome bug...

annevk: we're looking into it
... open issues with AVIF

cyril: open questions... is it reasonbale to assume that if a video codec is supported, will the image codec be too?

annevk: once we support both, sure

LOL

cyril: don't want to use multiple decoders

??: path for gecko, between image and video totally different

??: but can change

mstange: in the past the answer has been no. for webp and webm
... we don't want the web platform to rely on (something) image formats

yoav: it may be technically possible but there are other considerations

cyril: would media capabilities ?

mstange: i don't see why not

yoav: accept headers...

ChrisLilley: accept were expected to have that role, but failed

yoav: ok but sounds like a reasonable use case

markw: possible that the device can support both
... which should you choose

yoav: nowehere near that spec but sounds reasonable use case

cyril: AVIF format, supported in video
... how likely is it to get uniform support
... delta to achieve parsers

??: common demuxing and decoding video and images

<Yves> https://tools.ietf.org/id/draft-ietf-httpbis-variants-00.html is relevant to the 'accept' header discussion

annevk: for image loading there's some byte sniffing
... some challenge
... don't want to add new sniffing
... strict mime types for new formats

yoav: strict mime types seems reasonable

annevk: more than reasoanble

ChrisLilley: do you say ??????

??: only to determine the container, not the codec

zcorpan: for top level navigation, sniffing to figure out type of document

cyril: the way HEIF define mime types
... one for static images
... one for image sequences

??: discussion a while back with w3c on media whether you listen to the server mime type

??: can only determine by sniffing, not by what the server says

yoav: servers lie
... but if we add new stuff, we can break it

??: for firefox/safari (some format) , need sniffing

??: issue of trust

??: already don't trust mime type

yoav: we should trust and verify
... if they lie, nothing runs
... people should figure it out and fix it

annevk: we require the mime type to be correct, for webvtt, javascript modules...

??: video in particular

??: what makes major difference is the container

annevk: video element does a request

??: media source, you specify what you're going to be fed

annevk: you pass into an API

??: when you create your SourceBuffer

jya: (missed)

yoav: the server mime type is irrelevant

JakeA: sniffing is fine if you have full access to (something?)

annevk: sniffing i'm talking about if you navigate to a random url
... or if we load cross-origin resources
... safe-listing media resources
... others we'd block
... i'd like to restrict the sniffing. not keep adding

cyril: the first bytes of the file contains the type of file
... the mime type tells you that

jya: video src = bla

yoav: server sends a mime type
... it needs to correspond to the content
... otherwise we need to add to the sniffing

JakeA: if ... CORS, would just work

annevk: maybe

cyril: specific mime type
... issue is "why 2 mime types"
... either you change the spec, if the static mime type is given but the content is actually a sequence
... rendered as the static
... or get rid of the second mime type

ChrisLilley: given it has htumbnail
... show static image
... may want to do that deliberately

yoav: are there scenarios where static will be supported but sequence not?
... if so we need 2 mime types

cyril: when i talk to media folks at google, they don't know

mstange: animations

jya: does the user want to be annoyed with sound

yoav: can be scenarios where people will have multiple ... dunno

cyril: no requirement in the spec to treat the mime types differently

annevk: problem is the mime type has no impact on processing model, will end up the same

yoav: <picture>, preload
... if i specify a mime type that is not supported
... it will be skipped

JakeA: if you specify jpeg but return png, it works

yoav: if you support only one, you want Accept to reflect that (hopefully), and type="" should work

annevk: why would you switch between those two mime types?

yoav: may want to fall back to other format

cyril: let's say you intend to support an api
... would you support only static?

annevk: moz would support both

yoav: problem with apng

ChrisLilley: PNG group said "you should register video/png" but they didn't want to do that

yoav: question: does anyone want to ship one but not the other?

ChrisLilley: use case for static or animated?

yoav: for goth type attr and accept header
... if UA supports the format for both
... same signal
... no way for the web dev to distinguish based on mime type

cyril: third bullet point
... consistent support between <img> and <video>
... will we see ... will the overlap , unified model, convergance between the two elements?

yoav: wouldn't expect it

cyril: difference is subtle

yoav: there are multiple image formats that have animation, gif, webp
... conv with our media team
... needs convincing that the complexity is worth it
... for video codec in <img>
... if you have data
... byte-savings e.g.
... i can cummunicate it
... ability to sandbox

cyril: safari team?

smfr: if you think about... yes
... when we display an mp4 file
... use hardward decoder
... decode one frame at a time
... we don't need to keep them all around
... with animated gif
... may not want to decode each frame every loop
... savings

JakeA: comparison is <img src=video> vs <video muted>

smfr: can't use video as background image

yoav: problem of not everyone can control and modify their markup
... i argued for that internally

ChrisLilley: ...lost?

yoav: well yes
... animated webp is assumed to be enough
... for background-image
... lack of control for markup...
... otherwise <video loop muted> would work

smfr: arguing power and memory...

yoav: so... based on gpu decoding?

smfr: yes

yoav: argument was that images and animated gifs tend to come in lots vs video is only one
... you could have multiple animated gifs
... i will convey your skeptisism ericc/ ^

Richard: gpu or cpu, does it matter?
... <img> tag, or if they merge

mstange: video where img is supported...
... img is supported in many places

smfr: we support video everywhere

cyril: (missed)

yoav: bring data

annevk: why do you want that?

cyril: when only some browser support
... same format, same codec

jya: video format was just so much better at compressing compared to gif
... you see on reddit, 100MB gif
... is it relevant
... reason is it compresses better
... now images compress well, is it relevant?

annevk: also special cases

jya: would you like your images to be (missed)

cyril: depends on format?
... can follow image pipeline

jya: webp, use image decoder
... different objective
... want to be able to pause
... can't do that with video
... interrupts page loading
... purely technical

cyril: can still provide same decoding features

jya: ok if you have video that is gpu decoded, may not have same features in cpu

smfr: fundamental differences between img and video
... color profiles different
... not sure those would converge
... might be a while before those converge
... netflix is interested in mixing the two

cyril: yes

jya: if it were easy, everyone would have done it

cyril: some have done it

jya: one format

cyril: adjurn

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/YY/markw/
Succeeded: s/YY/markw/
Succeeded: s/??/jya/
Succeeded: s/webp/apng/
Succeeded: s/smfr/ericc/ ^/
Present: ChrisLilley zcorpan smfr gkatsev JakeA mstange MasakazuKitahara domfarolino markw yoav bdekoz bkardell_
Found ScribeNick: zcorpan
Inferring Scribes: zcorpan

WARNING: No "Topic:" lines found.


WARNING: No meeting chair found!
You should specify the meeting chair like this:
<dbooth> Chair: dbooth


WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: No "Topic: ..." lines found!  
Resulting HTML may have an empty (invalid) <ol>...</ol>.

Explanation: "Topic: ..." lines are used to indicate the start of 
new discussion topics or agenda items, such as:
<dbooth> Topic: Review of Amy's report


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)