W3C

HTML Media Task Force Teleconference

26 Mar 2013

Agenda

See also: IRC log

Attendees

Present
glenn, Bin, ddorwin, Plh, joesteele, markw, BobLund, MartinSoukup, adrianba, pladd, johnsim, Aaron_Colwell, pal, jdsmith, Henri
Regrets
Paul
Chair
Adrian
Scribe
plh

Contents


previous minutes

http://www.w3.org/2013/03/12-html-media-minutes.html

minutes are approved

Action items

action-10?

<trackbot> ACTION-10 -- Adrian Bateman to discuss bug 19208 with johnsim -- due 2013-04-08 -- OPEN

<trackbot> Keymessage event not needed when Key System already has Key

https://www.w3.org/Bugs/Public/show_bug.cgi?id=19208

Adrian: John and I talked about it last week
... need a bit more time
... the proposal we'll make is related to a discussion we had a few weeks ago avbout the lifetime of a session
... there are some suggestions we want to make about session management
... different from what the spec is suggesting at the momnet

Adrian: so we'll need to come back on this

Joe: I did update a related bug...
... [never mind]

Baseline documents

Adrian: those are just links
... no update since the previous discussion
... not sure where the Chairs are at for the moment
... waiting on feeback from Paul

plh: I don't know the status from the Chairs point of view either

Progression to FPWD

Adrian: see above

Discussion of outstanding bugs

http://tinyurl.com/7tfambo

<adrianba> http://lists.w3.org/Archives/Public/public-html-media/2013Mar/0070.html

Adrian: David made a proposal

SessionID may be assigned asynchronously in MediaKeys.createSession

Mark: we already have a mention that the sessionID may be assigned by the CDM
... we want it to be async
... so the sessionId might not be assigned when the MediaKeys.createSession returns
... you'll get it on the first event instead

<MartinSoukup> +1 to require assignment of session ID prior to event firing

?: is there proposed languagejohnsim

Mark: not yet

Martin: I understand that the proposal that the sessionId might be assigned before the first event

Mark: that's right

Adrian: this might be related that John and I have an action on
... if you already have the information about the keys in the init state, that might have an impact on when the sessionId gets set

Mark: right, you want to do that async. the spec implies otherwise at the moment

David: if we do go to reusing sessions, then we would already have the info.

Adrian: should there be a serie of events that fire before the progress event
... that reuse MSE
... we could have an event that fires that says that the session is open and running
... if Mark proposal is before the first event, that would mesh with any proposal we would make about progress event

John: the important part is that we make it async
... we can figure out the relationship later

EME should be explicit about its relationship with Web Platform APIs that allow video frames and audio samples to be extracted from an HTMLMediaElement

Henri: it seems there are at least two classes of CDMs. one that gives the pixel data back to the browser, and one that gives it to the GPU
... for the later, painting on canvas wouldn't work
... for the former, it seems we wouldn't want it to work either
... we need to be more explicit about the expected behavior
... ie we should cover both cases
... if you're writing JS, you should be able to know the expected by looking at the spec

Joe: agreed. I propose that we never pass back the data to the JS layer

Joe: that would introduce too much complexity to the CDM layer
... so let's not have that use case

Glenn: what API currently exist to allow video frame access?

Henri: pixels can be extracted using canvas. not quite sure about the Web Audio API
... there are restrictions on canvas due to same origin policy

Henri: it seems simpler to say that painting to a canvas with a non-null media key fails in some fashion

<acolwell> MediaElementAudioSourceNode allows audio samples to be exported to JS

Glenn: you're referring to createPattern?

<adrianba> drawImage -> http://www.w3.org/html/wg/drafts/2dcontext/html5_canvas/#dom-context-2d-drawimage

<pal> drawImage(document.getElementById("video1"),5,5,260,120,005)

David: re complexity in passing back the frames from the CDM. It depends on the CDM. If it's hardware backed, that's not possible. Otherwise, it would be more work to prevent it from happening. so not sure that prevention would be the easy solution

Henry: my reading is that it wouldn't be possible in webkit

Henri: I can live with a resolution that depends on the kind of CDM
... just want to make the spec is clear on the behavior

Pierre-Anthony: Is there any value in exposing to the app whether the CDM can or cannot expose the data back

Adrian: is it sufficient for the EME spec to simply that the media frame may not be available to other Web platform APIs that consume them

Adrian: it may be possible to do some operations
... like transforms
... would it be possible to say that some operations may not be possible, or should we be more explicit

Henri: there is to be a well defined failure mode

Adrian: possibly a no-op

Henri: if there are CDMs passing the data back to the browser, so is it ok for extensions to access it?

Plh: captioning tracks need to be considered as well

Mark: it would be useful to have a note on the failure mode(s). those methods can fail for other reasons
... the CDM might want to restrict what the browser can do

Joe: re the caption question. we exclude the use case of encrypted captions. the captions would always be available.
... if the app needs some logic based on the type of CDM, this is going to be a huge burden.
... if we allow any type of discovery mechanism for the type of restrictions, this will get complex. We could add one type of error
... but more would be complex
... if we don't allow to come back, folks would have to recompile the browser but wouldn't be able to access the CDM. so not an issue.

Glenn: (1) canvas 2d drawImage/createPattern has language about lack of decoability of image (frame), so may want to suggest expanding decodability to accessibility; (2) regarding caption/subtitles, text tracks can also include other metadata, some of which may need to remain encrypted so should be particular about which "kind" of text track should be exposed to JS in case a CDM is involved

Bob: I think captioning is a different issue than the video frame one. I would imagine captioning would be dealt with in the same way as text tracks in general.

Henri: what I hear about text tracks is scary.
... I would rather see the approach that captions are not part of the DRM sphere

Bob: Henri's point is interesting. UA is responsible for rendering of captions.

Bob: we need to think more about the case

Adrian: two issues:
... what should happen to video and audio data? two proposals: (1) until we find a use case, no access. (2) leave it up to the CDM to impose the restriction or not.
... captions are different. Either outside of EME scope, or we need to consider them specifically for the TextTrack API.

David: caption text tracks are outside the scope of EME
... no decoding etc.

Pierre-Anthony: if MSE and EME are supposed to work together, how would captions work?

David: different tracks at the moment and encryption is by tracks.

Pal: but MSE adds segments

?: it's multiplexed

Patrick: captions have requirements about sync with video frames
... and JS can't keep up for that

Glenn: 608/708 is embedded in the user data of the video stream, and will be encrypted along with the video, like MPEG-2 streams.

?: is that a use case for us? ie will any browser support that?

Glenn: well, I think it's possible from a functional perspective.

?: I don't know where the performance requirement is coming from...

Adrian: I'd like to see concrete proposals/bugs
... any volunteer?
... Joe? Mark?

Joe: I can try but don't have much bandwidth at the moment

Mark: at least we need to consider the case where the data may not be available and go from there

Pierre-Anthony: it is possibility that it is not available

Glenn: it's up to the APIs that give access to describe their own failure modes

?: the API needs to be able to tell if a failure happened or not

Mark: I'll write a proposal

<scribe> ACTION: Mark to write a proposal for the case where the data is not available to the JS [recorded in http://www.w3.org/2013/03/26-html-media-minutes.html#action01]

<trackbot> Created ACTION-11 - Write a proposal for the case where the data is not available to the JS [on Mark Watson - due 2013-04-02].

AOB?

none

Adrian: Paul will be back for the next meeting
... any volunteer to scribe?

Joe: I can

[adjourned]

Summary of Action Items

[NEW] ACTION: Mark to write a proposal for the case where the data is not available to the JS [recorded in http://www.w3.org/2013/03/26-html-media-minutes.html#action01]

Minutes formatted by David Booth's scribe.perl version 1.137 (CVS log)
$Date: 2013/03/26 19:58:02 $