This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 27542 - [MSE] Clarify that 'Initialization Segment Received' is the first place that changes ready state to HAVE_METADATA
Summary: [MSE] Clarify that 'Initialization Segment Received' is the first place that ...
Status: RESOLVED FIXED
Alias: None
Product: HTML WG
Classification: Unclassified
Component: Media Source Extensions (show other bugs)
Version: unspecified
Hardware: PC Linux
: P2 normal
Target Milestone: CR
Assignee: Aaron Colwell
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-12-09 05:28 UTC by Bartlomiej Gajda
Modified: 2014-12-11 18:48 UTC (History)
4 users (show)

See Also:


Attachments

Description Bartlomiej Gajda 2014-12-09 05:28:07 UTC
Currently we have: 

3.5.7 Initialization Segment Received
" 6. If the HTMLMediaElement.readyState attribute is HAVE_NOTHING, then run the following steps:
1  If one or more objects in sourceBuffers have first initialization segment flag set to false, then abort these steps.
2. Set the HTMLMediaElement.readyState attribute to HAVE_METADATA."

but also in 2.4.4 SourceBuffer Monitoring

"If buffered for all objects in activeSourceBuffers do not contain TimeRanges for the current playback position:

1.  Set the HTMLMediaElement.readyState attribute to HAVE_METADATA.
2. If this is the first transition to HAVE_METADATA, then queue a task to fire a simple event named loadedmetadata at the media element.
3. Abort these steps.
"

also SourceBuffer Monitoring states:
" The following steps are periodically run during *playback*"
but also :
"appending new segments and changes to activeSourceBuffers also cause these steps to run because they affect the conditions that trigger state transitions."

which could be interpreted as : run monitoring algorithm after any append.

Now what can happen is following:

Scenario 1:

1. create SB with audio
2. create SB with video
active sourcebuffers : none
3. append init segment to SB with audio
active sourcebuffers : SB with audio
4. monitoring algorithm can fire (according to spec) and changes ready state to HAVE_METADATA, as active SB (with audio) has 0 buffered.

5. monitoring algorithm schedules event `loadedmetadata` 
6. javascript receives and asks for HTMLMediaElement width/height, 
=> Error : there is no yet, as there was no init segment for video
7. append init segment to SB with video

Scenario 2:
it could be also written as:

add event listener on `loadedmetadata`: append media segments to video
add event listener on `updateend` : append init segment to SB with video
append init segment to audio.

which would make following order : append init to SB1, loadedmetadata, append media sample to SB2, End of stream.

so efectively media segments for video would be discarded and End of stream shall be called according to :
"3.5.1  If the append state equals PARSING_MEDIA_SEGMENT, then run the following steps:
6.1 If the first initialization segment flag is false, then run the end of stream algorithm with the error parameter set to "decode" and abort this algorithm."

--

What I believe is expected to happen is that `loadedmetadata` is fired after both SB with audio and SB with video have received initialization segments. (so complete metadata was loaded)

Proposed changed is to add at beggining of monitoring algorithm:

" If one or more objects in sourceBuffers have first initialization segment flag set to false, then abort these steps."

exactly the same as in 3.5.7 6.1, which would make it crystal clear.

Or You can state at then end of 3.5.7 6.3 that this is the place when You can start monitoring.
Comment 1 Aaron Colwell 2014-12-11 01:01:36 UTC
(In reply to Bartlomiej Gajda from comment #0)
> Currently we have: 
> 
> 3.5.7 Initialization Segment Received
> " 6. If the HTMLMediaElement.readyState attribute is HAVE_NOTHING, then run
> the following steps:
> 1  If one or more objects in sourceBuffers have first initialization segment
> flag set to false, then abort these steps.
> 2. Set the HTMLMediaElement.readyState attribute to HAVE_METADATA."
> 
> but also in 2.4.4 SourceBuffer Monitoring
> 
> "If buffered for all objects in activeSourceBuffers do not contain
> TimeRanges for the current playback position:
> 
> 1.  Set the HTMLMediaElement.readyState attribute to HAVE_METADATA.
> 2. If this is the first transition to HAVE_METADATA, then queue a task to
> fire a simple event named loadedmetadata at the media element.
> 3. Abort these steps.
> "
> 
> also SourceBuffer Monitoring states:
> " The following steps are periodically run during *playback*"
> but also :
> "appending new segments and changes to activeSourceBuffers also cause these
> steps to run because they affect the conditions that trigger state
> transitions."
> 
> which could be interpreted as : run monitoring algorithm after any append.

The "Appending new segments and" part of that sentence should probably be dropped since state transitions are triggered at the bottom of the "coded frame processing algorithm" and running the "SourceBuffer monitoring" algorithm after that, I believe, should be a noop.

> 
> Now what can happen is following:
> 
> Scenario 1:
> 
> 1. create SB with audio
> 2. create SB with video
> active sourcebuffers : none
> 3. append init segment to SB with audio
> active sourcebuffers : SB with audio
> 4. monitoring algorithm can fire (according to spec) and changes ready state
> to HAVE_METADATA, as active SB (with audio) has 0 buffered.
> 
> 5. monitoring algorithm schedules event `loadedmetadata` 
> 6. javascript receives and asks for HTMLMediaElement width/height, 
> => Error : there is no yet, as there was no init segment for video
> 7. append init segment to SB with video
> 
> Scenario 2:
> it could be also written as:
> 
> add event listener on `loadedmetadata`: append media segments to video
> add event listener on `updateend` : append init segment to SB with video
> append init segment to audio.
> 
> which would make following order : append init to SB1, loadedmetadata,
> append media sample to SB2, End of stream.
> 
> so efectively media segments for video would be discarded and End of stream
> shall be called according to :
> "3.5.1  If the append state equals PARSING_MEDIA_SEGMENT, then run the
> following steps:
> 6.1 If the first initialization segment flag is false, then run the end of
> stream algorithm with the error parameter set to "decode" and abort this
> algorithm."
> 
> --
> 
> What I believe is expected to happen is that `loadedmetadata` is fired after
> both SB with audio and SB with video have received initialization segments.
> (so complete metadata was loaded)
> 
> Proposed changed is to add at beggining of monitoring algorithm:
> 
> " If one or more objects in sourceBuffers have first initialization segment
> flag set to false, then abort these steps."

How about this as an alternative solution? Make the first step in the monitoring algorithm, "If HTMLMediaElement.readyState equals HAVE_NOTHING, then abort these steps." I could then also drop the step that fires "loadedmetadata" because then it is no longer possible for the monitoring algorithm to be the first transition to HAVE_METADATA. 

I believe this will have the same effect as your suggestions, but more clearly conveys there is only one place that can trigger the initial transition to HAVE_METADATA and fire the loadedmetadata event.
Comment 2 Aaron Colwell 2014-12-11 01:25:57 UTC
Actually I just realized that removal of a SourceBuffer should potentially trigger a transition from HAVE_NOTHING -> HAVE_METADATA if the SourceBuffer we were waiting for was removed. I'll file a separate bug to address that situation.