Can't this be covered by http://www.whatwg.org/specs/web-apps/current-work/multipage/infrastructure.html#hardwareLimitations ?

(In reply to comment #1)
> Can't this be covered by
> http://www.whatwg.org/specs/web-apps/current-work/multipage/infrastructure.
> html#hardwareLimitations ?

Works for me. Thanks Simon.

I wanted to get more input about whether or not a limit should be standardized. Having a large line cue setting not only doesn't make sense IMO, because it's going to be displayed off the video, but if not limited it also increases the time complexity of the step algorithm. I'm going to limit Mozilla's implementation to 1000 if there's no plan to standardize this.

Do you mean to clamp the line to [-1000, 1000], and if so at what point?

It's true that a cue at line 1000000 would result in a lot of stepping, but is that any more of a problem than a script that (say) increments a counter 1000000 times?

I guess one approach would be to clamp line to +- ceil("max dimension" / "step"), which should ensure that the first line is at least partially visible... or is it just outside the rendering area?

Of course, this doesn't work if you want to do the clamping at an earlier stage to save memory in your structs.

I've had similar thoughts to solving it like that Philip. To build on that, I can see two options for solving this:

1: We clamp it to some reasonable value, that we can see a use case for, as soon as it is parsed or is set via the DOM interface.

2: We can alter the positioning algorithm to skip all the lines between the specified position and the 'fraction of a line' that it would be at if the algorithm was stepped through. For example if we have a line at 1 000 000 and the normal algorithm would have it into the caption overlay with some kind of a percentage outside of the overlay. Then we just calculate that, move it there, and proceed with the normal algorithm. Alternatively, we could not worry if there would be an overlap if the normal step algorithm was done and just move it to the last line in the overlay and proceed with the algorithm.

I'm partial to number one as it's fewer alterations to the spec, fairly simple, and I don't really see the practical use case for large line numbers.

(In reply to Rick Eyre from comment #5)
> I've had similar thoughts to solving it like that Philip. To build on that,
> I can see two options for solving this:
> 
> 1: We clamp it to some reasonable value, that we can see a use case for, as
> soon as it is parsed or is set via the DOM interface.

I suppose my main problem with this is that the number we pick is going to be completely arbitrary. If the clamping is visible in the DOM API, then it's easier to have stuff come to depend on the numbers we pick.

> 2: We can alter the positioning algorithm to skip all the lines between the
> specified position and the 'fraction of a line' that it would be at if the
> algorithm was stepped through. For example if we have a line at 1 000 000
> and the normal algorithm would have it into the caption overlay with some
> kind of a percentage outside of the overlay. Then we just calculate that,
> move it there, and proceed with the normal algorithm. Alternatively, we
> could not worry if there would be an overlap if the normal step algorithm
> was done and just move it to the last line in the overlay and proceed with
> the algorithm.

Do you mean that while the cue is entirely outside the video it wouldn't check for overlap at each step, so that one could easily optimize the stepping to move it directly to the first step at which it intersects the video?

Right now I can't actually come up with a reason one couldn't already do that... if there are no cases where overlapping cues outside of the video would affect where the cue ultimately ends up, then it seems like we don't have a problem, right?

> I'm partial to number one as it's fewer alterations to the spec, fairly
> simple, and I don't really see the practical use case for large line numbers.

I agree, there isn't really a use case, it's just a question of how to make this

(In reply to Philip Jägenstedt from comment #6)

> I suppose my main problem with this is that the number we pick is going to
> be completely arbitrary. If the clamping is visible in the DOM API, then
> it's easier to have stuff come to depend on the numbers we pick.

Yeah, I was thinking this as well. That's the disadvantage of this way.

> Do you mean that while the cue is entirely outside the video it wouldn't
> check for overlap at each step, so that one could easily optimize the
> stepping to move it directly to the first step at which it intersects the
> video?

Exactly.

> Right now I can't actually come up with a reason one couldn't already do
> that... if there are no cases where overlapping cues outside of the video
> would affect where the cue ultimately ends up, then it seems like we don't
> have a problem, right?

Yeah I agree, however, my issue isn't so much with if this is possible, I can easily code that. It's whether or not we should standardize this process--whichever method we choose--or if this is even a problem people want to solve?

(In reply to Rick Eyre from comment #7)
> Yeah I agree, however, my issue isn't so much with if this is possible, I
> can easily code that. It's whether or not we should standardize this
> process--whichever method we choose--or if this is even a problem people
> want to solve?

If doing it isn't distinguishable from not doing it then it doesn't really need to be specified normatively. On the other hand, having it in the spec ensures that we don't later change the spec in a way to make that optimization impossible.

Silvia, I'd be fine with spec'ing this, do you have an opinion?

(In reply to Philip Jägenstedt from comment #8)
> (In reply to Rick Eyre from comment #7)
> > Yeah I agree, however, my issue isn't so much with if this is possible, I
> > can easily code that. It's whether or not we should standardize this
> > process--whichever method we choose--or if this is even a problem people
> > want to solve?
> 
> If doing it isn't distinguishable from not doing it then it doesn't really
> need to be specified normatively. On the other hand, having it in the spec
> ensures that we don't later change the spec in a way to make that
> optimization impossible.
> 
> Silvia, I'd be fine with spec'ing this, do you have an opinion?

I prefer spec'ing less than more if it doesn't make a difference, because in future we may have an actual use case and this way we restrict our future choices. For example, with 4k coming, we will have more lines to play with - and who knows what comes after that!

How about we just leave this as a browser quality of implementation issue and you can implement whatever you think makes sense?

(In reply to Silvia Pfeiffer from comment #9)
> (In reply to Philip Jägenstedt from comment #8)
> > (In reply to Rick Eyre from comment #7)
> > > Yeah I agree, however, my issue isn't so much with if this is possible, I
> > > can easily code that. It's whether or not we should standardize this
> > > process--whichever method we choose--or if this is even a problem people
> > > want to solve?
> > 
> > If doing it isn't distinguishable from not doing it then it doesn't really
> > need to be specified normatively. On the other hand, having it in the spec
> > ensures that we don't later change the spec in a way to make that
> > optimization impossible.
> > 
> > Silvia, I'd be fine with spec'ing this, do you have an opinion?
> 
> I prefer spec'ing less than more if it doesn't make a difference, because in
> future we may have an actual use case and this way we restrict our future
> choices. For example, with 4k coming, we will have more lines to play with -
> and who knows what comes after that!

Clamping to a fixed range was only one of the suggestions, any of the others would work with any video size and line height.

> How about we just leave this as a browser quality of implementation issue
> and you can implement whatever you think makes sense?

You certainly can't implement something that has any observable side effects, but I suppose we can just be careful when updating the algorithm to not specify something that would make the optimization impossible.

Rick, can you double-check to see if it's really possible to solve the line:1000000 problem with an optimization that's indistinguishable from actually stepping?

(In reply to Philip Jägenstedt from comment #10)

> You certainly can't implement something that has any observable side
> effects, but I suppose we can just be careful when updating the algorithm to
> not specify something that would make the optimization impossible.
> 
> Rick, can you double-check to see if it's really possible to solve the
> line:1000000 problem with an optimization that's indistinguishable from
> actually stepping?

I can definitely look into that and get back to you.

BTW: If you both think it's worth a change in the spec, I won't object.

(In reply to Silvia Pfeiffer from comment #9)
> I prefer spec'ing less than more if it doesn't make a difference, because in
> future we may have an actual use case and this way we restrict our future
> choices. For example, with 4k coming, we will have more lines to play with -
> and who knows what comes after that!

It doesn't follow that 4k gives you more lines. Assuming the user interaction model doesn't change, the number of lines you have will be the same.

Philip, I was able to optimize the algorithm without any visible side effects. It's actually a pretty easy fix too. I'm not sure if you'd like to know the details or not. I'm fine with leaving this out of the spec for the reasons that Silvia and you have listed.

(In reply to Rick Eyre from comment #14)
> Philip, I was able to optimize the algorithm without any visible side
> effects. It's actually a pretty easy fix too. I'm not sure if you'd like to
> know the details or not.

Do share - might it be something we should put in the spec so it's compatible across browsers?

Sure.

So the method I ended up going with is to calculate the minimum number of steps the cue would have to make to end up completely outside the video's rendering area and then compare that to the number of steps we've been asked to make via the line value. If the specified number of steps is greater then the minimum number of steps then only move the minimum number of steps. After that the algorithm runs as normal.

I think it's an easy enough optimization to put into the spec. It certainly makes more sense to move the cue to just outside the video rendering area if that has the same behaviour then moving it to n steps outside the rendering area. Consider if someone wrote a WebVTT file where each cue had a line value of 1000000000000, that would have some serious impact on the performance of the web page.

I'd be fine with spec'ing that behavior.

If you'd like you can see the code that does the optimization for Firefox here: https://github.com/mozilla/vtt.js/blob/master/lib/vtt.js#L1003

(In reply to Philip Jägenstedt from comment #17)
> I'd be fine with spec'ing that behavior.

Do go ahead. I like it, too.