ISSUE-199: timeExpression to video frame calculation is unclear in some cases

timeExpression to video frame calculation is unclear in some cases

State:
CLOSED
Product:
TTML 1.0
Raised by:
Mike Dolan
Opened on:
2013-01-09
Description:
When using timeBase="media", dropMode="noDrop", timeExpression of clock-time with frames, and frameRateMultiplier does not resolve to an integer, then it is ambiguous how to convert a timeExpression into video frames.

For example, with frameRate="24" and frameRateMultiplier="1000 1001", and using example timeExpressions, 00:00:41:23 and 00:00:42:00...

The users I am aware of treat the seconds as a real number of frames. For example, 00:00:41:23 is 41*24*1000/1001=983.0169... frames (not truncating or rounding to 983 and not 41*24=984); plus 23 frames = 1006.0169... frames. Similarly, 42 seconds is 42*24*1000/1001=1006.993... frames.

When syncing the text overlay to video, it is logical to synchronize to the first frame with a time that converts to frames (f) as follows: 0.000<=f<1.000 and so on throughout the media. Therefore, the two TTML example timeExpressions above resolve to the same video frame - 1006.

This non-intuitive - surely two times with a different "frame" value in the timeExpression (23 v 00) will resolve to different frames. But in this algorithm, they resolve to the same frame. This is odd, but this algorithm preserves the timeExpression values locked to the media time of the video (and audio).

There are other possible algorithms with differing properties of course. For example, one can treat the conversion of the HH:MM:SS to frames using integers, e.g. 41*24=984, plus 23 = 1007; and 42*24=1008. This results in a 1:1 mapping of tmeExpression to frames (more intuitive), but forces the timeExpression to diverge from real time (and the video and audio timebases).

TTML 1.0 is silent how to convert timeExpression values to frames in this scenario. This should be clarified for the above configuration scenario and using all the time options that are ambiguous.

Note that integer framerates or fraction (not frame) timeExpressions have obvious transforms into frames.
Related Actions Items:
No related actions
Related emails:
  1. {agenda} TTWG Meeting 29/5/2014 (from nigel.megitt@bbc.co.uk on 2014-05-28)
  2. Re: ISSUE-317 (IMSC should not require frame alignment): IMSC should not require frame alignment [TTML IMSC 1.0] (from nigel.megitt@bbc.co.uk on 2014-05-23)
  3. RE: ISSUE-317 (IMSC should not require frame alignment): IMSC should not require frame alignment [TTML IMSC 1.0] (from mdolan@newtbt.com on 2014-05-23)
  4. Re: ISSUE-317 (IMSC should not require frame alignment): IMSC should not require frame alignment [TTML IMSC 1.0] (from nigel.megitt@bbc.co.uk on 2014-05-23)
  5. Minutes - 2013-06-20 (from glenn@skynav.com on 2013-07-11)
  6. [minutes] Timed Text call June 20, 2013 (from plh@w3.org on 2013-06-27)
  7. Re: TTML Agenda for 20/06/13 -- ADVANCE NOTICE -- (from tai@irt.de on 2013-06-19)
  8. TTML Agenda for 20/06/13 -- update -- (from Sean.Hayes@microsoft.com on 2013-06-19)
  9. Re: TTML Agenda for 20/06/13 -- ADVANCE NOTICE -- (from glenn@skynav.com on 2013-06-19)
  10. Re: TTML Agenda for 20/06/13 -- ADVANCE NOTICE -- (from tai@irt.de on 2013-06-19)
  11. Re: TTML Agenda for 20/06/13 -- ADVANCE NOTICE -- (from glenn@skynav.com on 2013-06-18)
  12. Re: TTML Agenda for 20/06/13 -- ADVANCE NOTICE -- (from glenn@skynav.com on 2013-06-18)
  13. TTML Agenda for 20/06/13 -- ADVANCE NOTICE -- (from Sean.Hayes@microsoft.com on 2013-06-17)
  14. [ttml10se] add appendix on time expression semantics (from glenn@skynav.com on 2013-04-09)
  15. TTWG Meeting Minutes Jan 31, 2013 (from momartin@microsoft.com on 2013-02-06)
  16. TTML Agenda for 31/01/13 (from Sean.Hayes@microsoft.com on 2013-01-31)
  17. SDP-US and ISSUE-199 [was RE: TTWG Meeting Minutes, Jan 24, 2013] (from mdolan@newtbt.com on 2013-01-24)
  18. TTWG Meeting Minutes, Jan 24, 2013 (from momartin@microsoft.com on 2013-01-24)
  19. RE: TTML Agenda for 25/01/13 (from momartin@microsoft.com on 2013-01-24)
  20. TTML Agenda for 25/01/13 (from Sean.Hayes@microsoft.com on 2013-01-24)
  21. defer ISSUE-199 (from mdolan@newtbt.com on 2013-01-22)
  22. TTWG Meeting Minutes Jan 17, 2013 (from momartin@microsoft.com on 2013-01-17)
  23. RE: TTML Agenda for 17/01/13 (from mdolan@newtbt.com on 2013-01-17)
  24. TTML Agenda for 17/01/13 (from Sean.Hayes@microsoft.com on 2013-01-17)
  25. TTWG Meeting Minutes Jan 10, 2013 (from momartin@microsoft.com on 2013-01-10)
  26. RE: TTML Agenda for 10/01/13 (from Sean.Hayes@microsoft.com on 2013-01-10)
  27. ISSUE-199: timeExpression to video frame calculation is unclear in some cases [TTML 1.0] (from sysbot+tracker@w3.org on 2013-01-09)

Related notes:

See new Appendix N in current ED [1].

[1] https://dvcs.w3.org/hg/ttml/raw-file/default/ttml10/spec/ttaf1-dfxp.html

Glenn Adams, 12 May 2013, 20:52:59

[plh]: https://dvcs.w3.org/hg/ttml/raw-file/tip/ttml10/spec/ttaf1-dfxp.html#time-expression-semantics

20 Jun 2013, 14:54:46

Changelog:

Created issue 'timeExpression to video frame calculation is unclear in some cases' nickname owned by Mike Dolan on product TTML 1.0, description 'When using timeBase="media", dropMode="noDrop", timeExpression of clock-time with frames, and frameRateMultiplier does not resolve to an integer, then it is ambiguous how to convert a timeExpression into video frames.

For example, with frameRate="24" and frameRateMultiplier="1000 1001", and using example timeExpressions, 00:00:41:23 and 00:00:42:00...

The users I am aware of treat the seconds as a real number of frames. For example, 00:00:41:23 is 41*24*1000/1001=983.0169... frames (not truncating or rounding to 983 and not 41*24=984); plus 23 frames = 1006.0169... frames. Similarly, 42 seconds is 42*24*1000/1001=1006.993... frames.

When syncing the text overlay to video, it is logical to synchronize to the first frame with a time that converts to frames (f) as follows: 0.000<=f<1.000 and so on throughout the media. Therefore, the two TTML example timeExpressions above resolve to the same video frame - 1006.

This non-intuitive - surely two times with a different "frame" value in the timeExpression (23 v 00) will resolve to different frames. But in this algorithm, they resolve to the same frame. This is odd, but this algorithm preserves the timeExpression values locked to the media time of the video (and audio).

There are other possible algorithms with differing properties of course. For example, one can treat the conversion of the HH:MM:SS to frames using integers, e.g. 41*24=984, plus 23 = 1007; and 42*24=1008. This results in a 1:1 mapping of tmeExpression to frames (more intuitive), but forces the timeExpression to diverge from real time (and the video and audio timebases).

TTML 1.0 is silent how to convert timeExpression values to frames in this scenario. This should be clarified for the above configuration scenario and using all the time options that are ambiguous.

Note that integer framerates or fraction (not frame) timeExpressions have obvious transforms into frames.
' non-public

Mike Dolan, 9 Jan 2013, 22:00:43

Status changed to 'pending review'

Glenn Adams, 12 May 2013, 20:52:59

Status changed to 'closed'

20 Jun 2013, 14:55:40


David Singer <singer@apple.com>, Nigel Megitt <nigel.megitt@bbc.co.uk>, Chairs, Thierry Michel <tmichel@w3.org>, Philippe Le H├ęgaret <plh@w3.org>, Staff Contacts
Tracker: documentation, (configuration for this group), originally developed by Dean Jackson, is developed and maintained by the Systems Team <w3t-sys@w3.org>.
$Id: index.php,v 1.325 2014-09-10 21:42:02 ted Exp $