# Clarifications for Time Expression Semantics

• The following is a Change Proposal for Issue 199
• Date: January 28, 2013.

## Summary

TTML does not adequately define how to relate a time expression to media time when the effective frame rate is not an integer.

## Details

TTML defines the `media` time base as follows [1]:

If the time base is designated as media, then a time expression denotes a coordinate in some media object's time line, where the media object may be an external media object with which the content of a document instance is to be synchronized, or it may be the content of a document instance itself in a case where the timed text content is intended to establish an independent time line.

The media time base is related to local real time in accordance to the related media play rate and the related media real start time (i.e., the real time when the related media playback started), parameters not modeled by TTML itself. The relationship between media time (M) and local real time (R) is as follows:

```R = playRate * M + realStartTime
```

where

M ∈ ℜ | 0 ≤ M < ∞ | M in seconds
playRate ∈ ℜ | −∞ < playRate < ∞ | playRate is unit-less
realStartTime ∈ ℜ | 0 ≤ realStartTime < ∞ | realStartTime in seconds, with 0 being start of epoch

Without loss of generality, we will assume playRate is 1 (one) and realStartTime is 0 for the remainder of this document, which simplifies this relationship to R = M.

### Problem Example

A number of common non-integral frame rates occurs in common use in the U.S., originally deriving from NTSC video formats. An example of this is a frame rate of 30 * 1000/1001 = 29.970029970029… frames per second, or 41.708333... milliseconds per frame. In TTML, this frame rate would be denoted as follows:

```<tt ttp:frameRate='30' ttp:frameRateMulitiplier='1000 1001' ttp:timeBase='media' ...>
```

TTML time expressions allow specifying time using either a offset-time or clock-time format [2]. In the case of offset time expressions, one can use a variety of representations, such as a fractional number of seconds, a fractional number of frames, etc. In the case of clock time expressions, one can use a COLON (`:`) separated expression that includes hours, minutes, seconds, and, optionally, fraction of seconds or frames and optional sub-frames.

Valid time expressions include:

```30f
1s
1.25125s
00:00:01
00:00:01.25125
00:00:01:01
```

In general, it should be possible to convert between the different time expression formats without loss of information; for example, it should be possible to unambiguously convert all valid expressions to a fractional seconds offest or a fractional frames offset expression. However, because TTML does not clearly define this conversion, some ambiguity has appeared among readers as to how to interpret these expressions in the context of non-integral frame rates. In the following sections, this expression is interpreted according to two different methods.

### Method #1

According to this method, the time components of time expressions refer directly to media time coordinates, while frames refer to time intervals computed from the effective (possibly non-integral) frame rate. Using this method, the following formula applies, where effectiveFrameRate = frameRate * frameRateMultiplier:

```M = 60^2 * hours + 60^1 * minutes + 60^0 * seconds + ( frames / effectiveFrameRate )
```

Therefore, in order to convert the example time expression `00:00:01:01` to media time in fractional seconds, we have:

```M = 60^2 * 0 + 60^1 * 0 + 60^0 * 1 + ( 1 / 29.97002997002997 ) = 1.03336666666667s
```

To further convert this to a fractional frames, we have

```M = 1.03336666666667s * 29.97002997002997 (frames per second) = 30.97002997003007f
```

Finally, to convert this to an integral frame number, where the first frame is frame 1, we have

```frame number = floor(30.97002997003007) + 1 = frame 31
```

Let us compute for one more value, `02:00:00:00`, which yields:

```M = 60^2 * 2 + 60^1 * 0 + 60^0 * 0 + ( 0 / 29.97002997002997 ) = 7200s
= 7200s * 29.97002997002997 (frames per second) = 215784.215784215784f

frame number = floor(215784.215784215784) + 1 = frame 215785
```

### Method #2

According to this method, the time components of time expressions denote their equivalent in integral frames, i.e., without making use of the frame rate multiplier. As such, time expressions using this method operate like a frame counter, or, by analogy, like an odometer which counts off frames. Only when converting from this frame count to media time is the frame rate multiplier used (to compute the effective frame rate). Using this method, the following formula applies, where effectiveFrameDuration = 1 / (frameRate * frameRateMultiplier):

```M = (((60^2 * hours + 60^1 * minutes + 60^0 * seconds) * frameRate) + frames) * effectiveFrameDuration
```

Therefore, in order to convert the example time expression `00:00:01:01` to media time in fractional seconds, we have:

```M = (((60^2 * 0 + 60^1 * 0 + 60^0 * 1) * 30) + 1) * 0.03336666666667 seconds/frame = 1.03436666666677s
```

To further convert this to a fractional frames, we have

```M = 1.03436666666677s / 0.03336666666667 seconds/frame = 31f
```

Finally, to convert this to an integral frame number, where the first frame is frame 1, we have

```frame number = floor(31) + 1 = frame 32
```

Let us compute the results for one more example, `02:00:00:00`, which yields:

```M = (((60^2 * 2 + 60^1 * 0 + 60^0 * 0) * 30) + 0) * 0.03336666666667 seconds/frame = 7207.20000000072s
= 7207.20000000072s / 0.03336666666667 seconds/frame = 216000f

frame number = floor(216000) + 1 = frame 216001
```

### Comparison

As can be seen by comparing the values above, these two different methods yield different values when converting a time expression to media time and frame number:

Example 1
Method Input Seconds Frame
#1 00:00:01:01 1.033367s 31
#2 00:00:01:01 1.034367s 32
Example 2
Method Input Seconds Frame
#1 02:00:00:00 7200s 215785
#2 02:00:00:00 7207.2s 216000