IRC log of sync on 2016-09-22

Timestamps are in UTC.

14:39:25 [RRSAgent]
RRSAgent has joined #sync
14:39:25 [RRSAgent]
logging to http://www.w3.org/2016/09/22-sync-irc
14:39:33 [tidoust2]
RRSAgent, make logs public
14:39:46 [tidoust2]
Meeting: Multi-Device Timing CG
14:39:50 [tidoust2]
Chair: Ingar
14:40:22 [tidoust2]
Agenda: https://lists.w3.org/Archives/Public/public-webtiming/2016Sep/0003.html
14:41:15 [tidoust2]
Present+ Ingar_Arntzen(MotionCorp), Francois_Daoust(W3C), Mike_Assenti(Dolby), Louay_Bassbouss(Fraunhofer), Chris_Needham(BBC)
14:41:37 [tidoust2]
Present+ Kumanan_Yogaratnam(Espial)
14:41:53 [cpn_]
cpn_ has joined #sync
14:42:08 [tidoust2]
Present+ Toshihiko_Yamakami(Access)
14:42:27 [tidoust2]
RRSAgent, draft minutes
14:42:27 [RRSAgent]
I have made the request to generate http://www.w3.org/2016/09/22-sync-minutes.html tidoust2
14:42:50 [tidoust]
Present+ Anssi_Kostiainen(Intel)
14:43:13 [yam]
yam has joined #sync
14:43:25 [Louay]
Louay has joined #sync
14:43:36 [Louay]
present+ Louay_Bassbouss
14:43:36 [cpn_]
scribenick: cpn_
14:43:46 [cpn_]
scribe: Chris_Needham
14:44:28 [kumanan]
kumanan has joined #sync
14:46:17 [cpn_]
Ingar: This is the first F2F of the Multi-Device Timing CG
14:47:28 [cpn_]
... This work comes from the MediaScape EU project
14:47:58 [cpn_]
... The focus is on multi-device timing across web pages, and providing a programming model to deal with temporal aspects in web apps
14:48:12 [cpn_]
... This is a multi-device presentation
14:48:29 [cpn_]
https://goo.gl/AlR9Nm
14:49:39 [tidoust]
Present+ Shih-Chiang_Chien(Mozilla)
14:51:05 [cpn_]
Ingar: I'd like to show a demo
14:51:16 [cpn_]
http://mcorp.no/pres/vegas16/main.html
14:52:48 [cpn_]
Ingar: [ demonstrates showing a video on several devices in sync ]
14:54:19 [cpn_]
... If you look at the edges between the browsers, this is where you'll see any sync errors
14:55:08 [cpn_]
... The synchronisation was between two browser windows on a single machine
14:55:25 [cpn_]
... You can achieve similar precision in the global scope, eg, between devices in this room
14:56:00 [cpn_]
Toshihiko: What would happen when I return home to Japan?
14:56:31 [cpn_]
Ingar: ... It would still play in sync
14:57:07 [cpn_]
... It's symmetric, in the sense that we all have control
14:57:32 [cpn_]
... [ demonstration of pausing the video, all devices pause ]
14:58:06 [cpn_]
... I think we're in the single-digit millisecond region
14:58:43 [cpn_]
... The play and pause control commands do have a day, may take 0.2 seconds or so
14:59:26 [cpn_]
Mike: Are these commands also part of the spec?
14:59:29 [cpn_]
Ingar: Yes
15:00:03 [cpn_]
... We also handle re-establishing sync on page reload
15:00:14 [cpn_]
Topic: Introduction
15:00:29 [kumanan]
kumanan has joined #sync
15:00:31 [cpn_]
Ingar: Francois presented at TPAC last year
15:00:50 [cpn_]
... the draft Timing Object spec
15:01:17 [cpn_]
... using an online timing provider from Motion Corp
15:01:37 [cpn_]
... Since then we have an open source JavaScript implementation
15:02:07 [cpn_]
... We've demonstrated at NAB and IBC, and also published some papers
15:02:30 [cpn_]
... Media Element capabilities have also progressed
15:02:50 [cpn_]
... We were having trouble with Chrome on Android before, but this has become very good now
15:03:04 [cpn_]
... Safari has a bug
15:03:12 [cpn_]
Anssi: Has that been reported?
15:03:21 [cpn_]
Ingar: We'll be doing that, yes
15:04:21 [cpn_]
Ingar: Timestamping on the client side, using timing info in getUserMedia
15:04:39 [cpn_]
... We've also reported bugs in Chromium, now fixed
15:05:07 [cpn_]
... Multi-device timing is about making stuff happen at the right time
15:05:46 [cpn_]
... It's not only about having a clock, but also interactive capabilities to control it
15:06:27 [cpn_]
... Timing scopes: page internal, within the browser context, and also cross-page
15:06:53 [cpn_]
... Even though the group's name is Multi-Device Timing, both of these scopes are important
15:07:31 [cpn_]
... The Timing Object is a client-side API
15:07:50 [cpn_]
... We don't prescribe a particular solution for the cross-page timing
15:07:57 [cpn_]
... It can be implemented in different ways
15:08:16 [cpn_]
Topic: Timing Challenges
15:08:42 [cpn_]
... With the page internal timing, there could want to synchronise multiple videos
15:08:54 [cpn_]
s/there could/you could/
15:09:18 [cpn_]
Ingar: Ad insertion can be done with MSE
15:09:34 [cpn_]
Francois: There are some limitations with what you can do with MSE
15:10:01 [cpn_]
Ingar: Our approach gives more flexibility
15:10:33 [cpn_]
... It works with different types and frameworks
15:10:57 [cpn_]
... iframes are important, eg for independently delivered adverts
15:11:24 [cpn_]
... You can split content across windows or frames
15:11:54 [cpn_]
... You can also sync between web pages and native apps
15:12:10 [cpn_]
... Collaborative applications
15:12:46 [cpn_]
... It can work across different types of network connection, eg, 3G or wifi
15:14:10 [cpn_]
Media elements and animations have different APIs, making reusability and flexibility difficult
15:14:26 [cpn_]
s/Media elements/... Media elements/
15:14:55 [cpn_]
Ingar: The Media element currentTime attribute can be quite coarse
15:15:14 [cpn_]
Francois: The spec doesn't force the UA to apply a precise clock
15:15:54 [cpn_]
Anssi: We specified an extension to Web Audio, the context time, which is when the audio hits the speakers
15:16:22 [cpn_]
... We'll be adding timing to sensors, etc, anywhere it makes sense
15:16:37 [cpn_]
... We're working on low-level primitives that make your use cases possible
15:17:30 [cpn_]
... An example is screen touch events, knowing when these happen
15:18:11 [cpn_]
Ingar: It would be nice to write a sync wrapper for Web Audio
15:18:53 [cpn_]
... Non-determinism is a problem, especially in the distributed case
15:19:17 [cpn_]
... There's no support for cross-page timing at the moment
15:20:04 [cpn_]
... It's not a sound assumption to rely on NTP synchronised system clocks
15:20:53 [cpn_]
... Control and timing are typically dealt with separately, but we're trying to deal with these in a single abstraction
15:21:29 [cpn_]
Francois: Have you thought about how Web Audio synchronisation might apply to the Presentation API?
15:21:38 [cpn_]
Anssi: No, not yet.
15:23:12 [cpn_]
... [ some discussion of issues with Performance.now ]
15:23:48 [cpn_]
Ingar: We get around that, using a Timing Object, which you can give to two processes in the same architecture
15:23:54 [cpn_]
Topic: Goals for the group
15:24:14 [cpn_]
Ingar: We can to simplify things for developers
15:24:53 [cpn_]
... Interoperability is important. With the web everything is composeable, but not timing
15:25:20 [cpn_]
... Also synchronising web and broadcast content
15:25:46 [cpn_]
... We have a concrete proposal, and our demos show that it's feasible
15:26:03 [cpn_]
... [ another demo ]
15:26:24 [cpn_]
https://youtu.be/D6cBn65KXTk
15:26:44 [cpn_]
Ingar: It shows how to create a web page that uses timing
15:28:55 [cpn_]
Kumanan: There's an issue here with iOS, where it only allows one video playing at once
15:29:32 [cpn_]
... I have Blink in one and Firefox in the other
15:30:08 [cpn_]
Topic: Proposal
15:31:14 [cpn_]
Ingar: The timing object can play, pause, jump. But it's not connected to a media element
15:31:23 [cpn_]
... [ demo of local timing object ]
15:31:56 [cpn_]
... You can change position, velocity, acceleration
15:32:52 [kumanan]
kumanan has joined #sync
15:33:10 [cpn_]
... through the API using a vector
15:33:54 [cpn_]
Francois: There's the same kind of object being defined in WebVR, where a pause operation affects velocity
15:35:08 [cpn_]
Chris: What's the motiviation for acceleration? Could you have easing functions for changing between one velocity and another
15:35:41 [cpn_]
Ingar: Acceleration isn't a central thing in this. You could build such functions on top of the API
15:36:13 [cpn_]
... It's not critical, but could be useful
15:36:25 [cpn_]
Kumanan: Does velocity have direction?
15:36:43 [cpn_]
Ingar: Yes, so negative reverses the playback direction
15:37:04 [cpn_]
Mike: Are the units arbitrary?
15:37:15 [cpn_]
Ingar: Yes.
15:38:21 [cpn_]
Mike: If we want to relate this to a media content pipeline, For example, if you want to double the rate, how do you know what the scale of the timeline is?
15:38:52 [cpn_]
Francois: When you associate the Timing Object with a media element, position would be seconds, so the units are set there
15:39:22 [cpn_]
... We're also thinking about how to compose Timing Objects, different clocks at different rates
15:39:58 [cpn_]
Ingar: There are different options. You could define a mapping, so we've defined a TimingConverter
15:40:08 [cpn_]
... You'd use this with media objects having different timelines
15:40:40 [cpn_]
... You could also change the data to fit the different timeline
15:41:36 [cpn_]
Kumanan: When I change velocity, how do I know what my normal is?
15:42:00 [cpn_]
Mike: I think the question is how the position and velocity relate to the timestamp value in the vector
15:43:30 [cpn_]
Kumanan The timestamp comes from Performance.now, which is local
15:44:37 [cpn_]
[ discussion of how timestamp relates to the vector ]
15:44:52 [cpn_]
s/Kumanan The/Kumanan: The/
15:46:46 [cpn_]
Ingar; An application can remember what the normal is. The velocity doesn't jitter
15:46:55 [cpn_]
s/Ingar; An/Ingar: An/
15:48:03 [cpn_]
Ingar: The TimingObject has change and timeupdate events
15:49:54 [cpn_]
... When someone changes the TimingObject, all others are updated
15:50:44 [cpn_]
... The TimingObject is in control, and it's the component's responsibility to respond
15:51:51 [cpn_]
Toshihiko: Will the TimingObject cover other use cases, such as a carousel where you want to loop from the end to the start
15:52:10 [cpn_]
Ingar: It's quite simple to add that on top of this
15:52:22 [cpn_]
... It's a matter of preference
15:53:02 [cpn_]
Ingar: A hosted internet service can provide a timing service, so the local TimingObject becomes a proxy
15:53:35 [cpn_]
... There's latency but the state will be correct
15:54:52 [cpn_]
... One advantage of a hosted solution is availability
15:55:32 [cpn_]
... [ demo of online timing object ]
15:56:17 [cpn_]
... Doing this in user space JavaScript, you lose some precisiion
15:57:57 [cpn_]
... We have the TimingObject API for clients, but there's also an API for timing providers
15:58:27 [cpn_]
... This is simpler, as it just has to maintain two variables: the vector, and the skew
15:59:21 [cpn_]
... The main source of error is the step between the TimingObject and the media element.
15:59:48 [cpn_]
Kumanan: If this becomes a proper Web API, would there be a timing provider mandated?
16:00:07 [cpn_]
Francois: The default implementation would use Performance.now
16:01:30 [cpn_]
... [discussion of timing providers and skew ]
16:02:37 [cpn_]
Ingar: The skew would be that between Performance.now and the underlying timing source
16:03:47 [cpn_]
... There are lots of applications: collaborative viewing, multi-screen displays, multi-device in the home, showing interactive content with a main video feed
16:04:45 [cpn_]
... Also remote control, music, multi-device capture, timestamping material according to an app-specific timeline
16:05:04 [cpn_]
... Which makes the content re-playable in a multi-device setup
16:06:09 [cpn_]
Ingar: We've done some experiments on sync precision
16:06:37 [cpn_]
... We're measuring the difference between the TimingObject and the media position
16:06:53 [cpn_]
... When there are big differences, it's best to skip position
16:07:00 [cpn_]
... When it's closer, adjust the playbackRate
16:08:14 [cpn_]
... [ shows graphs of measurements taken on Android, Chrome ]
16:09:14 [cpn_]
... We'd like to be able to supply a timestamp to the media element's play command
16:11:11 [cpn_]
... [ graph shows problem with playbackRate on iPad ]
16:12:15 [cpn_]
Topic: Standardisation
16:13:01 [cpn_]
Ingar: Improvements needed with playBackRate, seek
16:13:45 [cpn_]
Francois: We may also need to look at Web Animation, you may not know the latency, ie, when the animation will actually start
16:15:13 [cpn_]
Ingar: Text tracks are limiting, we have a sequencer object that uses timing object
16:15:59 [cpn_]
... We use setTimeout in the sequencer, which gives precision of a few milliseconds, but delays if the browser is busy
16:16:35 [cpn_]
Francois: Due to the event loop, but could use tasks or micro tasks instead
16:18:01 [cpn_]
Kumanan: Wet set limits to the setTimeout interval
16:18:16 [cpn_]
Topic: Related work
16:18:28 [cpn_]
Ingar; There's the Second Screen WG
16:18:58 [cpn_]
Louay: HbbTV 2.0 has sync, discovery, communication, app launch in one package
16:19:07 [cpn_]
... Similar for Google Cast
16:20:12 [cpn_]
... The protocol work in the Second Screen group will touch on synchronisation
16:20:21 [cpn_]
... Should we have one group at W3C looking at this?
16:21:11 [cpn_]
Francois: I don't expect the Second Screen WG to take on this work
16:22:13 [cpn_]
... One thing in-scope for the Second Screen CG is reporting of currentTime for the Remote Playback API
16:22:53 [cpn_]
Kumanan: That just needs to address reporting
16:24:04 [cpn_]
Francois: I'm trying to gather interest in the multi-device timing CG
16:24:33 [cpn_]
... There have been improvements in the web platform, which make sync easier, but no specific work on sync itself
16:25:25 [cpn_]
Kumanan: I suggest a separate timing spec is valuable more generally than just media applications
16:26:38 [cpn_]
Francois: A native timing object isn't always needed, but it is needed where you want to connect to a media element
16:28:29 [cpn_]
Kumanan: There are applications such as real time collaboration where this is valuable
16:29:32 [cpn_]
Topic: Next steps
16:29:57 [cpn_]
Ingar: We want to attract support to the CG
16:30:25 [cpn_]
Francois: We could track progress in the existing specs that have timing involved
16:31:07 [cpn_]
Kumanan: What's the blocker to moving this forward?
16:31:33 [cpn_]
Francois: It needs someone willing to implement natively
16:32:00 [cpn_]
... [ discussion on differences between CGs, WGs, and IGs at W3C ]
16:34:47 [tidoust]
tidoust has joined #sync
16:36:11 [cpn_]
present+ Eric_Carlson(Apple)
16:36:34 [cpn_]
Ingar: [ recaps issues with iOS timing measurements ]
16:41:05 [cpn_]
Louay: Do you have any experience of syncing live streams, eg HLS?
16:41:18 [cpn_]
Ingar: Yes, i have a demo
16:44:36 [cpn_]
Louay: [ discussion of currentTime in live streams ]
16:44:47 [cpn_]
Eric: startDate should work with HLS streams if the information is in the manifest
16:46:29 [cpn_]
... [ discussion of the play method that returns a Promise ]
16:49:17 [tidoust]
RRSAgent, draft minutes
16:49:17 [RRSAgent]
I have made the request to generate http://www.w3.org/2016/09/22-sync-minutes.html tidoust
16:49:50 [cpn_]
Ingar: Thanks all for coming, please join the group, suggest use cases, point it out to others
16:49:56 [cpn_]
[ adjourned ]