Web Animations/Meetings/Meeting 8
< Web Animations | Meetings
Time, location: 1 Mar 16:00 PST / 2 Mar 11:00 AEDT / 2 Mar 09:00 JST, #webanimations
Agenda
- Accessibility - Shane
- Synchronization discussion
- Synchronisation with script—Do we need synchronous events? How feasible are they? Do we need groups? etc.
- Shane and Brian to look into feasibility / integration with DOM event model, RequestAnimationFrame etc.
Time Machine / Time as Artefact etc.—stepping back and designing declarative synchronisation first?
- Synchronisation with script—Do we need synchronous events? How feasible are they? Do we need groups? etc.
API / Object primitives- Brian to propose alternative API without stacking so we can compare
Actions
- Brian to ask what the basis was for including speed control in SVG Tiny 1.2
- Brian to add timed text track integration to the requirements doc
- Shane to ask about triggers and a11y
- Shane and Brian to find out about how rAF works and if it's possible to prioritise events
- Rik and Brian to comment on concerns with separation of play and edit heads
Log
<birtles> woah, it's web animations time! <birtles> who's here? <shans__> me! <birtles> yay! <birtles> cabanier, how about you? <birtles> shans__, by the way, did you book Tokyo yet? <shans__> birtles: not yet, but I'll be doing that pretty soon. Is there an event on or something? <birtles> like a port festival or soccer final? (as per hamburg and bulgaria) <shans__> yeah :) <shans__> I often book my travel pretty late <birtles> not that I'm aware of <cabanier> I'm here! <shans__> let's get started then <birtles> I'm not sure if Tokyo ever really books out---just transport gets messy in golden week, new year etc. <birtles> ok <birtles> the agenda so far: <birtles> 1) Accessibility <birtles> 2) Synchronization discussion <birtles> 3) API / Object primitives <birtles> http://www.w3.org/Graphics/fx/wiki/Web_Animations/Meetings/Meeting_8 <birtles> any changes? <cabanier> no <shans__> looks good <birtles> alright then, Shane, tell us about accessibility <shans__> ok <shans__> so what it all boils down to is that there's a single object in the DOM we need to keep in mind <shans__> object type rather <shans__> Timed Text or something like that <shans__> so basically all the different subtitle / a11y text formats (and there are hundreds), if they're to be used in the browser, will map into Timed Text objects in the DOM <shans__> and I think we need to consider them in 2 different ways <shans__> (1) they're currently being used to carry video/audio-aligned snippets of data, not just for a11y, but also for animations / layout changes / etc. <shans__> and <shans__> (2) we should support TT timelines being aligned with our animations, which would then enable our animations to be made accessible <birtles> sounds good <shans__> so that's basically it :) <shans__> any questions? <birtles> does that cover the subtitling use case as well? <shans__> yeah pretty much <shans__> well, half of it <shans__> if we've got an animation that needs subtitles <shans__> then the Timed Text objects will carry the subtitle information <cabanier> is this timed text object used by the accessiblity software? <shans__> but we'd still need to work out a way to make it convenient for that information to be presented <shans__> I believe that is the intention, yeah <birtles> ok, sounds good <birtles> I'll add that to the requirements list <shans__> but I'm slightly worried that I've got the name wrong - I'll check again with Silvia <shans__> great <birtles> on that note, I added a few other minor ideas about accessibility and animation: http://www.w3.org/Graphics/fx/wiki/Web_Animations/Requirements#Accessibility <birtles> stuff like role identifiers for animation triggers <cabanier> so, every browser is exposing this interface somehow <shans__> well, the interface is specified by the w3c <shans__> (which isn't quite the same thing, I know) <shans__> I'm not sure what browser support there is <shans__> well, I think WebKit supports it <birtles> is there a link for the DOM part of it? <shans__> that's what I'm trying to find <shans__> unfortunately Silvia's on her honeymoon this week <birtles> I've found heaps of links for the markup (TTML) but not the interface <shans__> but I can ask her next week <cabanier> What about non-subtitle case? <cabanier> I know for our software, people want to hover over part of a pdf or flash file and get a spoken description of what it is <cabanier> or is that out of scope? <shans__> right, but that's not animations <cabanier> OK <shans__> I think that's important, and I think there's a mechanism in HTML to allow that <shans__> what animations brings that is new is the fact that the information being presented is time-dependent <shans__> so the description needs to be time-dependent too <birtles> so you might have a variation on subtitles that has descriptions along the way <birtles> which would be covered by timed text <shans__> yeah <birtles> like on some DVDs they have the explanatory English track <shans__> and like I mentioned before, people are using the timed text for all sorts of things <birtles> which describes what's going on on the screen <shans__> for example, CSS styles that should apply to the page at different points in the video <cabanier> but you wouldn't do it per character <shans__> (that needs JavaScript to work, of course, but the TT provides the alignment mechanism) <cabanier> ie person a is playing, person b is now cheering, etc <cabanier> OK <birtles> ok, one other possible requirement... <birtles> I think the SVG WG had a requirement for speed control on animations (and video/audio) due to accessibility <birtles> for SVG Tiny 1.2 <shans__> there's a track on some video content specifically for the hard-of-hearing. Is that what you're talking about brian? <birtles> shans__, no there's sometimes a track for visually impaired people <shans__> it's like subtitles plus [loud explosion] .. [doorbell rings], etc. <shans__> ah, right. At any rate, all of those things are meant to map back to this primitive <birtles> it tells you "Now the logo for the production company is appearing on the screen" <birtles> "Now descending on a 1920s bustling street" <birtles> etc. <birtles> so anyway, I think speed control might be another requirement but I'm not sure exactly who the customer is for that <cabanier> if you want to slow things down, you probably want to slow everything down. So it could be a global browser switch and not something per animation <shans__> I think it has to be a global browser thing <birtles> perhaps I ought to just ask those involved <shans__> otherwise complex animations would break badly <birtles> in SVG Tiny 1.2 it's on a per <animation> level from memory <cabanier> that makes it a lot easier <cabanier> that wouldn't work I think <birtles> ACTION Brian to ask what the basis was for including speed control in SVG Tiny 1.2 <shans__> http://www.whatwg.org/specs/web-apps/current-work/#timed-text-tracks <shans__> that's a link to the web-facing API for Timed Text tracks <shans__> plus other stuff <birtles> ACTION Brian to add timed text track integration to the requirements doc <birtles> ok, well that probably feeds into however we approach synchronisation <birtles> where there any other suggestions for what accessibility features might be needed? <shans__> I think it happens at a higher level, actually <shans__> Silvia at least was reasonably dismissive of there being too many a11y concerns for animation <shans__> but I think she was considering it from the CSS Transitions POV - i.e. really basic layout animations <shans__> I need to talk to Alice too but I haven't had a chance to yet <birtles> well, I think having declarative animation triggers is one thing that would aid accessibility <shans__> dismissive is the wrong word <birtles> so you can support alternative input methods <shans__> interesting, yeah. <birtles> there are a few other things <birtles> like support role identifiers <shans__> I've been thinking about the trigger area as being outside web animations largely though <shans__> same with role identifiers <shans__> like, if you've got a div that animates between different states in response to user input, then you should put aria roles on the div <shans__> (but you should anyway, in the absence of animation) <shans__> or am I missing something? <birtles> I'm thinking about stuff like <animate begin="circle.click; accessKey('a')"> <shans__> well the thing is in that case the circle is a button <shans__> and should be marked as such <shans__> which means that the existing a11y mechanisms (tab order inclusion, trigger-via-enter, etc.) just work <shans__> if you also want to support hotKeys, I think that's a pretty cool feature, but not an a11y one specifically <birtles> there's already hotkey support in SVG--but one of the open requirements we have is about extending that <birtles> Vincent was keen to see DOM3 key identifiers supported <birtles> and in doing so we just need to be aware of making sure that information is available to accessibility agents <birtles> anyway, we can talk about the details later <birtles> perhaps we can move on to synchronisation <shans__> yep. Actually I see what you mean now. If someone *does* specify 'a' as a trigger, then how can we let the a11y agents know that this trigger is available and (e.g.) speak it to the user <birtles> yeah, or, if they are using an input device that doesn't have 'a', map it to something suitable <shans__> right <shans__> not sure if that's currently even possible, outside of animations <shans__> I'll ask about it <birtles> ok, cheers <shans__> ACTION Shane to ask about triggers and a11y <birtles> ok, synchronisation? <shans__> yep <birtles> oh, by the way I have to go in 30min <shans__> I probably do too <birtles> ok <cabanier> OK <birtles> I did a little research <birtles> It turns out there are already synchronous events in some specs <shans__> oh? <birtles> e.g. MutationEvents (which everyone hates and is now deprecated) <shans__> haha <shans__> yes <shans__> well, actually the thing is <shans__> *all* events are synchronous by the time they get to JavaScript <birtles> right <birtles> but the dispatch is synchronous for MutationEvents <shans__> so what you're talking about specifically is called-on-mutation rather than called-after-mutation, right? <cabanier> :-) <shans__> heh <birtles> yeah, it's the "dispatch" that's synchronous <shans__> ok great <birtles> I spoke to our DOM events expert about using that kind of event for animation <shans__> did they run away screaming? <cabanier> you mean: you change something in the DOM in JS and this event is called immediatly? <birtles> he seemed to think it's ok as long as you're careful about when you do the dispatch <cabanier> that would be ugly <shans__> oh that's a good sign :) <birtles> (i.e. you make sure you're in a safe state and don't make any assumptions about the state of the world when each event handler returns) <shans__> Rik - not sure about that <birtles> I think Opera already does this for SVG animation events <birtles> but I also spoke to David Baron about it and he was less enthusiastic <shans__> so there are two reasons that everyone hates DOM MutationEvents <shans__> (1) the programming model is hard for common tasks <shans__> (2) including the events slowed down DOM manipulations by ~ 10x, even after optimisation <birtles> yeah, the performance is terrible <cabanier> that makes sense <birtles> I think the Skype plugin was particularly infamous for this <shans__> I think your DOM expert was saying that (1) probably wouldn't apply to the animations case, right? <birtles> yeah, I don't think the programming model is hard, easier if anything <birtles> (2) is really where David Baron was concerned <birtles> can I give an example? <cabanier> yes <birtles> so if we have an animation that finishes at t=5s <birtles> and our next sample is at t=5.5s <birtles> in Gecko's implementation of SVG animation (but not CSS) we'll actually force a sample at t=5s <birtles> a "sample" here just involves updating the animation model <birtles> it doesn't render or anything like that <birtles> it's cheap <birtles> but it allows synchronisation to work as expected <birtles> then we'll go on and do another sample at t=5.5s <birtles> and *then* we'll render with the final values <birtles> IF, however, we decide to synchronously dispatch an end event at t=5s... <birtles> that handler could do anything <birtles> including causing a reflow <birtles> that's fine <birtles> but if this sort of situation comes up a lot <birtles> you could end up getting a lot of reflows and other expensive operations happenning at a frequency greater than the refresh rate <birtles> (i.e. sample rate) <birtles> and that's what David was concerned about <cabanier> that is true <birtles> poorly written web apps producing excessive reflows etc. <cabanier> but considered 'bad programming' <cabanier> I guess you could detect that and abort <birtles> right, but there's a lot of bad programming on the web <shans__> the thing about DOM layout was that the 10x slowdown occurs *even if you do nothing in the handler* <birtles> cabanier, ok, that's an idea <shans__> it's a consequence of just registering the handler <birtles> shans__, yeah, that's right <cabanier> the handlers should not execute that often <cabanier> and doesn't requestanimationframe suffer the same issue? <birtles> the mutation event stuff is expensive partly because you're supposed to supply "new" and "old" values <birtles> (and event dispatch in Gecko is expensive, or at least it used to be) <birtles> requestanimationframe only runs at the refresh rate <shans__> well we should look at what would need to be provided in an animation event too then <birtles> so you can only cause a reflow at the same frequency as the refresh rate <birtles> not more frequently <cabanier> true <shans__> we were talking about whether requestAnimationFrame is more privileged than other event handlers the other day. Did anyone discover if it is? <shans__> because I think if it isn't, that kinda sinks us being able to make animation events more privileged. <birtles> what do you mean (/ "what did we mean") about more privileged? <shans__> well, if I've got a bunch of outstanding events (e.g. DOM mutation events) and a rAF handler registered, is it guaranteed that the rAF handler gets called first? <birtles> hmm, not sure <cabanier> does it matter? <shans__> yeah <birtles> if we go with "synchronous dispatch" then the handler *will* be called first <shans__> if we don't do that for animation events then we have a situation where animation progress can be delayed by *other* event handlers too <cabanier> you mean, someone presses a button at the same time and handler is invoked? <shans__> Brian, that's not necessarily true. <birtles> how so? <shans__> ok, what happens in this situation: <shans__> (1) a key press occurs, causing a KeyEvent to be enqueued. JS has not yet been called. <shans__> (2) another key press occurs, causing another KeyEvent to be enqueued. <shans__> (3) the first key press KeyEvent is delivered and JavaScript modifies the DOM. A DOMEvent is generated. <shans__> which event is delivered first? <birtles> I can't see the relevance? <birtles> what is the final DOMEevent? <birtles> s/DOMEevent/DOMEvent/ <cabanier> is the even in 3 one of the mutationevents? <shans__> DOM Mutation of some kind <birtles> then it will be delivered first <cabanier> well, then 2) will be last <birtles> right <shans__> hmm <shans__> does 1) finish executing before (2) is called? <shans__> sorry <shans__> does (1) finish executing before (3) is delivered? <cabanier> (if they're synchronous like brian said) <birtles> no <shans__> then why is (3) delivered before (2)? <birtles> it's synchronous, it's like a function call <shans__> but it isn't <shans__> the mutation occurred partway through (1) <shans__> and it finishes executing <shans__> so there is still a delay <birtles> hang on, sorry, I misread your description <birtles> (1) finished a long time ago <birtles> so did (2) <shans__> no <shans__> the timeline is this: <shans__> KeyPress (1) --- KeyPress (2) --- KeyPress Event handler (1) { ------ DOM Mutation *A* --------- } *B* <shans__> now at *A* do we have DOM Mutation hander { ---- }, or nothing <shans__> and at *B* do we have DOM Mutation handler { ---- } KeyPress Event handler (2) { ---- } <shans__> or KeyPress Event handler (2) { ---- } DOM Mutation handler { ---- } <shans__> or just KeyPress Event handler (2) { ---- } <shans__> ? <birtles> sorry, I don't quite understand the diagram <cabanier> It sounds like the mutation events are generated AND dispatched when you change an attribute. You did't leave the keyevent handler <shans__> time 0 is at the left <shans__> things inside { } are JavaScript code executing <shans__> yeah it's a lame diagram. Wish we had a whiteboard :) <birtles> yeah, I was just thinking "can't wait until we meet face to face"! <birtles> ok, can I try... <birtles> 1. key press 1 <birtles> 2. key press 2 <birtles> 3. event handler from key press 1 <birtles> 4. event handler modifies something <birtles> 5. mutation event handler is run <birtles> 6. mutation event handler finishes <birtles> 7. control returns to event handler from key press 1 <birtles> 8. event handler from key press 1 finishes <birtles> 9. event handler from key press 2 executes <cabanier> yes <shans__> ok <cabanier> we definitely don't want this for the keyframe events <shans__> agree <birtles> I'm not saying this is great... but this is one path we "could" take <birtles> originally this is how I implement the animations events for SVG <shans__> it's very easy to end up in infinite modification loops with this approach <cabanier> yes <birtles> but you have to be really careful about the state of the world when you dispatch the events <cabanier> and it's very confusing to the author <birtles> so I changed it to async <birtles> but at that time Opera was doing sync (from my testing) <birtles> I thought this was closer to what Rik was looking for though? <shans__> the new DOM Mutation model btw is to push DOM changes into a list <shans__> then deliver the list to a listener some time later <shans__> so you'll end up with <shans__> 1. key press 1 <shans__> 2. key press 2 <shans__> 3. event handler from key press 1 <cabanier> You always want to return to the JS VM <shans__> 4. event handler modifies something <shans__> 5. event handler from key press 1 finishes <shans__> 6. event handler from key press 2 executes <shans__> 7. log of DOM changes delivered to mutation event handler <cabanier> that is reasonable <birtles> yeah <birtles> hmm, I'm not sure we've solved this one yet, but at least we've got a better idea of the options and issues <shans__> so if we do that approach for animation events <shans__> then we definitely have a situation where an animation can be paused waiting on resolution of unrelated handlers <shans__> (if we pause the animation on event generation) <shans__> 1. keypress 1 <birtles> gotcha <shans__> 2. keypress 2 <shans__> 3. animation event occurs. Animation paused <shans__> 4. key press handler 1 executes <shans__> 5. key press handler 2 executes <shans__> 6. animation event handler executes. Animation unpaused <cabanier> ah. 3 and 6 should happen at the same time though <cabanier> there should be no delay <cabanier> (or as short of a delay as possible) <shans__> I don't think we can support that model. It's one of the reasons why DOM mutations were hated <birtles> Shane's talking about if we don't have "synchronous dispatch" OR some sort of prioritisation for animation events <shans__> yes <birtles> so there's one option: async dispatch + pausing + prioritisation of animation events? <birtles> if that's even possible <birtles> as far as I know, there's no facility for prioritisation so far <shans__> right, I think the only two viable options are that one <shans__> and the edit / play head separation approach <shans__> there might be others of course :) <birtles> well, we've got to wrap it up soon <cabanier> I don't think the seperation approach will work <birtles> I'm inclined to agree although I'm not 100% sure the sync dispatch option is unworkable <birtles> I'm also hesitant about the separation approach <shans__> I think this is something we're going to end up talking about at our f2f <cabanier> it seems that it could tie in with the requestanimaitonframe <cabanier> yes <birtles> yes, I agree <shans__> it's not yet clear that requestAnimationFrame actually has priority <birtles> maybe we can continue it then <birtles> but in the meantime it would be worth researching if there is the possibility of prioritising events <shans__> it's not even guaranteed on mobile firefox (e.g.) that rAF will complete before rendering of the associated frame occurs <shans__> yeah I think we need to know what rAF actually does <birtles> ok, well, Shane, perhaps we can ask our respective contacts? <shans__> and I'd really like to see as many concrete objections to separation of play and edit heads as possible, written down somewhere so we capture them all <shans__> birtles: sounds good <birtles> ACTION Shane and Brian to find out about how rAF works and if it's possible to prioritise events <birtles> ACTION Brian and Rik to comment on concerns with separation of play and edit heads <birtles> next meeting? <shans__> next Monday/Tuesday? <cabanier> I might be able to do Monday <cabanier> but I might be a bit late <birtles> can do---not sure how much progress I'll make on my actions by then however (since I'm away on Monday) <birtles> alright, should we delay it 30min? <birtles> i.e. 4:30pm, 11.30am, 9.30am <cabanier> OK! <shans__> sounds good <birtles> great, thanks guys! <shans__> we can talk about the Animation API then <birtles> ok, we can prioritise that if you like <birtles> see you next week! <birtles> have a good weekend! <shans__> kk <shans__> you too