Nigel: [iterates through agenda]
… Any other things to cover today?
Andreas: Implementation demo if we can make 5-10 minutes
group: [introductions]
Gary: Around June I got pretty close to completing an Implementation Report for WebVTT.
… It's the first real one for it.
… There are definitely, to my surprise considering how badly historically WebVTT was implemented, there's quite a lot of support.
… There's a lot of support for the basic features.
… But there are some issues around the newer features like regions, as well as styling.
… There were specifically 7 issues opened from this IR
… because they did not pass the 2 independent implementations test.
… We should go through them today to discuss how to address them.
<gkatsev> WebVTT IR
Gary: There are a lot of failures in some cases
Nigel: The CR exit criteria are about features, so if some tests fail it doesn't necessarily mean the feature isn't supported.
… We can tell a story about the features.
Gary: Yes, I know in Firefox there's a specific "Edge cases" test that doesn't pass but we might be able to ignore that
… for the CR exit criteria.
… I have this pull request that's been ongoing for a while.
<gkatsev> snapshot PR
Gary: This collates all the things we want to mark as at risk because there are not enough implementations.
… It covers line alignment, position alignment etc which ultimately may need to be removed from the spec.
Andreas: Aren't those pretty core features of WebVTT?
Gary: They are fairly new features to WebVTT and there are basically no implementations that support them.
… I think the working mode is to update WebVTT to match what is out there, considering we should fix bugs, to get to V1
… and then if a feature is important then bring it in for the next version.
… Then with a Rec we would have more power to request browser vendors to implement the new features, and have
… more of a conversation rather than everyone ignoring it.
<gkatsev> github: https://github.com/w3c/webvtt/issues/463
Gary: This test is about the API for VTTCue.
… The API states that region is a property on the cue.
… It's failing across the board because Chrome and Edge don't support the region API at all.
… Firefox is failing because it's very strict and an older version of the spec said to throw if there's an incorrect value
… but the current version says ignore invalid values.
… The Safari one is interesting - it still implements the old version with a regionId property referencing a global
… list of VTT Regions.
… It does work but not spec-compliant to the current version of the spec.
… There is an open issue for it.
<gkatsev> safari bug for vttregion
Gary: Is that likely to get fixed soon?
Eric: Yes inasmuch as I am able to control things, yes.
Gary: We won't be able to exit CR unless we write a shim.
Eric: I will take that feedback back.
Gary: I spoke about a different issue about the parser in Firefox and they seem to be on board,
… so we can reach out to them and they'll be okay to update this.
Eric: Chrome?
Gary: I asked Mounir earlier in the week and he was unable to promise a timeframe but did seem willing.
Eric: I will relay that feedback to the people that figure out schedules, I understand it is really important.
Gary: Mounir did say there's someone working on it because they are updating inline styles.
Nigel: So exit strategy for this issue is not to mark as at risk but wait for implementations?
Gary: I think so.
… There's also VTT.js which we could add region support for as well.
… That would allow us to exit without necessarily waiting for Safari to update, if say Firefox gets updated.
… It depends on browsers, but VTT.js can be one implementation
PROPOSAL: Don't mark VTTCue region as at risk; instead wait for implementations to pass the tests.
Gary: Any objections?
Gary: No objections
Resolved: Don't mark VTTCue region as at risk; instead wait for implementations to pass the tests.
github: https://github.com/w3c/webvtt/issues/466
Eric: This seems to be the same issue.
Gary: yes
David: It was conformant to the standard when we made it, the spec changed after!
Gary: Yes
David: Agree it was a while ago
Gary: The statement is non-judgmental
Nigel: This issue mentions the test, which part of the spec are we considering?
Gary: This is a weird split in how WebVTT tests work. There are the API tests and parsing tests.
… The parsing tests rely on the API to introspect that the parsing was correct.
… This test is a parsing test to make sure the region block gets parsed correctly,
… but the test is failing because the API isn't implemented.
Eric: We can't look at the region parsing tests until the feature is implemented per spec.
… Until then testing for attributes of the spec doesn't make sense.
Gary: Makes sense, because it's all tied together.
… In that case the proposal is the same as for #463.
PROPOSAL: Don't mark VTTCue region as at risk; instead wait for implementations to pass the tests.
Eric: Ultimately this is the same issue as #463 at this point.
Resolved: Same as resolution for #463: Don't mark VTTCue region as at risk; instead wait for implementations to pass the tests.
github: https://github.com/w3c/webvtt/issues/465
Eric: I think this is the same
Gary: Partially, because of the Firefox issue.
… This is similar for Safari, but for Firefox they do parse it out but because this tests edge cases they end up stopping parsing early.
… So the tests fail.
… This could be a candidate for saying this test does not need to be passed for CR exit and we remove from the IR.
Nigel: Who has determined that they are edge cases? Edginess is in the eye of the beholder!
Gary: It's in the name...
Nigel: Yes but what about them is edgy?
Eric: It's testing things defined by the spec. Unless the spec says you should abort parsing the file it is exposing a bug
… in Firefox.
Nigel: So this tests that the parsing rules are met, and that's an important feature of WebVTT, that it is forgiving of
… badly formed files?
Eric: Right.
… Firefox fails because of a bug in the parser, or incorrect behaviour in the parser.
… Safari fails because it doesn't implement the current version of the spec.
… Technically it's a bug in the parser but it isn't even attempting to do the right thing.
Gary: Right, Firefox skips too much.
… So it makes sense to leave it in, and the proposal would be to wait for implementations.
Nigel: Should we also propose to rename the test, because this isn't really edge case, it's important.
Gary: Yes, that would be good as well, we don't want the connotation that this isn't really important.
PROPOSAL: Rename the test to be more clear about the purpose of the test and wait for the implementations to update; no change to the spec.
Gary: Any objections?
… No objections.
Resolved: Rename the test to be more clear about the purpose of the test and wait for the implementations to update; no change to the spec.
github: https://github.com/w3c/webvtt/issues/467
Gary: We've discussed this before and the issue is that there was an ambiguity in how to parse the lines attribute
… especially when there is a very large or very negative number.
… Safari seemed to do what makes sense for the correct behaviour.
… I reached out to the Firefox implementer and he was onboard with making the change.
… The pull request #460 updates the spec to match that behaviour.
Nigel: Has the test been updated? Or does it not need to be?
Gary: The test may need to be updated.
… I don't think I did it.
Eric: What does the pull request say?
Gary: In terms of the rounding?
Eric: Does it say what to do if it is not a number at all?
… Or is that implied?
Gary: I think it came from HTML
… Assuming that the value is a number this would be correct.
… I do not remember what happens if the value is not a number.
… [looks it up]
… The formal definition skips the parsing if there are non-ASCII digits, then parses the ASCII digits as an integer,
… and the new text specifies how to map that integer into the required range.
Gary: There's another issue about changing that to a non-negative signed long.
PROPOSAL: Update the spec with the rounding process and update the tests to support the new spec change, then wait for implementations to update based on this change.
Gary: Any objections?
… No objections.
Resolved: Update the spec with the rounding process and update the tests to support the new spec change, then wait for implementations to update based on this change.
github: https://github.com/w3c/webvtt/issues/464
Gary: This is similar to regions in that Safari has the older spec implementation.
… Safari and Firefox have it. Only some HTML character entities are whitelisted.
… More recently all HTML character entities are allowed.
… I guess the question here is, even given that not all character entities are supported, but the most popular ones are,
… would we be okay to move forward without waiting for implementation updates in this area?
Nigel: What was the motivation for the spec change to allow all entities? That's important for this discussion.
Gary: To align better with HTML and to allow people to use those character entities, especially since WebVTT has HTML
… syntax so if you are relying on that for special characters then you want to make sure your text is rendered appropriately.
Nigel: Are there workarounds for authors?
Gary: You'd be able to put the UTF-8 value in.
David: We changed to align with HTML on character entities. Is that a problem?
Gary: The implementations do not currently do that.
Eric: We haven't implemented the spec change.
Glenn: Named or character entities?
David: Named. We've always required support for numbered ones.
… There were I think 6 named ones before the change.
Eric: Yes, 6 exactly.
[discussion about numerical character references]
Glenn: That's strange, not to support all numeric character references.
Gary: Yes it was strange.
Glenn: There should be a way to get every numerical character reference into the document.
Gary: WebVTT is defined as UTF-8 so you can always include the actual character.
Eric: It's a bug in the implementation, it should be fixed.
Gary: Should this block CR exit?
Glenn: In TTML we only require only the XML spec-defined named character references, so having all the named ones
… from HTML seems redundant in my opinion, so I don't have any issue with limiting it to a small number.
… But there is an HTML compatibility issue if people think they should be able to use all the same entities.
… There may be a surprise factor if HTML named entities don't work.
Nigel: +1
Andreas: Your proposal is to wait for implementations, Gary?
Gary: My question was actually should we not wait for implementations but just move forward with the spec as is.
Andreas: So there will be spec text where you don't expect implementation?
… Why would you not revert the spec to match what is implemented?
… If you leave it in the spec then implementers will expect to use it.
Gary: Chrome does do the new spec behaviour.
Eric: Either leaving as is or reverting will require a change in the browser.
… It's more useful to allow them.
… The question is if it is worth blocking CR exit based on incomplete implementation?
Andreas: I understand, the change is reasonable, it's just not fully implemented yet.
Nigel: What is the behaviour for an unrecognised named character entity?
Eric: It gets output as is, e.g. &foo; comes out just like that.
Andreas: I don't think it is worth blocking for this.
Eric: I agree it is not worth blocking for this.
Gary: There are bugs against Firefox and Safari for this.
<gkatsev> webkit entities bug
<gkatsev> firefox entities bug
PROPOSAL: No spec change, when exiting CR, for this particular test, lack of 2 passing tests does not signify non-implementation of any feature.
Gary: Any objections?
… No objections
Resolved: No spec change, when exiting CR, for this particular test, lack of 2 passing tests does not signify non-implementation of any feature.
Andreas: Question: if you use Unicode code points as the reference for correct rendering, sometimes if the font is not available at authoring time, the safest way is to type the character reference, otherwise you cannot specify what you want.
… Is there a test for numerical code point support?
Nigel: That's a question about the test suite.
Eric: Yes, and about the spec too.
Gary: Does the spec need to specify conversion of code points directly?
Nigel: The spec uses "HTML Character Reference" which includes numerical ones. Are you saying there needs to be a test for that?
Andreas: The Rec should match implementations, but if implementations don't support numerical character references
… then should we add tests?
Nigel: There's already a test for it
Nigel: It currently fails except on Chrome
… 7 entities tests pass on stable builds right now.
github: https://github.com/w3c/webvtt/issues/468
Gary: This test needs splitting.
… The settings line test tests both the line property and line alignment.
… Line alignment is a fairly new feature.
… It would be best to split the test into two, one for each of those features.
… Line Alignment has no support, except Firefox theoretically - I'm not sure if it does the right thing, it may.
… There's also an interesting bug in Chrome and Safari which is probably from their shared heritage where
… the line value of .5 or 5. is processed as 0.5 or 5.0 but the spec explicitly says this is invalid.
Nigel: That's an interop problem then?
Gary: Yes
Eric: I believe that this is a result of a spec change because that value used to be defined as a float.
… The old logic means Safari and Chrome were doing the right thing.
… When it was defined as a float then it made sense.
… Now it is an integer... [looks at code] ... line position used to be a -, a % or a digit at the beginning.
… If it was digit then it was a floating point number, so the . was valid because it was part of the number.
Nigel: Was a . treated as a digit? Starting with a . seems to be none of those.
Eric: In any case it's a bug in the implementations vs the current spec.
Gary: It sounds like the current spec is what we want.
Eric: Correct
Gary: The other weird thing, maybe an implementation issue is Safari defaults to -1 instead of auto.
… And line alignment is not supported.
… To me it sounds like the way that we want to proceed is wait for implementations to fix the bug with line,
… and mark line alignment as at risk and likely to be removed from this version. Then hopefully it makes sense to add it
… back in to v.next, because I think it will take too long for implementations.
Nigel: Marking as at risk means you remove the dependency on implementations.
PROPOSAL: Split line alignment from the rest of the line tests, into a new test.
PROPOSAL: Mark line alignment as at risk
PROPOSAL: Wait for implementation updates for line options, leave as not at risk in the spec.
Gary: Any objections to any of those?
Eric: Not from me
Gary: No objections
Resolved: Split line alignment from the rest of the line tests, into a new test.
Resolved: Mark line alignment as at risk
Resolved: Wait for implementation updates for line options, leave as not at risk in the spec.
github: https://github.com/w3c/webvtt/issues/469
Gary: This is the same as #468 exept there are no bugs with position itself.
… Proposal will be to separate position and position alignment tests and mark position alignment as at risk.
PROPOSAL: Split position alignment from the rest of the position tests, into a new test.
PROPOSAL: Mark position alignment as at risk
Nigel: What does this do?
[discussion about what position alignment does]
Gary: It's a bit confusing which is why I'm not particularly bothered about marking is as at risk.
Glenn: Is it marked as at risk?
Gary: Yes in the snapshot we're working on.
… I won't remove it immediately.
Glenn: Ok
Gary: Position alignment controls left/center/right alignment in the writing direction
Nigel: That's quite a big deal compared to line alignment.
Andreas: there's also "align" which does start/center/end/left/right
… I think the use case you have is to align the text according to the writing direction?
Nigel: Hmm yes I think so. Is this the equivalent of multiRowAlign in IMSC?
group: [um, possibly]
Andreas: Everyone found this hard to understand. I think the use case to align text is met by "align".
Nigel: I'm not going to object to marking as at risk a feature that nobody understands!
Andreas: We do need to check that the use cases are all met.
Gary: Chrome opened a bug on WebVTT that blocked them from implementing, 4 years ago or something like that.
… It does seem like granular positioning is useful. If we realise it is not necessary we can just leave it out.
Gary: Any objections?
Gary: No objections
Resolved: Split position alignment from the rest of the position tests, into a new test.
Resolved: Mark position alignment as at risk
github: https://github.com/w3c/webvtt/issues/455
Gary: WebVTT currently says that the element should use text-wrap: balance.
… The CSS WG said not to use because implementing is complex and nobody seems interested in implementing it.
… It seems like people are interested in the feature, but it seems complicated so we should mark as at risk and
… remove it until it ships in browsers.
PROPOSAL: Mark text-wrap: balance as at risk
Nigel: Is it implemented by anyone?
Gary: No
Eric: Makes it easy then
Gary: It's ambiguous because there's no algorithm specified, so in that case you may end up with incompatible implementations
Nigel: I would have gone further because there's no evidence that line length balancing per se is what people want.
… There's more evidence that grammatical line breaking is useful.
… The most pragmatic thing to do is to accept this proposal.
Glenn: It's not implemented except in high end typographical applications.
Gary: Any objections?
… No objections
Resolved: Mark text-wrap: balance as at risk
github: https://github.com/w3c/webvtt/issues/456
Gary: This has an implementation in Chrome but in Safari it is still prefixed.
… I think Firefox does not whitelist it.
… In speaking with CSS WG earlier in the week they suggested not to rely on vendor prefixed properties for exit criteria.
… Therefore I propose we mark this as at risk.
PROPOSAL: Mark text-combine-upright as at risk.
Nigel: So what can't authors do if this is removed?
Gary: They can't do tate chu yoko
Glenn: It doesn't prevent a smart implementation from mapping a couple of digits in Japanese into a combined character.
… But then the author could not hint not to do that.
Gary: Any objections?
… No objections
Resolved: Mark text-combine-upright as at risk.
github: https://github.com/w3c/webvtt/issues/457
Gary: This is the one that changes the line attribute to a long.
… This issue was because in the spec region lines is an unsigned long but implementations treat it as an unsigned int
… so we felt it would be easier to change the webidl to match the implementations rather than ask all the implementations
… to update.
… But we realised that there is no unsigned int in webidl.
… However I see that the spec disallows negative numbers explicitly.
… A signed long would be the correct range with the addition of disallowing negative numbers.
… So the idea is to mark it as a regular long and disallow negative numbers.
Eric: That seems fine.
Gary: Implementations won't change, the spec will.
PROPOSAL: Change VTT Lines to a signed long
Nigel: Tests?
Gary: The tests need to be updated to delete all the out of range values.
Nigel: Then those tests will pass.
Gary: I have a PR now that splits the values into allowed and disallowed ranges, so I just need to delete the disallowed values file in the PR.
Nigel: A PR into wpt?
Gary: yes
<gkatsev> WPT lines PR
Gary: Any objection to the proposal?
Eric: No
Gary: No objections
Resolved: Change VTT Lines to a signed long
Gary: [shows images of WebVTT region support in various implementations]
Nigel: Could you please summarise where we're up to?
Gary: We looked at the open IR issues.
… A lot of them require us to wait for implementation updates before we can exit CR.
Eric: Some of them
Gary: Other features, we have marked as at risk.
Gary: We have a plan of action for each of the issues for exiting CR.
Andreas: We have an open issue on subtitles for 360o but we also include in scope
… augmented reality (AR) and virtual reality (VR), not only 360o.
… I want to report about the results. IRT is a partner in the EU project Immersive Accessibility (ImAc)
… alongside several partners.
… I first want to talk about current implementations,
… and the requirements that came out of user tests.
… Then I want to go into the discussions we had at W3C so far,
… and then discuss with you the next steps.
… [shows demo]
Nigel: Aside to point out those access service icons were developed by Danish Radio (DR)
… and can be typed on a keyboard. I'm often asked to promote universal icons like that.
Andreas: 1st option is subtitles that are fixed on the display screen, not fixed on the 360 sphere or on the speaker.
… They stay in the same place even when moving around the 360o.
… If the source of audio is not in the field of view, the viewer will not know what the subtitle refers to.
… If you hear the sound you may recognise someone and turn to face them.
… Here we show an arrow pointing to the speaker.
Samira: Don't subtitles show the name of the speaker?
Nigel: Different editorial practices around the world, some do, some don't.
Andreas: Even someone who hears doesn't have the information always.
… It is new for captioning that you have the subtitle refer to a person in the scene but not in view.
Glenn: Are these subtitles burned in, in this demo?
… I want to know how it discovers the current orientation of the speaker.
Andreas: It cannot be burned in because of that, it's a good lead in to my next slide.
… [shows customisation options]
… I made a diagram to show how it works.
… (link to follow)
… Usually a flattened 2D version of the 3D video is transmitted
… For captions, you bring it together with the subtitle format, in this case IMSC, in an application,
… running here in the browser.
… Big issue also that Samira mentioned this week is the browser does not have support for 360o videos
… yet, so you need an external library.
Samira: In WebGL
Andreas: One is three.js, another is react360.
Samira: Also babylon.js, they all use WebGL.
Andreas: They take the video and subtitles together and project it on a sphere.
… This representation gets handed over to the XR devices using the VR API by W3C (may be called XR API).
… That API is supported natively in the browser.
… It also gives a back channel from the XR device about change of location.
… The application can then generate a different kind of WebGL.
Glenn: This new Tex Track API, not as sophisticated as WebGL obviously, could potentially be used t
… to hand the presentation task to the browser at a higher level.
… Right now the web application is formatting the IMSC, right?
Andreas: Yes, that's also an issue.
… Processing is in the hands of the implementer.
… There is no conformant way of presenting IMSC for this kind of media.
… They do not really acknowledge all the details of IMSC for example although it is in the container.
… Ideally, something like an isSpherical attribute on the video element that then gives native support.
Samira: Yes, then you get performance benefits, DRM and all the other benefits.
… This is my pie in the sky idea.
… I was thinking how nice it would be to have this.
… The browser could do the work for you.
Samira: The Media side fed back yesterday that it's a lot of work.
… It could be that AR is higher priority than 360o for a lot of folks.
Andreas: Mounir suggested that we just need two new attributes to the text track cue v2 api, but you also need the 360o video support.
… [Requirements]
… The consortium tried different options.
… 1. Tie subtitle to a position in the 3D environment. The user may miss this if they're not looking the right way.
… 2. Subtitles stay on a 2D plane and don't move.
… 3. Locate audio source "horizontally" - does not need vertical information.
… 4. Locate the audio source of the timed text both horizontally and vertically.
… [Standardisation efforts]
… Lots of discussions and activity to get attention.
… [lists calls and meetings in W3C] TPAC, M&E IG, TTWG, Immersive Web CG, GitHub, F2F, WICG, Breakouts
… This week was very good for discussions especially Samira's breakout session.
… The response was that there is no 360o support yet, natively, so hard to standardise how to deal with captions.
… Also not clear the best place for this work to be done.
… Some colleagues think Immersive Web CG for 360o in general.
Samira: Yes the Immersive Web CG have done a lot of work in their DOM overlay, which could be very good for subtitles.
Andreas: One option is to do it separately from the WebGL, send some separate information.
Samira: I need to take a closer look at how the overlay API looks.
Vladimir: Question on architecture slide. According to this the XR API gives you the location information.
… Then why do you split positional requirements for H vs H+V into two different requirements?
Andreas: It's easier, to minimise the requirements for this service.
Vladimir: The two implementations would be the same though.
Andreas: Yes but for the author it is easier just to point to a point on a circle than on a sphere.
… This can be discussed, I just reduced to the minimum that is definitely needed.
… It would be good to add that as it gets it back from the XR device.
… [Next Steps]
… From the discussions I think the whole ecosystem for 360 video is not mature enough to start a spec activity for captions and subtitles.
… I don't know what kind of application we are actually targeting.
… For the moment it is just user defined applications based on libraries, nothing for the web platform.
… I was possibly too optimistic in January when I wanted an extension module this year.
… It is not satisfying because at the moment a lot of 360o videos have no captioning and are not accessible.
… My proposal is to incubate, in a CG or WICG, I'm not sure.
… We need more than the people here in the room to solve the problem.
… We need people from Immersive Web.
… XR devices are different from this view in the browser. They have different constraints to consider.
… If we think that it is ready for starting a spec activity then it could be handed over to a WG and the TTML part would be
… dealt with here.
… The only problem is resource to follow up on this - there's not enough.
Samira: How do you know right now who is speaking and where to put the arrow?
Andreas: In the project there is a caption preparation tool provider who includes a tool for specifying the location and
… the name of the speaker.
Gary: How does it look in the caption format?
Andreas: Longitude and Latitude in angles.
Gary: Then it compares to the camera angle?
Andreas: Absolutely, yes.
Gary: Then is that the idea for the XR API to know the camera angle?
Andreas: I think so, yes
Gary: For WebVTT, to do it natively we would have to figure out how to do it properly.
… Understand the video is spherical, etc. A lot of moving parts.
Andreas: You get this information from the XR device session.
… At first the information looks easy, but then there are other things to consider,
… how to define normatively, or what behaviour to define.
Gary: When I played with 360o captions we gave the positional data out of band from the text tracks, but it could be in band.
… Because we also decided that it should only be on a single plane and not move, that meant you could have a normalised
… vector for the position, and compare it. It sounds like you don't need anything more than that.
Andreas: No, it would be interesting to compare implementations.
… Process wise, I have no proposal. In my desperation a bit I suggested on Monday if there are not enough people
… to do it in a CG, just do it in a private repo, put an MIT licence on it, and do it there.
… We make a lot of fuss about it and use time.
… Ideally there would be a Web Immersive CG 360 repo we could use to discuss this kind of stuff.
… There's a new CG proposed by Google called immersive Captions but the Immersive Web CG doesn't know about this activity.
… There were 5 supporters so it has started, so there would also be the alternative of doing it there.
… Views?
Samira: You can also put your comment in the issue I opened.
Andreas: I didn't comment on that. We opened another issue on December and it was hard to get attention on it.
Samira: Native support is probably not in the near future but as a middle ground I could create a web component.
… Then the subtitles can be a part of that.
Andreas: Wouldn't it be a good idea to build the web component based on something in a repository rather than an issue?
Samira: Yes I need to find out how to get the issue promoted.
Nigel: I would propose a TTML extension that simply states the related 3D location for a bit of content.
Andreas: My opinion is we need more expertise outside the group, and the better way might be to generate a proposal
… for WebVTT and TTML in a CG and then bring it here.
Gary: If nobody else cares then we should not spend effort on it.
Andreas: My proposal is to bring it when we have something more concrete.
Nigel: Nobody will object to doing less work here!
Andreas: Here, it seems like both TTML and WebVTT extensions are of interest.
… So we should combine the effort.
Gary: It doesn't seem like a lot of work but we shouldn't do it if nobody will use it.
Nigel: We should change the status of the issue on tt-reqs to say it is on hold for the time being.
Andreas: I am happy to do that.
github: https://github.com/w3c/tt-reqs/issues/8
SUMMARY: This issue on hold for the time being; pick it up again when there's a concrete proposal e.g. from a CG.
Andreas: I'd like to demo some work we've been doing on an IMSC editor.
Andreas: Open source, based on vue.js and imsc.js.
… The idea is to have a complete editor based on everything IMSC supports.
… We parse into javascript, manipulate it, and create TextTrackCues for every ISD with onenter and onexit events
… to overlay HTML fragments and remove them again.
… What I want to show with this editor is that the complexity and feature richness of IMSC is good but not
… easy to handle in an authoring environment.
… We tried to solve it by making the interface customisable, in layout by config or extension.
… We started with a plain layout with styles to manipulate on each layer, but that may not be very approachable.
… So we chose a more up to date layout based on bootstrap.
… Then we can manipulate the different styles based on the content element level.
… It is also possible to configure the tabs e.g. adding more style attributes to get just the features that are wanted for the use case.
… Then you can change whatever you like to change, including more advanced things like text shadow.
… We made regions draggable and resizable.
… We struggled a bit with fullscreen mode, and implemented it.
… That's mostly it.
… What we struggled with is the different layers in content elements in TTML, and guiding the author
… through what they need to change.
… We didn't implement nested spans, which may not be very easy to display.
… We did not add any possibility to change timings and add content because it is very difficult
… to keep the authoring data structure in sync with the text track data structure.
… After we have parsed the document we have taken the ISD times.
… If you add content or do timing changes you have to reparse the entire document and regenerate the cues.
… The authoring happens not on an ISD basis but on cue generation but they are two different concepts.
… This makes it hard to deal with.
Nigel: I've never seen an editor that tries to offer the full space of IMSC's options.
… Normally the editor is more constrained to a use case, but that does mean if you try to load a generic
… file then you may find that there are options that don't fit the model of the editor, and you have to decide what to do.
Andreas: If you're interested, the general approach and what we are using it for is to help our broadcasters develop new styles of subtitles.
… We created a demo for creative subtitles with emojis for speaker id instead of colours.
… We used different fonts to enable certain features useful for deaf and hard of hearing.
… You need to know the context of if the character is reading the letter or speaking to the person next to her.
… We used a Google font for the typed letter font.
… Credit to my colleague Melina Mohnle
… She developed a lot of this and is hard of hearing herself.
<atai> s/Melena Muhnle/Melina Mohnle/
Nigel: My summary is there is nothing for us to do right now.
… We learned more about the danmaku use case and how it is implemented,
… and they learned a bit more about the scope and capabilities of timed text formats,
… and they need to think more about exactly what they want to achieve in terms of standardisation,
… and then depending on what they come back with we can proceed from there.
Michael: We have the same sentiments, largely, we don't have anything more yet.
… Mr Song invited us to have a Skype meeting to plan out where we're going with this, next week.
Glenn: I have some personal thoughts.
… There are many things that I saw today that you could accomplish with TTML2.
… TTML2 adds support for continuous animations.
… It adds support for downloadable fonts.
… It adds support for downloadable images and audio, and also all of those fonts, images and audio can
… not only be referenced by URL but can also be embedded with base64 encoding.
… With the animation support and embedded content support you can do many things.
… However at the same time if I were implementing something like I saw today I would probably
… use a combination of Javascript and some declarative payload format that can be parsed by that code,
… as a way to create some interoperability, for example
… instead of creating a common API if you created a common library that could be shared
… by different vendors supporting this service, and also a common payload format that
… could be consumed by that library then that library could translate to HTML and CSS or possibly the new TextTrackCue
… API to download the rendered format to the user agent so it could perform the display.
… The two things that I think would be important to keep in mind are
… accessibility and translatability. Given that many user agents support translation and accessibility, sometimes with plugins,
… for example screen readers, then making sure that the solution could support those would be important.
Gary: I think that the next step, for Chinese IG is to do a gap analysis and provide specific things that they are trying
… to do and if they are already supported by the web platform, and what is lacking that is being done manually now.
… I think Eric's idea makes sense, where you probably need HTML and CSS animations, potentially new animations that
… don't exist currently. That would be the next step, to figure out where those gaps are.
Michael: I believe they did do something of a gap analysis in the document.
Gary: It needs to be more specific, saying for some particular requirement technique X can be used or no technique is available
… e.g. it can use TTML2 continuous animation or it cannot.
Nigel: I support the point about accessibility, I couldn't see how it could be done, but I wouldn't rule it out!
… I also thought it was interesting that it can work on static images, and wonder what the event model is, if
… the same danmaku always appears a fixed time after loading the page, or if it is clock based, so everyone sees the same thing at 1500.
… There may be something that does not fit well with Timed Text potentially.
Cyril: One question. Do you expect that other types of devices will render it the way the browser does now?
… That has an impact on what you want to standardise.
… If everything can be done in the browser there may be no need to standardise anything.
… But if you want to exchange it with a TV or a non-web-based player, maybe there's a need for a format.
… I agree with what has been said. From what I saw, I think everything is possible with TTML2, I don't know if it is too
… complex.
Michael: One thing I forgot to mention earlier, in relation to TTML.
… One thing that is very important to us is the ability to add and delete subtitle elements live, and to do so
… without having to load or reload large amounts of data, or buffer only a few at a time, if a video has 10s of thousands
… of comments, we don't want to load them all at once.
Glenn: That's usually a decision made in packaging and transmission. Lots of strategies are available.
… It's not a function of TTML itself. For example, a service provider could deliver each caption as a separate
… TTML2 file and it is possible to have multiple TTML2 files synchronised to the same content, or asynchronous.
… It is part of the application layer that uses TTML2 to make those decisions. The TTML2 language does not provide
… any constraints. There are different time bases, media and clock, for example, so you can specify a clock time, or a
… media time. It's part of the semantics of the application at a higher level than TTML itself.
Michael: Any overhead sending 10,000 comments to the player as separate TTML2 files over the duration of the programme?
Glenn: Sure, you can construct a very small TTML file. It doesn't need to be very large.
Nigel: If you're sending 10,000 of anything there's going to be overhead!
Nigel: There is also the ability to send live subtitles in TTML, in temporal chunks, so you would send only the content
… that is relevant to, say, a 5s chunk of video, and that allows the data to be loaded more quickly for that 5s.
Michael: Thank you for your input.
Atsushi: I'm not sure what the Chinese IG wants to do.
Nigel: I don't think there's anything to follow up on right now for this?
group: nothing more to add now
nigel: this work is one of the things we agreed to do this year
… there is a TT requirement for it
… we have a repo
<nigel> ADPT repo
nigel: with a draft spec
<nigel> draft spec
nigel: it needs some editing work
… comes out of the CG
… quite a small document
… defines no extension
… simple profile
… has examples
… scope is to support the process of audio description
… from writing the script to mixing the audio
… the audio could be mixed before delivery to a player or mixed in the player
… the text, which is the script, can be made available to the player
… in our implementation, the text is given to the screen reader of the player
… [shows an example with the audio attached]
… [shows example with text to speech]
… it needs to allow for controling the audio mixing by animating the gain
… the gain only affects the input to the span in this example
… [discussing the fill=freeze impact]
… the profile looks very similar to IMSC
cyril_: you could follow the model of IMSC and indicate only the feature that are permitted
nigel: yes, we should do it
cyril_: if we add a new feature, it might be ambiguous
nigel: just support media time base
cyril_: do you have a profile identifier?
nigel: no
cyril_: what about the designator?
nigel: not yet defined
cyril_: you can also recommend a value for the registry identifier
nigel: the document contains lots of informative text (requirements)
nigel: in terms of implementers, we've got several
… one that creates the files and one that consumes it
nigel: I'm confident we'll pass the exit criteria
cyril_: do you have a beginning of a test suite?
nigel: there are no new features
cyril_: what are the exit criteria for a profile document
nigel: I don't know
nigel: there are some issues open
… we agreed as part of the charter to work on this document
… what I need to do is do the editorial fixes
… and then publish it as FPWD
… there is a need for HR probably
… the deadline to get a document ready could be Nov 1st
cyril_: about the profile signaling, you should use the same constraints given in IMSC
nigel: good feedback
<nigel> Pull request about support for contentProfiles and processorProfiles
cyril_: could we envisage an update of the profile to cover dubbing scripts
Nigel: Sure, the #metadata feature is permitted, so it doesn't need anything except an example, which you can propose.
Cyril: The dubbing script looks like it could be very similar.
Nigel: Yes, maybe we shouldn't call it AD, but change the name to something more generic.
<cyril_> nigel: ok, then let's try to produce FPWD for Nov 1st
nigel: we've done 4 joint meetings
nigel: done everything needed for IMSC 1.2
… reviewed all TTML2 issues identified
… worked on Karaoke
… and CJK extensions
… worked out our CR exit strategies for WebVTT
… learned about Danmaku
… decided not to do anything for the time being on 360 video
… and a first review of the Audio Description profile
… in terms of goals, we decided not to progress TTML3 for the time being
… we can thank every one and adjourn
<nigel> s|s/Melina Muhnle/Melina Mohnle/||
<nigel> s/Melena/Melina
<nigel> Log: https://www.w3.org/2019/09/20-tt-irc