WYSIWIG Captioning with IMSC
Presenter: Andreas Tai
Demo from Timed Text Working Group
IMSC offers many options to enhance the production of subtitles and captions. The demo will show some of this potential by using an open source IMSC subtitle editor.
Transcript
My name is Andreas Tai.
I work for IRT and I am a member of the Timed Text Working group.
I show an implementation of the Internet Media Subtitling and Caption profile, better known as IMSC.
IMSC is a subset of the Timed Text Markup Language and referenced by many video and broadcast standards.
At IRT we built an open source IMSC subtitle editor because professionals came to us saying: It is great to have a feature rich subtitle format but where are the tools so I can use these features.
So one of the goals has been to demonstrate the potential of IMSC.
For this demo I picked four features: Unicode, relative length metrics, free positioning and the use of different fonts.
In web development these features may seem trivial but in subtitle workflows they are rarely used because of technical limitations.
IMSC supports Unicode.
This is of course a breakthrough for internationalization but it also opens the door for new creative ways to produce subtitles.
In subtitles for deaf and hard of hearing you need to identify the speaker.
If you do not have access to the audio it is often not possible to relate the subtitle to the speaking person.
One option is to add a dash at the beginning of a subtitle line whenever there is a speaker change.
But this is often not unambiguous.
An alternative way to identify the speaker could be the use of an emoji.
Because of the Unicode support in IMSC I can use a baseball cap emoji in the subtitle and relate it to a person with a baseball cap in the picture.
An emoji picker in the editor makes it easy to add an emoji of choice.
Another strategy for speaker identification is positioning.
You can position the caption close to the speaker.
In IMSC you can position a subtitle freely and reliably.
In the subtitle editor you can change the IMSC values directly or you can use drag and drop.
Because IMSC length metrics that are relative to the video dimensions the position is kept even when the size of video changes.
But sometimes you need an even finer control.
Imagine two friends are sitting together and talking to each other.
One of them got a letter and is reading it out loud.
From the text alone it cannot be inferred if the friend is reading the letter or speaking to a friend.
IMSC allows you to use a font family off your choice.
To distinguish the reading situation from the informal conversation you could use different fonts.
For the reading situation you could use for example a typewriter font.
We named our software and imscED for IMSC editor.
imscED builds on top of the IMSC renderer imscJS.
This open source software is maintained by Pierre Lemieux and is an important reference implementation for IMSC.
imscJS parses an IMSC XML file and translates it into a JavaScript object.
For any point in time in the video playtime It is then possible to generate a CSS styled HTML DOM fragment as subtitle representation.
This can be used as video overlay.
The subtitle editor imscED uses not only the parsing and rendering features from imscJS.
It also uses the generated JavaScript object to change style properties or to add new content.
This works pretty well and demonstrates that IMSC is not tied to XML but also works as content model in other languages.
The open sourcec project can be found at github.com/IRT-Open-Source/imsced The current build is available at subtitling.irt.de/imsced As a heads up: although the editor is fully functional we still regard it as an early alpha release.
Thanks for your attention to look at this demo.
