Meeting minutes
https://
AMPL
<kaz> (Advanced Multimedia Processing Language)
Adam: It's under development, it's based on HTML, JS. Related to SMIL
… It supports sequential content, supports multiple video streams combined into one output stream
… Place logo onto a video
… Choice points include switch/case and goto are in the markup language
… Example of an intro video with menu
… Paths converge on an outro video
… Choice points may not use a menu, but instead be JS based
… Education scenarios, different application logic and user context
… Metadata to provide context for users
… Metadata can describe the context and framing of the choices presented
Attaching semantics to video
Adam: Spatial bounding boxes, auxiliary video tracks containing output of computer vision algorithms
… Animated silhouettes. Can be referred to by URI
Multipart Data Markup
Adam: Can enhance i18n
… We'll consider those approaches on the menu markup language
… Accessibility, multimodality will be considered
Web Remote Protocol
Adam: AMPL servers could render to one or more video streams for HTML5 video or WebRTC
… Stream interactive multimedia content
… More efficient encoding than traditional video
… Instructions for layout boxes, stream back input events for mouse and touchscreen
… More advance than existing remote desktop software
… Can be used for optimising WebRTC
… 3D game engines and CAD
… Can stream, modify, delete layout boxes
… Layout boxes on top of the video could contain hyperlinks
… Layout tracks can use new protocols, style and position, animate on the screen
Publishing
Adam: Interactive fiction is a use case
Media
Adam: Could focus on hypertext overlays, e.g., lower thirds
… Example of two video streams, replace overlay video with hypertext
Live streaming
Adam: Important use case, e.g., live news ticker
AI
Adam: Speech recognition, could ensure interop with simple or advanced dialog systems
… Synchronisation between resources enable chatbots
… Synchronsation with dialog systems involve time-based metadata
… The CG is interested in all forms of interactive media. The group discusses how standardisation can help
… I'd like to invite your participation
<AdamSobieski> https://
Q&A
Zheng: It's interesting to hear about AMPL. It's trying to solve a lot of problems. What are the issues with current HTML5 or digital publishing that are solved?
Adam: AMPL is a general purpose tool. It uses HTML and CSS
… I've been focusing on synchronising clients and servers, animations on top of videos for clients. Hypervideo
Zheng: Any existing prototypes?
Adam: Not yet, it's quite new
Kaz: It seems to me that this possible too generic. There were several generic frameworks in W3C such as MMI Architecture and SCXML.
… I'd like to see the typical target use cases and a gap analysis with existing frameworks, to identify what's expected and required
… And then see what kind of concrete data model is needed
Adam: Interactive video is a first use case
Kaz: You've also input to the WoT group, there's potential integration with video content and hardware for playback or recording. Could think of that kind of use case to identify requirements
Leonard: Interesting presentation. Have you thought about how the group might try to choose between potential directions?
… Any one could be a significant investment. What are your thoughts on the process?
Adam: We'd use prefixes on the mailing list to distinguish topics. I'm hoping to work on the menu markup language
Leonard: This assumes AMPL is a foundational technology
Adam: Optimistically yes, needs a markup for menus and multimodality. I'm putting the technology forward as an option
Marisa: I'm working on accessbility in multimedia. Could you switch on rich content such as caption tracks. Introduces authoring complexities. Where does that sit in the plan?
Adam: Focus on menus
Wendy: Does the CG have a use cases document yet?
Adam: Not yet, that would be something to do early on
<Karen> +1 use cases document
Wendy: I think that would help to help alignment
Zheng: Interactive video is interesting. Netflix and YouTube. Users can choose a story, is that a vision here?
Adam: Yes, that's a use case we can include
cpn: In my org we've been working on a technology called object based media
… I think there is some overlap, branching narratives, responsive overlays on videos
… more adaptive to devices
… have you established any other connections to communities interested in these topics?
AdamSobieski: I've reached out to Twine, an interactive fiction group
… at BBC there's a tool called Story Former, interactive video and film, I'm new to that community as well
Craig: Use case of accessible menus and choice points
… have you played around with implementing that in modern browsers
… what limitations did you encounter?
AdamSobieski: I haven't started yet aside from visualziing
… it's just getting started, no foreseeable obstacles
… just a standard way to make a menu
zheng_xu: In the presentation, in some parts it looks like there is overlap with SMIL
… how to map text or other media to audio to read to the user
… have you looked at this?
AdamSobieski: We would definitely be referencing the SMIL standard in our documents
… some of the differences include using JS and CSS
cpn: Another question, where we see needs for interoperability
… one aspect to consider, limitations in existing browsers for playback and rendering
… for things like SMIL, are you thinking we end up with a standardized runtime environment for interactive experiences
AdamSobieski: I was considering an AMPL player that uses all of the methods I presented, reference implementation for an AMPL player
leonardr: You mentioned before, one of the big issues in publishing land is around security
… one of the concerns there is presence of JS in EPUB
… presence of scripting in EPUB is considered problematic
… AMPL would use scripting, do you see it as required or optional?
… or declarative
AdamSobieski: In the example with a switch case, you could use declarative markup for the spine in an EPUB
… a non-linear graph
… Twine makes interactive fiction, I would be advocating for some JS, but I understand your points
leonardr: I look forward to those conversations
Craig: It feels like the logical first step is bridging the gap between the current standards and what we need
… review the current browser implementations and then use this as a use case to push some initiatives ahead
… a layer on top to address some of the security concerns
AdamSobieski: Looking forward to feedback from others
cpn: Where do we go next? What is the status of the CG?
AdamSobieski: The group is new and ready to join
… it's currently growing and we'll be working on structure and chairs and use case documents next
cpn: Could you share a link?
<AdamSobieski> https://
cpn: one thing I might recommend is to set a charter to establish a scope and goals for the group
… it seems broad, and drafting a charter can help scope the work
… there is a template somewhere
https://
AdamSobieski: I think a charter will be a useful first step
… in there we can describe the process for chairs, co-chairs, work items
… use case documents, AMPL, menus, and other interesting projects for standardization
zheng_xu: If there is anything the Publishing CG can help with, please let us know
… it's more focused on work with the EPUB3 WG and incubation
… use cases and documentation for digital publishing
AdamSobieski: Thanks
cpn: I'd like to echo that for the Media and Entertainment Interest Group, we are interested in media related use cases
… and also from the Media WG point of vieew
… if there is a gap analysis in the browser APIs, we can take that forward as well
… we welcome this new group and want to keep in contact
gpellegrino: I joined a little late, so you might have answered, I work in accessibility
… do you see use cases for accessible interactive media, aside from media overlays
… do you see possible accessibility issues
… anything that is not just text poses problems
AdamSobieski: My own interest pertains to education and educational technology, so accessibility is important
… speech synthesis and speech recognition, accessibility is key to our work
Charles: I'm curious about interactive video, having a live transcript that you can click on to select words or phrases and go to that part of the book and highlight it
… Work that Kim Patch has donw with WordPress plugins
Adam: Interesting, I can see how that would help with multimedia search. Something to consider
… We talked about dialog systems and synchronise to the video. There might be some overlap there
Wendy: What are the next steps? Sounds like you have a good idea of what to do with the CG
… I can share the link with the publishing community
cpn: Craig, any communities we can connect Adam with?
Craig: There's the BBC MakerBox community. This CG could be a helpful point of contact for us
Adam: Thank you. Everyone from MakerBox and StoryFormer is welcome
We've/Craig: We've We've published some things, got people to look at it, but need to look at the tech to support it
[adjourned]