Multimodal Authoring systems
Discussion on what is a valued tool and what is desired.
Semantic Binding
How clients and servers choose to use a specific modality and what that might mean.
Requirements:
Time lines- semantic support into the markup.
Note this is to develop APPLICATIONS
Dialogue steps:
Dialogue steps- The story board approach
Data model
Interactions
Presentation
Video, audio, etc.
Binding input modalities to
Events, interactions, etc.
Action rules for state control
Introduction entry to shop
Selection of items- choose cd type
Preview of choice
Put in shopping basket
Buy
Leave store
AS’s or run time agents all seem to require some magic to link inputs, data, flow control etc.
This is typically done by specific code being written – scripts, java …and no help exists.
There may be other equally valid AS’s.models.
Persistent semantic information across layers or states.
Where is lower left?
Procedural code limits the system deciding where they are bound, - in the server, client etc.
Who controls binding
Author
System- client / server
Event extensions
To respond/understand connection/disconnected
Lack of Modality Preferences
Choices
Required limitations
Fixed single modality ( eg. Blind )
Prefer Voice
Mix Visual first, Aural next, etc.
Requires
Exposed profiles
Choice of sync points, or presentation, or action rules, by profile.
Should Allow output for transcoders to add value.
Async support.
Also requires message passing including push capabilities
Note:
Authoring tools will also require control support for handling async events for clients and parameterisation for senders.