W3C Ubiquitous Web Domain

Multimodal Interaction Working Group Charter

Status of this Document

Per section 6 Working Groups, Interest Groups, and Coordination Groups of the W3C Process, this charter, and any changes to it, take effect by way of an announcement to the W3C Membership via w3c-ac-members.

1. Mission

The Multimodal Interaction Activity is focused on developing open standards that enable the following vision:

The primary goal of this Charter is to develop W3C Recommendations that enable multimodal interaction with mobile phones and other devices with limited resources. For rapid adoption on a global scale, it should be possible to add simple multimodal capabilities to existing markup languages in a way that is backwards compatible with widely deployed devices, and which builds upon widespread familarity with existing Web technologies. The standards should be scalable to enable richer capabilities for subsequent generations of multimodal devices.

Users will be able provide input via speech, handwriting, or keystrokes, with output presented via displays, pre-recorded and synthetic speech, audio, and tactile mechanisms such as mobile phone vibrators and Braille strips. Application developers will be able to provide an effective user interface for whichever modes the user selects. To encourage rapid adoption, the same content can be designed for use on both old and new devices. People with new phones will get to experience the multimodal capabilities, while users with old phones will get to use the keypad and/or stylus in the same way as now.

The specifications developed by the Multimodal Interaction Working Group under this charter must fall within the scope defined in section 4, and should be implementable on a royalty-free basis, see section 9.

2. Target Audience

The Multimodal Interaction Working Group should be of interest to a range of organizations in different industry sectors:

Multimodal applications are of particular interest for mobile devices. Speech offers a welcome means to interact with smaller devices, allowing one-handed and hands-free operation. Users benefit from being able to choose which modalities they find convenient in any situation. The Working Group should be of interest to companies developing smart phones and personal digital assistants or who are interested in providing tools and technology to support the delivery of multimodal services to such devices.
Automotive Telematics
With the emergence of dashboard integrated high resolution color displays for navigation, communication and entertainment services, W3C's work on open standards for multimodal interaction should be of interest to companies working on developing the next generation of in-car systems.
Multimodal interfaces in the office
Multimodal has benefits for desktops, wall mounted interactive displays, multi-function copiers, and other office equipment, offering a richer user experience and the chance to use speech and pens as alternatives to the mouse and keyboard. W3C's standardization work in this area should be of interest to companies developing client software and application authoring technologies, and who wish to ensure that the resulting standards live up to their needs.
Multimodal interfaces in the home
In addition to desktop access to the Web, multimodal interfaces are expected to add value to remote control of home entertainment systems, as well as finding a role for other systems around the home. Companies involved in developing embedded systems and consumer electronics should be interested in W3C's work on multimodal interaction.

3. Context

The Multimodal Interaction Working Group was launched in 2002 following a joint workshop between the W3C and the WAP Forum. Relevant W3C Member contributions were received on SALT and X+V. The Working Group's initial focus was on use cases and requirements. This led to the publication of the W3C Multimodal Interaction Framework, and in turn to work on extensible multi-modal annotations (EMMA), and InkML, an XML language for ink traces. The Working Group has also worked on integration of composite multimodal input; dynamic adaptation to the user, device and environmental conditions; modality component interfaces; and a study of current approaches to interaction management. The Working Group is now being re-chartered for a further two years. Work on the System and Environment framework and sessions is being spun off to other W3C working groups.

4. Scope

All work items carried out under this Charter must fall within the scope defined by this section.

The aim is to develop W3C Recommendations that will enable the widespread deployment of multimodal applications on mobile phones and other devices, and to complete existing work on EMMA and InkML.

Multimodal Authoring Standards

These will be a combination of CSS, XML and scripting, designed to enable multimodal interaction on mobile phones in particular as well as other devices. The approach must allow simple multimodal capabilities to be added to existing markup languages in a way that is backwards compatible with widely deployed mobile devices, and which builds upon widespread familarity with existing Web technologies. The standards should be scalable to enable richer capabilities on subsequent generations of multimodal devices. This involves the separation of the user interface from the application, to enable different user interfaces according to the user's preferences and the capabilities available to the devices. Work is expected on:

Modality Component Interfaces

Future Web applications will allow developers to define applications in terms of markup of their own choosing, with the means to define the corresponding runtime behavior in terms of scriptable objects and shadow markup, such as SVG for visualization. To assist with realizing this goal, the Multimodal Interaction Working Group is tasked with extending the W3C DOM with platform and language neutral interfaces for:

Where practical this should leverage existing W3C work.

Integration of Composite Multimodal Input
This will define a basis for an interoperable treatment of composite multimodal input, for instance, a combination of speech and pointer gestures.
Coordinated Multimodal Output
There is lots of potential for coordinated multimodal output, for instance, the use of SSML for speech synthesis in synchronization with animated graphics expressed in SVG. The Working Group is tasked with investigating whether a framework for coordinated multimodal output can be defined in terms of a combination of existing standards, including SSML, SMIL, SVG and Timed-Text, and if it proves necessary, developing a corresponding specification in cooperation with the relevant W3C working groups.
Extensible Multi-Modal Annotations (EMMA)
EMMA is being developed as a data exchange format for the interface between input processors and interaction management systems. It will define the means for recognizers to annotate application specific data with information such as confidence scores, time stamps, input mode (e.g. key strokes, speech or pen), alternative recognition hypotheses, and partial recognition results etc. EMMA is a target data format for the semantic interpretation specification being developed in the W3C Voice Browser Activity, and which describes annotations to speech grammars for extracting application specific data as a result of speech recognition. EMMA supercedes earlier work on the natural language semantics markup language in the Voice Browser Activity.
InkML - an XML language for ink traces
This work item sets out to define an XML data exchange format for ink entered with an electronic pen or stylus as part of a multimodal system. This will enable the capture and processing of handwriting, gestures, drawings, and specific notations for mathematics, music, chemistry and other fields, as well as supporting further research on this processing.

5. Deliverables and Schedule

This Working Group is chartered to last until 31 January 2007. The first face to face meeting after re-chartering is planned to be held at the Technical Plenary in March 2005.

Here is a list of milestones identified at the time of re-chartering. Others may be added later at the discretion of the Working Group, provided they fall within the scope as described in section 4. The dates are for guidance only and subject to change.

Multimodal Interaction Authoring
1st Working Draft, December 2005
Last Call Working Draft, August 2006
Candidate Recommendation, December 2006
Recommendation, June 2007
Modality Component Interface Framework
1st Working Draft, December 2005
Last Call Working Draft, August 2006
Candidate Recommendation, December 2006
Recommendation, June 2007
Current Working Draft, September 2004
Last Call Working Draft, March 2005
Candidate Recommendation, September 2005
Recommendation, January 2006
Current Working Draft, February 2004
Last Call Working Draft, April 2005
Candidate Recommendation, October 2005
Recommendation, March 2006

6. Relationship with other activities

These are related activities that we may need to interact with in ways to be determined, for example, to ask them to review our draft specifications, and for us to take advantage of their work to fulfil our needs. Collaboration across working groups will be essential to realizing the mission of the Multimodal Interaction Activity.

6.1 W3C-related activities

These are W3C activities that may be asked to review documents produced by the Multimodal Interaction Working Group, or which may be involved in closer collaboration as appropriate to achieving the goals of the Charter.

This is incomplete, and additional links will be added before the Charter is sent out for AC Review to work on Compound Documents, and the System and Environment Framework etc.

6.2 External groups

This is an indication of external groups with complementary goals to the Multimodal Interaction activity. W3C has formal liaison agreements with some of them, e.g. OMA and VoiceXML Forum.

7. Membership, Meetings, and Logistics

To become a participant of the Working Group, a representative of a W3C Member organization must be nominated by their Advisory Committee Representative as described in the W3C Process. The associated IPR disclosure must further satisfy the requirements specified in the W3C Patent Policy (5 February 2004 Version).

Experts from appropriate communities may also be invited to join the working group, following the provisions for this in the W3C Process.

Each Working Group participant is expected to contribute 20%, or at least a day per week to this group.

All proceedings of the Working Group (mail archives, telecon minutes, ftf minutes) will be available to W3C Members.

Working Group participants are not obligated to participate in every work item, however the Working Group as a whole is responsible for reviewing and accepting all work items.

7.1 Email communication

The archived member-only mailing list w3c-mmi-wg@w3.org is the primary means of discussion within the group.

Certain topics need coordination with external groups. The Chair and the Working Group can agree to discuss these topics on a public mailing list. The archived mailing list www-multimodal@w3.org is used for public discussion of W3C proposals for Multimodal Interaction, and Working Group members are encouraged to subscribe. To subscribe send a message with the word subscribe in the subject line to www-multimodal-request@w3.org.

For discussions relating purely to speech, there is the public mailing list www-voice@w3.org. The archive is available online.

7.2 Group home page

The Working Group page will record the history of the group, provides access to the archives, meeting minutes, updated schedule of deliverables, membership list, and relevant documents and resources. The page will be available to W3C Members and Invited Experts, and will be maintained by the W3C team contacts in collaboration with the Working Group Chair.

7.3 Telephone meetings

A weekly one-hour phone conference will be held. The exact details, dates and times will be published in advance on the working group page. Additional phone conferences may be scheduled as necessary on specific topics. An IRC channel may be used to supplement teleconferences. Meeting records should be made available in a timely fashion, in accordance with the W3C Process.

7.4 Face-to-face meetings

Face to face meetings will be arranged 3 to 4 times a year. The Chair will make Working Group meeting dates and locations available to the group in a timely manner according to the W3C Process. The Chair is responsible for providing publically accessible summaries of Working Group face to face meetings, which will be announced on www-multimodal@w3.org.

8. Resources

8.1 Working Group participation

This is expected to be a large working group. At the end of the previous charter, the Working Group had 88 participants from 47 organizations. To make effective use of this number of people, work may be carried out in task forces following the W3C Process "Requirements for All Working, Interest, and Coordination Groups". We also expect a large public review group that will participate in the public mailing list discussions.

8.2 W3C Team involvement

The W3C Team will be responsible for the mailing lists, public and working group pages, and for liaison with the W3C communications staff for the publication of working drafts. W3C team members are expected to adopt the same requirements for meeting attendance, timely response and information disclosure as are required of W3C Members. The W3C Team expects to allocate the equivalent of 60% of a full-time person to this work for the duration of this working group. This time includes the Team Contact effort as well as additional participation.

9. Patent Policy

This Working Group operates under the W3C Patent Policy (5 February 2004 Version). To promote the widest adoption of Web standards, W3C seeks to issue Recommendations that can be implemented, according to this policy, on a Royalty-Free basis.

Copyright © 2004 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply. Your interactions with this site are in accordance with our public and Member privacy statements. This page was last updated on $Date: 2007/04/20 01:28:24 $