Multimodal Interaction Activity


  • 22 May 2014: Emotion Markup Language (EmotionML) 1.0 is a W3C Recommendation
    The Multimodal Interaction Working Group has published a W3C Recommendation of Emotion Markup Language (EmotionML) 1.0. As the Web is becoming ubiquitous, interactive, and multimodal, technology needs to deal increasingly with human factors, including emotions. The specification of Emotion Markup Language 1.0 aims to strike a balance between practical applicability and scientific well-foundedness. The language is conceived as a “plug-in” language suitable for use in three different areas: (1) manual annotation of data; (2) automatic recognition of emotion-related states from user behavior; and (3) generation of emotion-related system behavior. Learn more about the Multimodal Interaction Activity.

More News ...

Why Multimodal Interaction?

Multimodal interaction offers significant ease of use benefits over uni-modal interaction, for instance, when hands-free operation is needed, for mobile devices with limited keyboards, and for controlling other devices when a traditional desktop computer is unavailable to host the application user interface. This is being driven by advances in embedded and network-based speech processing that are creating opportunities for integrated multimodal Web browsers and for solutions that separate the handling of visual and aural modalities, for example, by coupling a local HTML5 user agent with a remote speech service.

Target audience >>

Current Situation

The goal of the Multimodal Interaction Working Group is to provide standards that will enable interaction using a wide variety of modalities. These modalities include both those currently available, such as touch, keyboard and speech, as well as emerging modalities such as handwriting, camera, and accelerometers.. Because of the ever-expanding set of interaction modalities, the group has focused on a generic architecture that defines communication between modality components and an interaction manager, based on standard life cycle events. This architecture is described in the Multimodal Architecture and Interfaces specification. The group is now launching a complementary work item to address the areas of registration and discovery of MMI Architecture components.

The details of the interpretation of user input captured by the various modalities and sent to the Interaction Manager are expressed using the Extensible Multi-Modal Annotation (EMMA) specification. The Working Group is also working to address the underlying representation of two basic forms of user input-- ink and emotion, because there were no existing standards for these forms of input. The Ink Markup Language (InkML) standard describes how ink and gesture inputs can be represented in XML, and the Emotion Markup Language specification describes an XML representation for emotion.

The work of the Multimodal Interaction Working Group is applicable to a wide variety of types of interactions -- not only interactions with the traditional desktop browser and keyboard, but also in mobile contexts. In addition, the work also applies to use cases where the devices involved, such as household appliances, automobiles, or televisions, have very diverse forms of displays and input controls.

The Working Group is chartered through 31 July 2013 under the terms of the W3C Patent Policy (5 February 2004 Version). To promote the widest adoption of Web standards, W3C seeks to issue Recommendations that can be implemented, according to this policy, on a Royalty-Free basis.

The Working Group is chaired by Deborah Dahl. The W3C Team Contact is Kazuyuki Ashimura.

We want to hear from you!

We are very interested in your comments and suggestions. If you have implemented multimodal interfaces, please share your experiences with us, as we are particularly interested in reports on implementations and their usability for both end-users and application developers. We welcome comments on any of our published documents on our public mailing list archive. To subscribe to the discussion list send an email to www-multimodal-request@w3.org with the word subscribe in the subject header. Previous discussion can be found in the public To unsubscribe send an email to www-multimodal-request@w3.org with the word unsubscribe in the subject header.

How to join the Working Group

New participants are always welcome.If your organization is already a member of W3C, ask your W3C Advisory Committee Representative (member only link) to fill out the online registration form to confirm that your organization is prepared to commit the time and expense involved in participating in the group. You will be expected to attend weekly Working Group teleconferences, all Working Group face to face meetings (about 2 or 3 times a year) and to respond in a timely fashion to email requests. Further details about joining are available on the Working Group (member only link) page. Requirements for patent disclosures, as well as terms and conditions for licensing essential IPR are given in the W3C Patent Policy.

More information about the W3C is available, as is information about joining the W3C.

Patent Disclosures

W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent.