Workshops, Presentations, Tutorials, Papers and Demos




Multimodal Interaction Working Group Activities Overview

Multimodal Architecture and Interfaces Implementation

EMMA Introduction

This video explains the value of a common format for representing multimodal user input and introduces EMMA.

Press, Articles and Technical Papers

Multimodal Architecture

Multimodal Architecture and Interfaces Wikipedia page

The W3C Multimodal Architecture and Interfaces Standard, Deborah A. Dahl, J. on Multimodal Interfaces, Volume 7, Issue 3, November 2013.


From MIT SketchML to InkML or There and Back Again, Rui Hu and Stephen M. Watt, pp. 26-27, Proc. 10th IAPR International Workshop on Document Analysis Systems Short Papers, (DAS 2012), March 27-29 2012, Gold Coast, Australia, IAPR

A Collaborative Interface for Multimodal Ink and Audio Documents, Amit Regmi and Stephen M. Watt, pp. 901-905, Proc. 10th International Conference on Document Analysis and Recognition, (ICDAR 2009), July 26-29 2009, Barcelona, Spain, IEEE Computer Society.

Sharing Digital Ink in Heterogeneous Collaborative Environments, Birendra Keshari, Sriganesh Madhvanath, Manoj Prasad A, Muthuselvam Selvaraj and Stephen M. Watt, pp. 580-585, Proc. IAPR International Conference on Frontiers in Handwriting Recognition, (ICFHR 2008), August 19-21 2008, Montreal, Canada, CENPARMI Concordia University, ISBN 1-895193-03-6.

New Aspects of InkML for Pen-Based Computing, Stephen M. Watt, pp. 457-460, Proc. International Conference on Document Analysis and Recognition, (ICDAR), September 23-26 2007, Curitiba, Brazil, IEEE Computer Society.


CNET News EmotionML: Will computers tap into your feelings? August 30, 2010


Extensible Multimodal Annotation (EMMA) for Intelligent Virtual Agents, Deborah Dahl, 10th International Conference on Intelligent Virtual Agents in Philadelphia, September 20-22, 2010.


  • Openstream Multimodal Interaction use case demo (Macromedia Flash video).
  • InkChat Video: This video shows InkML being used in Microsoft Office 2010, an interactive ink chat application and in a browser with HTML 5. It was presented at the 2011 W3C Technical Plenary meeting.
  • Openstream Personal Health Record Management System: allows users to capture images, videos and use the convenience of speech, location and gesture to update and personal health information with authorized healthcare personnel. This solution combines several W3C and OSGi standards such as InkML, SCXML, HTML Speech etc.

Other Related Resources

Open Source SCXML Implementations

MMI Architecture Libraries