EMMA: Extensible MultiModal Annotation markup language

W3C Working Draft 16 September 2005

This version:

http://www.w3.org/TR/2005/WD-emma-20050916/

Latest version:

http://www.w3.org/TR/emma/

Previous version:

http://www.w3.org/TR/2004/WD-emma-20041214/

Editor in chief:

Michael Johnston, AT&T

Editors:

Wu Chou, Avaya

Deborah A. Dahl, Independent Consultant

Gerry McCobb, IBM

Dave Raggett, W3C/Canon

Copyright © 2005 W3C ^® (MIT , ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use rules apply.

---

Abstract

The W3C Multimodal Interaction working group aims to develop specifications to enable access to the Web using multimodal interaction. This document is part of a set of specifications for multimodal systems, and provides details of an XML markup language for containing and annotating the interpretation of user input. Examples of interpretation of user input are a transcription into words of a raw signal, for instance derived from speech, pen or keystroke input, a set of attribute/value pairs describing their meaning, or a set of attribute/value pairs describing a gesture. The interpretation of the user's input is expected to be generated by signal interpretation processes, such as speech and ink recognition, semantic interpreters, and other types of processors for use by components that act on the user's inputs such as interaction managers.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document is a W3C Last Call Working Draft for review by W3C members and other interested parties. Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This specification describes markup for representing interpretations of user input (speech, keystrokes, pen input etc.) together with annotations for confidence scores, timestamps, input medium etc., and forms part of the proposals for the W3C Multimodal Interaction Framework.

This document has been produced as part of the W3C Multimodal Interaction Activity, following the procedures set out for the W3C Process, with the intention of advancing it along the W3C Recommendation track. The authors of this document are members of the W3C Multimodal Interaction Working Group (members only).

This document was produced under the 5 February 2004 W3C Patent Policy. The Working Group maintains a public list of patent disclosures relevant to this document; that page also includes instructions for disclosing [and excluding] a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification should disclose the information in accordance with section 6 of the W3C Patent Policy.

Your feedback is welcomed. Comments on this document are due 28 October 2005; please send them to the public mailing list: www-multimodal@w3.org (public archives). See W3C mailing list and archive usage guidelines.

Table of Contents

• 1. Introduction
  • 1.1 Uses of EMMA
  • 1.2 Terminology
• 2. Structure of EMMA documents
  • 2.1 Instance data, data model and annotations
  • 2.2 Data model
  • 2.3 EMMA namespace prefixes
  • 2.4 Conformance
    • 2.4.1 Conforming EMMA Documents
    • 2.4.2 Using EMMA with other Namespaces
    • 2.4.3 Conforming EMMA Processors
• 3. EMMA structural elements
  • 3.1 Root element: emma:emma
  • 3.2 Interpretation element: emma:interpretation
  • 3.3 Container elements
    • 3.3.1 emma:one-of element
    • 3.3.2 emma:group element
      • 3.3.2.1 Indirect grouping criteria: emma:group-info element
    • 3.3.3 emma:sequence element
  • 3.4 Lattice element
    • 3.4.1 Lattice markup: emma:lattice, emma:arc, emma:node elements
    • 3.4.2 Annotations on lattices
    • 3.4.3 Relative timestamps on lattices
  • 3.5 Literal semantics: emma:literal element
• 4 EMMA annotations
  • 4.1 EMMA annotation elements
    • 4.1.1 Data model: emma:model element
    • 4.1.2 Interpretation derivation: emma:derived-from element and emma:derivation element
    • 4.1.3 Reference to grammar used: emma:grammar element
    • 4.1.4 Extensibility to application/vendor specific annotations: emma:info element
    • 4.1.5 Endpoint reference: emma:endpoint-info element and emma:endpoint element
  • 4.2 EMMA annotation attributes
    • 4.2.1 Tokens of input: emma:tokens attribute
    • 4.2.2 Reference to processing: emma:process attribute
    • 4.2.3 Lack of input: emma:no-input attribute
    • 4.2.4 Uninterpretable input: emma:uninterpreted attribute
    • 4.2.5 Human language of input: emma:lang attribute
    • 4.2.6 Reference to signal: emma:signal attribute
    • 4.2.7 Media type: emma:media-type attribute
    • 4.2.8 Confidence scores: emma:confidence attribute
    • 4.2.9 Input source: emma:source attribute
    • 4.2.10 Timestamps
      • 4.2.10.1 Absolute timestamps: emma:start, emma:end attributes
      • 4.2.10.2 Relative timestamps: emma:time-ref-uri, emma:time-ref-anchor-point, emma:offset-to-start, emma:duration attributes
    • 4.2.11 Medium, mode, and function of user inputs: emma:medium, emma:mode, emma:function, emma:verbal attributes
    • 4.2.12 Support for composite multimodality: emma:hook attribute
      • 4.2.12.1 Description of required annotation
      • 4.2.12.2 Simple example
      • 4.2.12.3 Possible integration algorithms
      • 4.2.12.4 More detailed example
    • 4.2.13 Cost: emma:cost attribute
    • 4.2.14 Endpoint properties: emma:endpoint-role, emma:endpoint-address, emma:port-type, emma:port-num, emma:message-id, emma:service-name, emma:endpoint-pair-ref attributes
      • 4.2.14.1 Reference to emma:endpoint-info element: emma:endpoint-info-ref attribute
    • 4.2.15 Reference to emma:grammar element: emma:grammar-ref attribute
    • 4.2.16 Reference to emma:model element: emma:model-ref attribute
  • 4.3 Scope of EMMA annotations
    • 4.3.1 Scope of EMMA annotations in sequential derivations
    • 4.3.2 Scope of EMMA annotations in composite derivations
• Appendices
  • Appendix A. XML schema
  • Appendix B. MIME type
  • Appendix C. emma:hook and SRGS
  • Appendix D. EMMA event interface
  • Appendix E. References
    • E.1 Normative references
    • E.2 Descriptive references
  • Appendix F. Changes since last draft
  • Appendix G. Acknowledgements

1. Introduction

This document presents an XML specification for EMMA, an Extensible MultiModal Annotation markup language, responding to the requirements documented in [W3C Requirements for EMMA]. This markup language is intended for use by systems that provide semantic interpretations for a variety of inputs, including but not necessarily limited to, speech, natural language text, GUI and ink input.

It is expected that this markup will be used primarily as a standard data interchange format between the components of a multimodal system; in particular, it will normally be automatically generated by interpretation components to represent the semantics of users' inputs, not directly authored by developers.

The language is focused on annotating the interpretation information of single and composed inputs, as opposed to (possibly identical) information that might have been collected over the course of a dialog.

The language provides a set of elements and attributes that are focused on accurately representing annotations on the input interpretations.

An EMMA document can be considered to hold three types of data:

• instance data

  Application-specific markup corresponding to input information which is meaningful to the consumer of an EMMA document. Instances are application-specific and built by input processors at runtime. Given that utterances may be ambiguous with respect to input values, an EMMA document may hold more than one instance.

• data model

  Constraints on structure and content of an instance. The data model is typically pre-established by an application, and may be implicit, that is, unspecified.

• metadata

  Annotations associated with the data contained in the instance. Annotation values are added by input processors at runtime.

Given the assumptions above about the nature of data represented in an EMMA document, the following general principles apply to the design of EMMA:

• The main prescriptive content of the EMMA specification will consist of metadata: EMMA will provide a means to express the metadata annotations which require standardization. (Notice, however, that such annotations may express the relationship among all the types of data within an EMMA document.)
• The instance and its data model are assumed to be specified in XML, but EMMA will remain agnostic to the XML format used to express these. (The instance XML is assumed to be sufficiently structured to enable the association of annotative data.)
• The extensibility of EMMA lies in the ability for additional kinds of metadata to be included in application specific vocabularies. EMMA itself can be extended with application and vendor specific annotations contained within the emma:info element.

The annotations of EMMA should be considered 'normative' in the sense that if an EMMA component produces annotations as described in Section 3, these annotations must be represented using the EMMA syntax. The Multimodal Interaction Working Group may address in later drafts the issues of modularization and profiling, that is: which sets of annotations are to be supported by which classes of EMMA component.

1.1 Uses of EMMA

The general purpose of EMMA is to represent information automatically extracted from a user's input by an interpretation component, where input is to be taken in the general sense of a meaningful user input in any modality supported by the platform. The reader should refer to the sample architecture in [W3C Multimodal Interaction Framework], which shows EMMA conveying content between user input modality components and an interaction manager.

Components that generate EMMA markup:

1. Speech recognizers
2. Handwriting recognizers
3. Natural language understanding engines
4. Other input media interpreters (e.g. DTMF, pointing, keyboard)
5. Multimodal integration component

Components that use EMMA include:

1. Interaction manager
2. Multimodal integration component

Although not a primary goal of EMMA, a platform may also choose to use this general format as the basis of a general semantic result that is carried along and filled out during each stage of processing. In addition, future systems may also potentially make use of this markup to convey abstract semantic content to be rendered into natural language by a natural language generation component.

1.2 Terminology

anchor-point

When referencing an input interval with emma:time-ref-uri, emma:time-ref-anchor allows you to specify whether the referenced anchor is the start or end of the interval.

annotation

Information about the interpreted input, for example, timestamps, confidence scores, links to raw input, etc.

composite input

An input formed from several pieces, often in different modes, for example, a combination of speech and pen gesture, such as saying "zoom in here" and circling a region on a map.

confidence

A numerical score describing the degree of certainty in a particular interpretation of user input.

data model

For EMMA, a data model defines a set of constraints on possible interpretations of user input.

derivation

Interpretations of user input are said to be derived from that input, and higher levels interpretations may be derived from lower level ones. EMMA allows you to reference the user input or interpretation a given interpretation was derived from, see semantic interpretation.

dialogue

For EMMA, dialogue can be considered as a sequence of interactions between the users and the application.

end point

In EMMA, this refers to a network location which is the source or receipient of an EMMA document. It should be noted that the usage of the term "endpoint" in this context is different from the way that the term is used in speech processing, where it refers to the end of a speech input.

gestures

In multimodal applications gestures are communicative acts made by the user or application. An example is circling an area on a map to indicate a region of interest. Users may be able to gesture with a pen, keystrokes, hand movements or sound. Gestures often form part of composite input. Application gestures are typically animations and/or sound effects.

grammar

A set of rules that describe a sequence of tokens expected in a given input. These can be used by speech and handwriting recognizers to increase recognition accuracy.

handwriting recognition

The process of converting pen strokes into text.

ink recognition

This includes the recognition of handwriting and pen gestures.

input cost

In EMMA, this refers to a numerical measure indicating the weight or processing cost associated with a user's input or part of their input.

input device

The device proving a particular input, for example, a microphone, a pen, a mouse, a camera, or a keyboard.

input function

In EMMA, this refers to use a particular input is serving, for example, as part of a recording or transcription, as part of a dialogue, or as a means to verify the user's identity.

input medium

Whether the input is acoustic, visual, or tactile, for instance, a spoken utterance is an example of an aural input, a hand gesture as seen by a camera is an example of a visual input, pointing with a mouse or pen is an example of a tactile input.

input mode

This distinguishes a particular means of providing an input within a general input medium, for example, speech, DTMF, ink, key strokes, video, photograph, etc.

input source

This is the device that provided the input, for example a particular microphone or camera. EMMA allows you to identify these with a URI.

input tokens

In EMMA, this refers to a sequence of characters, words or other discrete units of input.

instance data

A representation in XML of an interpretation of user input.

interaction manager

A processor that determines how an application interacts with a user. This can be at multiple levels of abstraction, for example, at a detailed level, determining what prompts to present to the user and what actions to take in response to user input, versus a higher level treatment in terms of goals and tasks for achieving those goals. Interaction managers are typically event driven.

interpretation

In EMMA, an interpretation of user input refers to information derived from the user input that is meaningful to the application.

keystroke input

Input provided by the user pressing on a sequence of keys (buttons), such as a computer keyboard or keypad.

lattice

A set of nodes interconnected with directed arcs such that by following an arc, you can never find yourself back at a node you have already visited (i.e. a directed acyclic graph). Lattices provide a flexible means to represent the results of speech and handwriting recognition, in terms of arcs representing words or character sequences. Different arcs from the same node represent different local hypotheses as to what the user said or wrote.

metadata

Information describing another set of data, for instance, a library catalog card with information on the author, title and location of a book. EMMA is designed to support input processors in providing metadata for interpretations of user input.

multimodal integration

The process of combining inputs from different modes to create an interpretation of composite input. This is also sometimes refered to as multimodal fusion.

multimodal interaction

The means for a user to interact with an application using more than one mode on interaction, for instance, offering the user the choice of speaking or typing, or in some cases, allowing the user to provide a composite input involving multiple modes.

natural language understanding

The process of interpreting text in terms that are useful for an application.

N-best list

An N-best list is a list of the most likely hypotheses for what the user actually said or wrote, where N stands for an integral number such as 5 for the 5 most likely hypotheses.

raw signal

An uninterpreted input, such as an audio waveform captured from a microphone.

semantic interpretation

A normalized representation of the meaning of a user input, for instance, mapping the speech for "San Francisco"into the airport code "SFO".

semantic processor

In EMMA, this refers to systems that can derive interpretations of user input, for instance, mapping the speech for "San Francisco"into the airport code "SFO".

signal interpretation

The process of mapping a discrete or continuous signal into a symbolic representation that can be used by an application, for instance, transforming the audio waveform corresponding to someone saying "2005" into the number 2005.

speech recognition

The process of determining the textual transcription of a piece of speech.

speech synthesis

The process of rendering a piece of text into the corresponding speech, i.e. synthesising speech from text.

text to speech

The process of rendering a piece of text into the corresponding speech.

time stamp

The time that a particular input or part of an input began or ended.

URI: Uniform Resource Identifier

A URI is a unifying syntax for the expression of names and addresses of objects on the network as used in the World Wide Web. A URI is defined as any legal anyURI primitive as defined in XML Schema Part 2: Datatypes Second Edition Section 3.2.17 [SCHEMA2]. In this specification URIs are provided as attributes to elements, for example in the emma:time-ref-uri attribute.

user input

An input provided by a user as opposed to something generated automatically.

2. Structure of EMMA documents

2.1 Instance data, data model and annotations

As noted above, the main components of an interpreted user input in EMMA are the instance data, an optional data model, and the metadata annotations that may be applied to that input. The realization of these components in EMMA is as follows:

• instance data is contained within an EMMA interpretation
• the data model is optionally specified as an annotation of that instance
• EMMA annotations may be applied at different levels of an EMMA document.

An EMMA interpretation is the primary unit for holding user input as interpreted by an EMMA processor. As will be seen below, multiple interpretations of a single input are possible.

EMMA provides a simple structural syntax for the organization of interpretations and instances, and an annotative syntax to apply the annotation to the input data at different levels.

An outline of the structural syntax and annotations found in EMMA documents is as follows. A fuller definition may be found in the description of individual elements and attributes in section 3 and section 4.

• EMMA Structural Elements (Section 3)
  • Root element: The root node of an EMMA document, the emma:emma element, holds EMMA version and namespace information, and provides a container for one or more of the following interpretation and container elements (Section 3.1)
  • Interpretation element: The emma:interpretation element contains a given interpretation of the input and holds application specific markup (Section 3.2)
  • Container elements:
    • emma:one-of is a container for one or more interpretation elements or container elements and denotes that these are mutually exclusive interpretations (Section 3.3.1)
    • emma:group is a general container for one or more interpretation elements or container elements. It can be associated with arbitrary grouping criteria (Section 3.3.2).
    • emma:sequence is a container for one or more interpretation elements or container elements and denotes that these are sequential in time (Section 3.3.3).
  • Lattice element: The emma:lattice element is used to contain a series of emma:arc and emma:node elements that define a lattice of words, gestures, meanings or other symbols. The emma:lattice element appears within the emma:interpretation element (Section 3.4)
  • Literal element: The emma:literal element is used as a wrapper when the application semantics is a string literal. (Section 3.5)
• EMMA annotations (Section 4)
  • EMMA annotation elements: These are EMMA annotations such as emma:derived-from, emma:endpoint-info, and emma:info which are represented as elements so that they can occur more than once within an element and can contain internal structure. (Section 4.1)
  • EMMA annotation attributes: These are EMMA annotations such as emma:start, emma:end , emma:confidence, and emma:tokens which are represented as attributes. They can appear on emma:interpretation elements, some can appear on container elements, lattice elements, and elements in the application-specific markup. (Section 4.2)

From the defined root node emma:emma the structure of an EMMA document consists of a tree of EMMA container elements (emma:one-of,emma:sequence, emma:group) terminating in a number of interpretation elements (emma:interpretation). The emma:interpretation elements serve as wrappers for either application namespace markup describing the interpretation of the users input or an emma:lattice element or emma:literal element . A single emma:interpretation may also appear directly under the root node.

To illustrate this here is an example EMMA document for input to a flight reservation application. In this example there are two speech recognition results and associated semantic representations of the input. The system is uncertain whether the user meant "flights from Boston to Denver"or "flights from Austin to Denver". The annotations to be captured are timestamps and confidence scores for the two inputs.

Example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:one-of id="r1" emma:start="1087995961542" emma:end="1087995963542">
    <emma:interpretation id="int1" emma:confidence="0.75" 
    emma:tokens="flights from boston to denver">
      <origin>Boston</origin>
      <destination>Denver</destination>
    </emma:interpretation>

    <emma:interpretation id="int2" emma:confidence="0.68" 
    emma:tokens="flights from austin to denver">
      <origin>Austin</origin>
      <destination>Denver</destination>
    </emma:interpretation>
  </emma:one-of>
</emma:emma>

Attributes on the root emma:emma element indicate the version and namespace. The emma:emma element contains an emma:one-of element which contains a disjunctive list of possible interpretations of the input. The actual semantic representation of each interpretation is within the application namespace. In the example here the application specific semantics involves elements origin and destination indicating the origin and destination cities for looking up a flight. The timestamp is the same for both interpretations and it is annotated using values in milliseconds in the emma:start and emma:end attributes on the emma:one-of. The confidence scores and tokens associated with each of the inputs are annotated using the EMMA annotation attributes emma:confidence and emma:tokens on each of the emma:interpretation elements.

2.2 Data model

An EMMA data model expresses the constraints on the structure and content of instance data, for the purposes of validation. As such, the data model may be considered as a particular kind of annotation (although, unlike other EMMA annotations, it is not a feature pertaining a specific user input at a specific moment in time, it is rather a static and, by its very definition, application-specific structure). Its specification in EMMA is optional.

Since Web applications today use different formats to specify data models, e.g. XML Schema, XForms, Relax-NG, etc., EMMA itself is agnostic to the format of data model used.

Data model definition and reference is defined in Section 4.1.1.

2.3 EMMA namespace prefixes

An EMMA attribute is prefixed with the EMMA namespace identifier if the attribute can also be used as an in-line annotation on elements in the application's namespace. Most of the EMMA annotation attributes in Section 4.2 are in this category. An EMMA attribute is not prefixed if the attribute only appears on an EMMA element. This rule ensures consistent usage of the attributes across all examples.

2.4 Conformance

2.4.1 Conforming EMMA Documents

A document is a Conforming EMMA Document if it meets both the following conditions:

• It is a well-formed XML document [XML] conforming to Namespaces in XML [XMLNS].
• It adheres to the specification described in this document (EMMA Specification) including the constraints expressed in the Schema (see Appendix A) and having an XML Prolog and root element as specified in Section 3.1.

The EMMA specification and these conformance criteria provide no designated size limits on any aspect of EMMA documents. There are no maximum values on the number of elements, the amount of character data, or the number of characters in attribute values.

2.4.2 Using EMMA with other Namespaces

The EMMA namespace is intended to be used with other XML namespaces as per the Namespaces in XML Recommendation [XMLNS]. Future work by W3C is expected to address ways to specify conformance for documents involving multiple namespaces.

2.4.3 Conforming EMMA Processors

A EMMA processor is a program that can process and/or generate Conforming EMMA documents.

In a Conforming EMMA Processor, the XML parser must be able to parse and process all XML constructs defined by XML 1.0 [XML] and Namespaces in XML [XMLNS]. It is not required that a Conforming EMMA Processor uses a validating XML parser.

A Conforming EMMA Processor must correctly understand and apply the semantics of each markup element or attribute as described by this document.

There is, however, no conformance requirement with respect to performance characteristics of the EMMA Processor. For instance, no statement is required regarding the accuracy, speed or other characteristics of output produced by the processor. No statement is made regarding the size of input that a EMMA Processor must support.

3. EMMA structural elements

This section defines elements in the EMMA namespace which provide the structural syntax of EMMA documents.

3.1 Root element: emma:emma

The root element of an EMMA document is named emma:emma. It holds a single emma:interpretation or EMMA container element (emma:one-of, emma:sequence, emma:group). It can also optionally contain a single emma:derivation element containing earlier stages of the processing of the input (See Section 4.1.2). It can also contain an optional single annotation element: emma:info and multiple optional emma:grammar, emma:model, and emma:endpoint-info elements. It can hold attributes for information pertaining to EMMA itself, along with any namespaces which are declared for the entire document, and any other EMMA annotative data. The emma:emma element and other elements and attributes defined in this specification belong to the XML namespace identified by the URI "http://www.w3.org/2003/04/emma". In the examples, the EMMA namespace is generally declared using the attribute xmlns:emma on the root emma:emma element. EMMA processors must support the full range of ways of declaring XML namespaces as defined by the W3C Recommendation "Namespaces in XML 1.1" [XMLNS]. Application markup can be declared in an explicit application namespace, or an undefined namespace (equivalent to setting xmlns="").

For example:

<emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma">
    ....
</emma:emma>

or

<emma version="1.0" xmlns="http://www.w3.org/2003/04/emma">
    ....
</emma>

3.2 Interpretation element: emma:interpretation

The emma:interpretation element holds a single interpretation represented in application specific markup, or a single emma:lattice element, or a single emma:literal element.

The emma:interpretation element can also be empty but it must be annotated with either emma:no-input="true" or emma:uninterpreted="true". If emma:interpretation is marked with emma:no-input="true" then it must be empty.

Attributes:

1. id a required xsd:ID value that uniquely identifies the interpretation within the EMMA document.

<emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma">
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="r1">
    ...
  </emma:interpretation>
</emma:emma>

3.3 Container elements

3.3.1 emma:one-of element

The emma:one-of element acts a container for a collection of one or more interpretation (emma:interpretation) or container elements (emma:one-of, emma:group, emma:sequence), and denotes that these are mutually exclusive interpretations.

An N-best list of choices in EMMA, such as a series of different recognition results in speech recognition, should be represented as a set of emma:interpretation elements contained within an emma:one-of element.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:one-of id="r1">
    <emma:interpretation id="int1">
      <origin>Boston</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>

    <emma:interpretation id="int2">
      <origin>Austin</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>
  </emma:one-of>
</emma:emma>

The interpretations must be sorted best-first by some measure of quality. The quality measure is emma:confidence if present, otherwise, the quality metric is platform-specific.

The emma:one-of element can appear within another emma:one-of element, allowing for easy combination of N-best lists from different devices or recognizers processing the same signal. This also allows for annotations which apply to a subset of an N-best list to be specified once on a emma:one-of element embedded within another emma:one-of element.

3.3.2 emma:group element

The emma:group element is used to indicate that the contained interpretations are from distinct user inputs that are related in some manner. emma:group should not be used for containing the multiple stages of processing of a single user input. Those should be contained in the emma:derivation element instead. For groups of inputs in temporal order the more specialized container emma:sequence should be used. The following example shows three interpretations derived from the speech input "Move this ambulance here" and the tactile input related to two consecutive points on a map.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:group id="grp"
      emma:start="1087995961542"
      emma:end="1087995964542">
    <emma:interpretation id="int1">
      <action>move</action>
      <object>ambulance</object>
      <destination>here</destination>
    </emma:interpretation>

    <emma:interpretation id="int2">
      <x>0.253</x>
      <y>0.124</y>
    </emma:interpretation>

    <emma:interpretation id="int3">
      <x>0.866</x>
      <y>0.724</y>
    </emma:interpretation>
  </emma:group>
</emma:emma>

The emma:one-of and emma:group containers can be nested arbitrarily.

3.3.2.1 Indirect grouping criteria: emma:group-info element

Sometimes it may be convenient to indirectly associate a given group with information, such as grouping criteria. The emma:group-info element can be used to make explicit the criteria by which members of a group are associated. In the following example, a group of two points is associated with a description of grouping criteria based upon a sliding temporal window of two seconds duration.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example"
    xmlns:ex="http://www.example.com/ns/group">
  <emma:group id="grp">
    <emma:group-info>
      <ex:mode>temporal</ex:mode>
      <ex:duration>2s</ex:duration>
    </emma:group-info>

    <emma:interpretation id="int1">
      <x>0.253</x>
      <y>0.124</y>
    </emma:interpretation>

    <emma:interpretation id="int2">
      <x>0.866</x>
      <y>0.724</y>
    </emma:interpretation>
  </emma:group>
</emma:emma>

You can also use emma:group-info to refer to a named grouping criterion using external reference, for instance:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example"
    xmlns:ex="http://www.example.com/ns/group">
  <emma:group id="grp">
    <emma:group-info ref="http://www.example.com/criterion42"/>
    <emma:interpretation id="int1">
      <x>0.253</x>
      <y>0.124</y>
    </emma:interpretation>

    <emma:interpretation id="int2">
      <x>0.866</x>
      <y>0.724</y>
    </emma:interpretation>
  </emma:group>
</emma:emma>

3.3.3 emma:sequence element

The emma:sequence element is used to indicate that the contained interpretations are sequential in time, as in the following example, which indicates that two points made with a pen are in temporal order.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:sequence id="seq1">
    <emma:interpretation id="int1" emma:medium="tactile" 
        emma:mode="ink">
      <x>0.253</x>
      <y>0.124</y>
    </emma:interpretation>

    <emma:interpretation id="int2" emma:medium="tactile" 
        emma:mode="ink">
      <x>0.866</x>
      <y>0.724</y>
    </emma:interpretation>
  </emma:sequence>
</emma:emma>

The emma:sequence container can be combined with emma:one-of and emma:group in arbitrary nesting structures. The order of children in the content of the emma:sequence element corresponds to a sequence of interpretations. This ordering does not imply any particular definition of sequentiality. EMMA processors may therefore use the emma:sequence element to hold interpretations which are either strictly sequential in nature (e.g. the end-time of an interpretation precedes the start-time of its follower), or which overlap in some manner (e.g. the start-time of a follower interpretation precedes the end-time of its precedent). It is possible to use timestamps to provide fine grained annotation for the sequence of interpretations that are sequential in time.

In the following more complex example, a sequence of two pen gestures in emma:sequence and a speech input in emma:interpretation are contained in an emma:group.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:group id="grp">
     <emma:interpretation id="int1" emma:medium="acoustic" 
         emma:mode="speech">
       <action>move</action>
       <object>this-battleship</object>
       <destination>here</destination>
     </emma:interpretation>

     <emma:sequence id="seq1">
       <emma:interpretation id="int2" emma:medium="tactile" 
           emma:mode="ink">
         <x>0.253</x>
         <y>0.124</y>
       </emma:interpretation>

     <emma:interpretation id="int3" emma:medium="tactile" 
         emma:mode="ink">
       <x>0.866</x>
       <y>0.724</y>
     </emma:interpretation>
   </emma:sequence>
 </emma:group>
</emma:emma>

3.4 Lattice element

In addition to providing the ability to represent N-best lists of interpretations using emma:one-of, EMMA also provides the capability to represent lattices of words or other symbols using the emma:lattice element. Lattices provide a compact representation of large lists of possible recognition results or interpretations for speech, pen, or multimodal inputs.

In addition to providing a representation for lattice output from speech recognition, another important use case for lattices is for representation of the results of gesture and handwriting recognition from a pen modality component. Lattices can also be uses to compactly represent multiple possible meaning representations. Another use case for the lattice representation is that it enables the association of confidence scores and other annotations with individual words within a speech recognition result string.

Lattices can be compactly described by a list of transitions between nodes. For each transition the start and end nodes need to be defined, along with the label for the transition. Initial and final nodes also need to be indicated. The following figure provides a graphical representation of a speech recognition lattice which compactly represents eight different sequences of words.

which expands to:

a. flights to boston from portland today please
b. flights to austin from portland today please
c. flights to boston from oakland today please
d. flights to austin from oakland today please
e. flights to boston from portland tomorrow
f. flights to austin from portland tomorrow
g. flights to boston from oakland tomorrow
h. flights to austin from oakland tomorrow

3.4.1 Lattice markup: emma:lattice, emma:arc, emma:node elements

In EMMA, a lattice is represented using an element emma:lattice, which has attributes initial and final for indicating the initial and final nodes of the lattice. For the lattice above, this will be: <emma:lattice initial="1"final="8"/>. The nodes are numbered with integers. If there is more than one distinct final node in the lattice the nodes should be represented as a space separated list in the value of the final attribute e.g. <emma:lattice initial="1" final="9 10 23"/>. There can only be one initial node in an EMMA lattice. Each transition in the lattice is represented as an element emma:arc with attributes from and to which indicate the nodes where the transition starts and ends. The arc's label is represented as the content of the emma:arc element, and can be any well-formed character or XML content. In the example here the contents are words. Empty (epsilon) transitions in a lattice should be represented in the emma:lattice representation as emma:arc elements with no content, e.g <emma:arc from="1" to="8"/>.

The example speech lattice above would be represented in EMMA markup as follows:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="interp1">
    <emma:lattice initial="1" final="8">
      <emma:arc from="1" to="2">flights</emma:arc>

      <emma:arc from="2" to="3">to</emma:arc>
      <emma:arc from="3" to="4">boston</emma:arc>
      <emma:arc from="3" to="4">austin</emma:arc>
      <emma:arc from="4" to="5">from</emma:arc>

      <emma:arc from="5" to="6">portland</emma:arc>
      <emma:arc from="5" to="6">oakland</emma:arc>
      <emma:arc from="6" to="7">today</emma:arc>
      <emma:arc from="7" to="8">please</emma:arc>

      <emma:arc from="6" to="8">tomorrow</emma:arc>
    </emma:lattice>
  </emma:interpretation>
</emma:emma>

Alternatively, if we wish to represent the same information as an N-best list using emma:one-of, we would have the more verbose representation:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:one-of id="nbest1">
    <emma:interpretation id="interp1">
      <text>flights to boston from portland today please</text>
    </emma:interpretation>

    <emma:interpretationid="interp2">
      <text>flights to boston from portland tomorrow</text>
    </emma:interpretation>

    <emma:interpretation id="interp3">
      <text>flights to austin from portland today please</text>
    </emma:interpretation>

    <emma:interpretation id="interp4">
      <text>flights to austin from portland tomorrow</text>
    </emma:interpretation>

    <emma:interpretation id="interp5">
      <text>flights to boston from oakland today please</text>
    </emma:interpretation>

    <emma:interpretation id="interp6">
      <text>flights to boston from oakland tomorrow</text>
    </emma:interpretation>

    <emma:interpretation id="interp7">
      <text>flights to austin from oakland today please</text>
    </emma:interpretation>

    <emma:interpretation id="interp8">
      <text>flights to austin from oakland tomorrow</text>
    </emma:interpretation>
  </emma:one-of>
</emma:emma>

The lattice representation avoids the need to enumerate all of the possible word sequences. Also, as detailed below, the emma:lattice representation enables placement of annotations on individual words in the input.

For use cases involving the representation of gesture/ink lattices and use cases involving lattices of semantic interpretations, EMMA allows for application namespace elements to appear within emma:arc.

For example a sequence of two gestures, each of which is recognized as either a line or an circle could be represented as follows:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="interp1">
    <emma:lattice initial="1" final="3">
      <emma:arc from="1" to="2">
        <circle radius="100"/>
      </emma:arc>
      <emma:arc from="2" to="3">
        <line length="628"/>
      </emma:arc>
      <emma:arc from="1" to="2">
        <circle radius="200"/>
      </emma:arc>
      <emma:arc from="2" to="3">
        <line length="1256"/>
      </emma:arc>
    </emma:lattice>
  </emma:interpretation>
</emma:emma>

As an example of a lattice of semantic interpretations, in a travel application where the source is either "Boston" or "Austin"and the destination is either "Newark" or "New York", the possibilities could be represented in a lattice as follows:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="interp1">
    <emma:lattice initial="1" final="3">
      <emma:arc from="1" to="2">
        <source city="boston"/>
      </emma:arc>
      <emma:arc from="2" to="3">
        <destination city="newark"/>
      </emma:arc>
      <emma:arc from="1" to="2">
        <source city="austin"/>
      </emma:arc>
      <emma:arc from="2" to="3">
        <destination city="new york"/>
      </emma:arc>
    </emma:lattice>
  </emma:interpretation>
</emma:emma>

The emma:arc element can contain either an application namespace element or character data. It cannot contain combinations of application namespace elements and character data. However, an emma:info element can appear within an emma:arc element alongside character data, in order to allow for the association of vendor or application specific annotations on a single word or symbol in a lattice.

So, in summary, there are four groupings of content that can appear within emma:arc:

• Character Data e.g. a recognized word in a speech lattice.
• Character Data and a single emma:info element providing vendor or application specific annotations that apply to the character data.
• An application namespace element e.g. the gesture and meaning lattice examples above.
• An application namespace element and a single emma:info element providing vendor or application specific annotations that apply to the character data.

3.4.2 Annotations on lattices

The encoding of lattice arcs as XML elements (emma:arc) enables arcs to be annotated with metadata such as timestamps, costs, or confidence scores:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="interp1">
    <emma:lattice initial="1" final="8">
      <emma:arc
       from="1"
       to="2"
       emma:start="1087995961542"
       emma:end="1087995962042"
       emma:cost="30">
         flights
      </emma:arc>

      <emma:arc
       from="2"
       to="3"
       emma:start="1087995962042"
       emma:end="1087995962542"
       emma:cost="20">
         to
      </emma:arc>

      <emma:arc
       from="3"
       to="4"
       emma:start="1087995962542"
       emma:end="1087995963042"
       emma:cost="50">
         boston
      </emma:arc>

      <emma:arc
       from="3"
       to="4"
       emma:start="1087995963042"
       emma:end="1087995963742"
       emma:cost="60">
         austin
      </emma:arc>
      ...
    </emma:lattice>
  </emma:interpretation>
</emma:emma>

The following EMMA attributes may optionally be placed on emma:arc elements: absolute timestamps (emma:start, emma:end), relative timestamps ( emma:offset-to-start, emma:duration), emma:confidence, emma:cost, the human language of the input (emma:lang), emma:medium, emma:mode, and emma:source. The use case for emma:medium, emma:mode, and emma:source is for lattices which contains content from different input modes. The emma:arc element can also contain an optional emma:info element for specification of vendor and application specific annotations on the arc.

Costs are typically application and device dependent. There are a variety of ways that individual arc costs can be combined to produce costs for specific paths through the lattice. This specification does not standardize the way for these costs to be combined; it is up to the applications and devices to determine how such derived costs would be computed and used.

For some lattice formats, it is also desirable to annotate the nodes in the lattice themselves with information such as costs. For example in speech recognition, costs may be placed on nodes as a result of word penalities or redistribution of costs. For this purpose EMMA also provides an emma:node element which can host annotations such as emma:cost. The emma:node element must have an attribute node-number which indicates the number of the node. There can only be one emma:node specification for a given numbered node in the lattice. In our example, if there was a cost of 100 on the final state this could be represented as follows:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma
     http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="interp1">
    <emma:lattice initial="1" final="8">
      <emma:arc
       from="1"
       to="2"
       emma:start="1087995961542"
       emma:end="1087995962042"
       emma:cost="30">
         flights
      </emma:arc>
      <emma:arc
       from="2"
       to="3"
       emma:start="1087995962042"
       emma:end="1087995962542"
       emma:cost="20">
         to
      </emma:arc>

      <emma:arc
       from="3"
       to="4"
       emma:start="1087995962542"
       emma:end="1087995963042"
       emma:cost="50">
         boston
      </emma:arc>
      <emma:arc
       from="3"
       to="4"
       emma:start="1087995963042"
       emma:end="1087995963742"
       emma:cost="60">
         austin
      </emma:arc>
        ...
      <emma:node node-number="8" emma:cost="100"/>
    </emma:lattice>
  </emma:interpretation>
</emma:emma>

3.4.3 Relative timestamps on lattices

The relative timestamp mechanism in EMMA can be used to provide temporal information about arcs in a lattice in relative terms using offsets in milliseconds. In order to do this the absolute time should be specified on emma:interpretation. Since emma:time-ref-uri and emma:time-ref-anchor-point apply to emma:lattice and can be used there to set the anchor point for offset to the start of the absolute time specified on emma:interpretation. The offset in milliseconds to the beginning of each arc can then be indicated on each emma:arc in the emma:offset-to-start attribute.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">

  <emma:interpretation id="interp1"
          emma:start="1087995961542" emma:end="1087995963042">
    <emma:lattice emma:time-ref-uri="#interp1" 
        emma:time-ref-anchor-point="start"
        initial="1" final="4">
      <emma:arc
       from="1"
       to="2"
       emma:offset-to-start="0">
         flights
      </emma:arc>
      <emma:arc
       from="2"
       to="3"
       emma:offset-to-start="500">
         to
      </emma:arc>

      <emma:arc
       from="3"
       to="4"
       emma:offset-to-start="1000">
         boston
      </emma:arc>
    </emma:lattice>
  </emma:interpretation>
</emma:emma>

Note that the offset for the first emma:arc will always be zero since the EMMA attribute emma:offset-to-start indicates the number of milliseconds from the anchor point to the start of the piece of input associated with the emma:arc, in this case the word "flights".

3.5 Literal semantics: emma:literal element

Certain EMMA processing components may produce semantic results in the form of string literals without any surrounding application namespace markup. These should be placed with the EMMA element emma:literal within emma:interpretation. For example, if a semantic interpreter simply returned "boston" this could be represented in EMMA as:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
    <emma:interpretation>
      <emma:literal>boston</emma:literal>
    </emma:interpretation>
</emma:emma>

4. EMMA annotations

This section defines annotations in the EMMA namespace including both attributes and elements. The values are specified in terms of the data types defined by XML Schema Part 2: Datatypes [XSD].

4.1 EMMA annotation elements

4.1.1 Data model: emma:model element

The data model that may be used to express constraints on the structure and content of instance data is specified as one of the annotations of the instance. Specifying the data model is optional, in which case the data model can be said to be implicit. Typically the data model is pre-established by the application.

The data model is specified with the emma:model annotation defined as an element in the EMMA namespace. The attribute emma:model-ref must be specified on emma:interpretation, container elements, or application namespace elements in order to refer to the data model for the contents of that element. Note that since multiple emma:model elements can be specified under the emma:emma it is possible to refer to multiple data models in within a single EMMA document. For example, different alternative interpretations under an emma:one-of might have different data models. In this case, an emma:model-ref attribute would appear on each emma:interpretation element in the N-best list with its value being the id of the emma:model element for that particular interpretation.

The data model is closely related to the interpretation data, and is typically specified as the annotation related to the emma:interpretation or emma:one-of elements.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:model id="model1" ref="http://myserver/models/city.xml"/>
  <emma:interpretation id="int1" emma:model-ref="model1">
    <city> London </city>
    <country> UK </country>
  </emma:interpretation>
</emma:emma>

The emma:model annotation can reference any element or attribute in the application instance data, as well as any EMMA container element (emma:one-of, emma:group, or emma:sequence).

The data model annotation can be used to either reference an external data model with the "ref" attribute or provide a data model as in-line content. Either a "ref" attribute or in-line data model (but not both) must be specified.

4.1.2 Interpretation derivation: emma:derived-from element and emma:derivation element

Instances of interpretations are in general derived from other instances of interpretation in a process that goes from raw data to increasingly refined representations of the input. The derivation annotation is used to link any two interpretations that are related by representing the source and the outcome of an interpretation process. For instance, a speech recognition process can return the following result in the form of raw text:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="raw">
    <answer>From Boston to Denver tomorrow</answer>
  </emma:interpretation>

</emma:emma>

A first interpretation process will produce:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="better">
    <origin>Boston</origin>
    <destination>Denver</destination>
    <date>tomorrow</date>
  </emma:interpretation>
</emma:emma>

A second interpretation process, aware of the current date, will be able to produce a more refined instance, such as:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="best">
    <origin>Boston</origin>
    <destination>Denver</destination>
    <date>20030315</date>
  </emma:interpretation>
</emma:emma>

The interaction manager may need to have access to the three levels of interpretation. The emma:derived-from annotation element can be used to establish a chain of derivation relationships as in the following example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:derivation>
    <emma:interpretation id="raw">
      <answer>From Boston to Denver tomorrow</answer>
    </emma:interpretation>

    <emma:interpretation id="better">
      <emma:derived-from resource="#raw" composite="false"/>
      <origin>Boston</origin>
      <destination>Denver</destination>
      <date>tomorrow</date>
    </emma:interpretation>
  </emma:derivation>
  
  <emma:interpretation id="best">
    <emma:derived-from resource="#better" composite="false"/>
    <origin>Boston</origin>
    <destination>Denver</destination>
    <date>20030315</date>
  </emma:interpretation>
</emma:emma>

The emma:derivation element is used as a container for representations of the earlier stages in the interpretation of the input. The latest stage of processing a direct child of emma:emma.

In order to indicate whether an emma:derived-from element describes a sequential derivation step or a composite derivation step, the emma:derived-from element has an attribute composite which has a boolean value. A composite emma:derived-from needs to be marked as composite="true" while a sequential emma:derived-from element is marked as composite="false". If this attribute is not specified the value is false by default.

In annotating derivations of the processing of the input, EMMA provides the flexibility of both course-grained or fine-grained annotation of relations among interpretations. For example, when relating two N-best lists, within emma:one-of elements either there can be a single emma:derived-from element under emma:one-of referring to the ID of the emma:one-of for the earlier processing stage:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:one-of id="nbest2">
    <emma:derived-from resource="#nbest1" composite="false"/>
    <emma:interpretation id="int1b">
      <origin>Boston</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>

    <emma:interpretation id="int2b">
      <origin>Austin</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>
  </emma:one-of>

  <emma:derivation>
    <emma:one-of id="nbest1">
      <emma:interpretation id="int1">
        <result>from boston to denver on march eleven two thousand three</result>
      </emma:interpretation>

      <emma:interpretation id="int2">
        <result>from austin to denver on march eleven two thousand three</result>
      </emma:interpretation>
  </emma:one-of>
</emma:derivation>
</emma:emma>

Or there can be a separate emma:derived-from element on each emma:interpretation element referring to the specific emma:interpretation element it was derived from.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:one-of id="nbest2">
    <emma:interpretation id="int1b">
     <emma:derived-from resource="#int1" composite="false"/>
      <origin>Boston</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>

    <emma:interpretation id="int2b">
     <emma:derived-from resource="#int2" composite="false"/>
      <origin>Austin</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>
  </emma:one-of>
  <emma:derivation>
    <emma:one-of id="nbest1">
      <emma:interpretation id="int1">
        <result>from boston to denver on march eleven two thousand three</result>
      </emma:interpretation>

      <emma:interpretation id="int2">
        <result>from austin to denver on march eleven two thousand three</result>
      </emma:interpretation>
    </emma:one-of>
  </emma:derivation>
</emma:emma>

Section 4.3 provides further examples of the use of emma:derived-from to represent both sequential derivations like those above and composite derivations in which inputs from multiple different modalities are combined, and addresses the issue of the scope of EMMA annotations across derivations of user input.

4.1.3 Reference to grammar used: emma:grammar element

The grammar that was used to derive the EMMA result is specified with the emma:grammar annotation defined as an element in the EMMA namespace.

Example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:grammar id="gram1" href="someURI"/>
  <emma:grammar id="gram2" href="anotherURI"/>
  <emma:one-of id="r1">
    <emma:interpretation id="int1" emma:grammar-ref="gram1">
      <origin>Boston</origin>
    </emma:interpretation>

    <emma:interpretation id="int2" emma:grammar-ref="gram1">
        <origin>Austin</origin>
    </emma:interpretation>

    <emma:interpretation id="int3" emma:grammar-ref="gram2">
        <command>help</command>
    </emma:interpretation>
  </emma:one-of>
</emma:emma>

The emma:grammar annotation is a child of emma:emma.

4.1.4 Extensibility to application/vendor specific annotations: emma:info element

In Section 3.2, a series of attributes are defined for representation of metadata about user inputs in a standardized form. EMMA also provides an extensibility mechanism for annotation of user inputs with vendor or application specific metadata not covered by the standard set of EMMA annotations. The element emma:info should be used as a container for these annotations. For example, if an input to a dialog system needed to be annotated with the number that the call originated from, their state, some indication of the type of customer, and the name of the service, these pieces of information could be represented within emma:info as in the following example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:info>
    <caller_id>
      <phone_number>2121234567</phone_number>
      <state>NY</state>
    </caller_id>

    <customer_type>residential</customer_type>
    <service_name>acme_travel_service</service_name>
  </emma:info>

  <emma:one-of id="r1" emma:start="1087995961542" 
      emma:end="1087995963542">
    <emma:interpretation id="int1" emma:confidence="0.75">
      <origin>Boston</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>

    <emma:interpretation id="int2" emma:confidence="0.68">
      <origin>Austin</origin>
      <destination>Denver</destination>
      <date>03112003</date>
    </emma:interpretation>
  </emma:one-of>
</emma:emma>

It is important to have an EMMA container element for application/vendor specific annotations since EMMA elements provide a structure for representation of multiple possible interpretations of the input. As a result it is cumbersome to state application/vendor specific metadata as part of the application data within each emma:interpretation. An element is used rather than an attribute so that internal structure can be given to the annotations within emma:info.

In addition to emma:emma, emma:info can also appear as a child of other structural elements such as emma:interpretation, emma:info and so on. When emma:info appears as a child of one of these elements the application/vendor specific annotations contained within emma:info are assumed to apply to all of the emma:interpretation elements within the containing element. The semantics of conflicting annotations in emma:info, for example when different values are found within emma:emma and emma:interpretation, are left to the developer of the vendor/application specific annotations.

4.1.5 Endpoint reference: emma:endpoint-info element and emma:endpoint element

In order to conduct multimodal interaction, there is a need in EMMA to specify the properties of the endpoint that receives the input which leads to the EMMA annotation. This allows subsequent components to utilize the endpoint properties as well as the annotated inputs to conduct meaningful multimodal interaction. EMMA element emma:endpoint can be used for this purpose. It can specify the endpoint properties based on a set of common endpoint property attributes in EMMA, such as emma:endpoint-address, emma:port-num, emma:port-type, etc. (See Section 4.2.14). Moreover, it provides an extensible annotation structure that allows the inclusion of application and vendor specific endpoint properties.

It should be noted that the usage of the term "endpoint" in this context is different from the way that the term is used in speech processing, where it refers to the end of a speech input. As used here, "endpoint" refers to a network location which is the source or receipient of an EMMA document.

In multimodal interaction, multiple devices can be used and each device can open multiple communication endpoints at the same time. These endpoints are used to transmit and receive data, such as raw input, EMMA documents, etc. Moreover, these communication endpoints can be based on a varity of protocols and data formats, such as SIP, TCP, SOAP, HTTP, SMTP, MRCP, etc. The EMMA element emma:endpoint provides a generic representation of endpoint information which is relevant to multimodal interaction. It allows the annotation to be interoperable, and it eliminates the need for EMMA processors to create their own specialized annotations for existing protocols, potential protocols or yet undefined private protocols that they may use.

Moreover, emma:endpoint-info provides a container to hold all annotations regarding the endpoint information, including emma:endpoint and other application and vendor specific annotations that are related to the communication, allowing the same communication environment to be referenced and used in multiple interpretations.

It should be noted that EMMA provides two locations (i.e. emma:info and emma:endpoint-info) for specifying vendor/application specific annotations. If the annotation is specifically related to the description of the endpoint, then the vendor/application specific annotation should be placed within emma:endpoint-info, otherwise it should be placed within emma:info.

The following example illustrates the annotation of endpoint reference properties in EMMA.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example"
    xmlns:ex="http://www.example.com/emma/port">
  <emma:endpoint-info id="audio-channel-1">
    <emma:endpoint id="endpoint1"
        emma:endpoint-role="sink"
        emma:endpoint-address="135.61.71.103"
        emma:port-num="50204"
        emma:port-type="rtp"
        emma:endpoint-pair-ref="endpoint2"
        emma:media-type="audio/dsr-202212; rate:8000; maxptime:40"
        emma:service-name="travel"
        emma:mode="speech">
      <ex:app-protocol>SIP</ex:app-protocol>
    </emma:endpoint>

    <emma:endpoint id="endpoint2"
        emma:endpoint-role="source"
        emma:endpoint-address="136.62.72.104"
        emma:port-num="50204"
        emma:port-type="rtp"
        emma:endpoint-pair-ref="endpoint1"
        emma:media-type="audio/dsr-202212; rate:8000; maxptime:40"
        emma:service-name="travel"
        emma:mode="speech">
      <ex:app-protocol>SIP</ex:app-protocol>
    </emma:endpoint>
  </emma:endpoint-info>

  <emma:interpretation id="int1"
      emma:start="1087995961542" emma:end="1087995963542"
      emma:endpoint-info-ref="audio-channel-1">
    <destination>Chicago</destination>
  </emma:interpretation>
</emma:emma>

The ex:app-protocol is provided by the application or the vendor specification. It specifies that the application layer protocol used to establish the speech transmission from the "source" port to the "sink" port is Session Initiation Protocol (SIP). This is specific to SIP based VoIP communication, in which the actual media transmission and the call signaling that controls the communication sessions, are separated and typically based on different protocols. In the above example, the Real-time Transmission Protocol (RTP) is used in the media transmission between the source port and the sink port.

4.2 EMMA annotation attributes

4.2.1 Tokens of input: emma:tokens attribute

The emma:tokens annotation holds a list of input tokens. In the following description, the term tokens is used in the computational and syntactic sense of units of input, and not in the sense of XML tokens.

The value held in emma:tokens is the list of the tokens of input as produced by the processor which generated the EMMA document. In the case where a grammar is used to constrain input, the value will correspond to tokens as defined by the grammar. So for an EMMA document produced by input to a W3C SRGS grammar [SRGS], the value of emma:tokens will be the list of words and/or phrases that are defined as tokens in SRGS (through white-spaced character data or the token; element, see SRGS (Section 2.1 Tokens). Items in the emma:tokens list are delimited by white space and/or quotation marks for phrases containing white space. For example:

emma:tokens="arriving at 'Liverpool Street'"

where the three tokens of input are arriving, at and Liverpool Street.

The tokens annotation may be applied not just to the lexical words and phrases of language but to any level of input processing. Other examples of tokenization include phonemes, ink strokes, gestures and any other discrete units of input at any level.

Examples:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="int1"
      emma:tokens="From Cambridge to London tomorrow">
    <origin emma:tokens="From Cambridge">Cambridge</origin>
    <destination emma:tokens="to London">London</destination>
    <date emma:tokens="tomorrow">20030315</date>
  </emma:interpretation>
</emma:emma>

4.2.2 Reference to processing: emma:process attribute

A reference to the information concerning the processing that was used for generating an interpretation can be made as in the following example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma
     http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:derivation>
    <emma:interpretation id="raw">
      <answer>From Boston to Denver tomorrow</answer>
    </emma:interpretation>

    <emma:interpretation id="better"
        emma:process="http://example.com/mysemproc1.xml">
      <origin>Boston</origin>
      <destination>Denver</destination>
      <date>tomorrow</date>
      <emma:derived-from resource="#raw"/>
    </emma:interpretation>
  </emma:derivation>

  <emma:interpretation id="best"
      emma:process="http://example.com/mysemproc2.xml">
    <origin>Boston</origin>
    <destination>Denver</destination>
    <date>03152003</date>
    <emma:derived-from resource="#better"/>
  </emma:interpretation>
</emma:emma>

The process description document, referenced by the emma:process annotation can include information on the process itself, such as grammar, type of parser, etc. EMMA is not normative about the format of the process description document.

4.2.3 Lack of input: emma:no-input attribute

The case of lack of input can be annotated as follows:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="int1" emma:no-input="true" />
</emma:emma>

If the emma:interpretation is annotated with emma:no-input="true" then the emma:interpretation must be empty. The emma:interpretation is empty only if the emma:interpretation is annotated with either emma:no-input="true" or emma:uninterpreted="true".

4.2.4 Uninterpretable input: emma:uninterpreted attribute

Input that cannot be interpreted is annotated as in the following example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="interp1" emma:uninterpreted="true"/>
</emma:emma>

The notation for uninterpretable input can refer to any possible stage of interpretation processing, including raw transcriptions. For instance, if input speech cannot be correctly recognized or the spoken input is not matched by a grammar (or by a language constraint given to the recognition), it can be tagged as emma:uninterpreted as in the following example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="raw"
      emma:process="http://example.com/myasr.xml"
      emma:uninterpreted="true" emma:tokens="From Cambridge to London tomorrow"/>
</emma:emma>

Note that sometimes an input is classified as "uninterpreted"because its score falls below a confidence threshold set in the processor. In this case it still may be useful for further stages of processing to know what the highest scoring interpretation was, even if that interpretation's confidence did not exceed the threshold. If the interpretation is a raw speech recognition result, an emma:tokens attribute can be used to represent the best scoring result, as in the above example. If the interpretation is a semantic result, the best scoring interpretation can be included within the emma:interpretation element, as in the following example:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="interp1" emma:uninterpreted="true">
    <source>philadelphia</source>
    <destination>boston</destination>
  </emma:interpretation>
</emma:emma>

The emma:interpretation is empty only if the emma:interpretation is annotated with either emma:uninterpreted="true" or emma:no-input="true".

4.2.5 Human language of input: emma:lang attribute

The emma:lang annotation is used to indicate the human language for the input that it annotates. The values of the emma:lang attribute are language identifiers as defined by [IETF RFC 1766]. For example, emma:lang="fr" denotes French, and emma:lang="en-US" denotes US English. emma:lang can be applied to any emma:interpretation element. Its annotative scope follows the annotative scope of these elements. In contrast, the attribute xml:lang in XML 1.0 is used to specify the language used in the contents and attribute values of any element in an XML document. The attribute emma:lang must be used if the xml:lang can no longer apply. For example, the contents and attribute values of an element in the EMMA document are from different languages, such as in the case where the input language is in French, and the language of the annotated attributes is in English.

The following example shows the use of emma:lang for annotating an input interpretation.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="int1" emma:lang="fr">
    <answer>arretez</answer>
  </emma:interpretation>
</emma:emma>

In order handle inputs involving multiple languages, such as through code switching, the emma:lang tag can contain several language identifiers separated by spaces.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="int1"
      emma:tokens="please stop arretez s'il vous plait" 
      emma:lang="en fr">
    <command> CANCEL </command>
  </emma:interpretation>
</emma:emma>

4.2.6 Reference to signal: emma:signal attribute

A URI reference to the signal that originated the input recognition process may be represented in EMMA using the emma:signal annotation.

Here is an example where the reference to the signal is applied to the emma:interpretation element:

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="intp1"
      emma:signal="http://example.com/signals/sg23.bin">
    <origin>Boston</origin>
    <destination>Denver</destination>
    <date>03152003</date>
  </emma:interpretation>
</emma:emma>

4.2.7 Media type: emma:media-type attribute

The data format of the signal that originated the input may be represented in EMMA using the emma:media-type annotation. An initial set of MIME media types is defined by [RFC2046].

Here is an example where the media type for the ETSI ES 202 212 audio codec for Distributed Speech Recognition (DSR) is applied to the emma:interpretation element. The example also specifies an optional sampling rate of 8 kHz and maxptime of 40 milliseconds.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma
     http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="intp1"
      emma:media-type="audio/dsr-202212; rate:8000; maxptime:40">
    <origin>Boston</origin>
    <destination>Denver</destination>
    <date>03152003</date>
  </emma:interpretation>
</emma:emma>

4.2.8 Confidence scores: emma:confidence attribute

The confidence score in EMMA is used to indicate the quality of the input, and it is the value assigned to emma:confidence in the EMMA namespace. The confidence score is a number in the range from 0.0 to 1.0 inclusive. A value of 0.0 indicates minimum confidence, and a value of 1.0 indicates maximum confidence. Note that emma:confidence should not be assumed to mean only the confidence of the speech recognizer, but rather the confidence of the whatever processor was responsible for creating the EMMA result, based on whatever evidence it has. For a natural language interpretation, for example, this might include semantic heuristics in addition to speech recognition scores. Moreover, the confidence score values do not have to be interpreted as probabilities. In fact confidence score values are platform-dependent, since their computation is likely to differ between platforms and different EMMA processors. Confidence scores are annotated explicitly in EMMA in order to provide this information to the subsequent processes for multimodal interaction. The example below illustrates how confidence scores are annotated in EMMA.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:one-of id="nbest1">
    <emma:interpretation id="meaning1" emma:confidence="0.6">
      <location>Boston</location>
    </emma:interpretation>

    <emma:interpretation id="meaning2" emma:confidence="0.4">
      <location> Austin </location>
    </emma:interpretation>
  </emma:one-of>
</emma:emma>

In addition to its use as an attribute on the EMMA interpretation and container elements, the emma:confidence attribute can also be used to assign confidences to elements in instance data in the application namespace. This can be seen in the following example, where the <destination> and <origin> elements have confidences.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  <emma:interpretation id="meaning1" emma:confidence="0.6">
     <destination emma:confidence="0.8"> Boston</destination>
     <origin emma:confidence="0.6"> Austin </origin>
  </emma:interpretation>
</emma:emma>

Although in general instance data can be represented in XML using a combination of elements and attributes in the application namespace, EMMA does not provide a standard way to annotate processors' confidences in attributes. Consequently, instance data that is expected to be assigned confidences should be represented using elements, as in the above example.

4.2.9 Input source: emma:source attribute

The source of an interpreted input may be represented in EMMA as a URI resource using the emma:source annotation.

Here is an example that shows different input sources for different input interpretations.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example" xmlns:myapp="http://www.example.com/myapp">
   <emma:one-of id="nbest1">
     <emma:interpretation id="intp1"
     emma:source="http://example.com/microphone/NC-61">
       <myapp:destination>Boston</myapp:destination>
     </emma:interpretation>

     <emma:interpretation id="intp2"
         emma:source="http://example.com/microphone/NC-4024">
       <myapp:destination>Austin</myapp:destination>
     </emma:interpretation>
   </emma:one-of>
 </emma:emma>

4.2.10 Timestamps

The start and end times for input can be indicated using either absolute timestamps or relative timestamps. Both are in milliseconds for ease in processing timestamps. Note that the absolute time may be conveniently determined using the ECMAScript Date object's getTime() function.

4.2.10.1 Absolute timestamps: emma:start, emma:end attributes

Here is an example of a timestamp for an absolute time.

<emma:emma version="1.0" 
    xmlns:emma:="http://www.w3.org/2003/04/emma" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.w3.org/2003/04/emma 
    http://www.w3.org/TR/emma/emma10.xsd" 
    xmlns="http://www.example.com/example">
  &