EMMA: Extensible MultiModal Annotation 1.0:
Implementation Report Plan

23 September 2008

Michael Johnston, AT&T
Paolo Baggia, Loquendo
Patrizio Bergallo, Loquendo
Daniel C. Burnett, Nuance
Jerry Carter, Nuance
Deborah Dahl, Invited Expert

Table of Contents

1. Introduction

The EMMA Specification entered the Candidate Recommendation period on 14 April 2008.

Preparation of an Implementation Report is a key criterion for moving beyond the Candidate Recommendation phase. This document describes the requirements for the Implementation Report and the process that the Multimodal Working Group will follow in preparing the report.

Note: This Implementation Report Plan was modified as follows:

22 January 2008
23 September 2008

1.1 Implementation report objectives

  1. Must verify that the specification is implementable.

1.2 Implementation report non-objectives

  1. The IR does not attempt conformance testing of implementations.

2. Work during the Candidate Recommendation period

During the CR period, the Working Group will carry out the following activities:

  1. Clarification and improvement of the exposition of the specification.
  2. Disposing of comments that are communicated to the WG during the CR period.
  3. Preparation of an Implementation Report meeting the criteria outlined in this document.

3. Participating in the implementation report

You are invited to contribute to the assessment of the W3C EMMA 1.0 Specification by participating in the Implementation Report process.

4. Entrance criteria for the Proposed Recommendation phase

The Multimodal Working Group established the following entrance criteria for the Proposed Recommendation phase in the Request for CR:

  1. Sufficient reports of implementation experience have been gathered to demonstrate that EMMA processors based on the specification are implementable and have compatible behavior.
  2. Specific Implementation Report Requirements (outlined below) have been met.
  3. The Working Group has formally addressed and responded to all public comments received by the Working Group.

5. Implementation report requirements

5.1 Detailed requirements for implementation report

  1. Testimonials from implementers will be included in the IR when provided to document the utility and implementability of the specification.
  2. IR must cover all specified features in the specification. For each feature the IR should indicate:
  3. Feature status is a factor in test coverage in the report:

5.2 Notes on testing

  1. A implementation report must indicate the outcome of evaluating the implementation with respect to each of the test assertions. Possible outcomes are "pass", "fail" or "not-impl". "

5.3 Out of scope

EMMA Implementation Report will not cover:

6. Systems

Note: The testimonials with pink background from "Acme Labs" and "Beta Inc." are just examples and will be replaced with the actually submitted testimonials.

Acme Labs

Exec Summary

The W3C EMMA 1.0 Specification is well-written, easily implementable and extremely useful. Acme Labs used it to describe the recipe for haggis.

Beta Inc.

Exec Summary

The W3C EMMA 1.0 Specification is the best thing since sliced bread, extremely useful, easily implementable and well-written. Beta Inc. used it to build a new generation of interactive services, establish world peace, build a new chianti-spaghetti hybrid vehicle, and mix a perfect martini.

7. Test assertions

The aim of this section is to describe the range of test assertions developed for the EMMA 1.0 Specification. The table lists all the assertions that were derived from the EMMA 1.0 Specification.

The Assert ID column uniquely identifies the assertion. The Feature column indicates the specific elements or attributes which the test assertion applies to. The Spec column identifies the section of the EMMA 1.0 Specification from which the assertion was derived. The REQ column is a Y/N value indicating whether the test assertion is for a feature which is required. The SUB column is a Y/N value indicating whether the test assertion is a subconstraint which is dependent on the implementation of the preceding non subconstraint feature test assertion. The Semantics column specifies the semantics of the feature or the constraint which must be met. The Result column will be annotated with the number of 'pass', 'fail', and 'not implemented' (P/F/NI) in the set of implementation reports.

7.1 Classification of test assertions

Test assertions are classified into two types, basic test assertions which test for the presence of each feature, and sub constraints which only apply if that particular feature is implemented. Generally, sub constraints encode structural constraints that could not be expressed in the EMMA schema. Sub constraints are marked with 'SUB CONSTRAINT:' in the Semantics field.

7.2 EMMA XML Schema conformance

The most fundamental test of a conforming EMMA implementation is that the EMMA documents it utilizes must successfully validate with respect to the EMMA XML Schema.

7.3 EMMA Test assertions

Assert ID Feature Spec Req Sub Semantics Results
Structural Elements
100 emma:emma [3.1] YN EMMA documents MUST have emma:emma as the root element.
200 emma:interpretation [3.2] YN Application namespace markup representing the interpretation of inputs MUST be contained within the emma:interpretation element.
201 [3.2] YY SUB CONSTRAINT: If the emma:interpretation element is empty, then it MUST be annotated as emma:uninterpreted="true' or emma:no-input="true".
300 emma:one-of [3.3.1] YN EMMA N-best interpretations MUST be contained within an emma:one-of element.
301 [3.3.1] YY SUB CONSTRAINT: EMMA N-best interpretations contained within an emma:one-of element MUST be ordered best-first in document order where the measure is emma:confidence if present, otherwise the quality metric is platform specific.
310 [3.3.1] YY SUB CONSTRAINT: If an emma:one-of element contains other emma:one-of elements, (embedded one-of), emma:one-of elements MUST contain a disjunction-type attribute indicating the reason for the multiple interpretations.
400 emma:group [3.3.2] NN Interpretations of distinct inputs MAY be represented in a single EMMA document within the emma:group element.
402 emma:group-info [] NN Information describing the criteria used in determining the grouping of interpretations within an emma:group MAY be indicated within an emma:group-info child element within emma:group.
500 emma:sequence [3.3.3] NN A temporally ordered sequence of distinct inputs MAY be contained within an emma:sequence element with document order corresponding to temporal order.
600 emma:lattice [3.4] NN Lattices SHOULD be represented as an emma:lattice element containing a series of emma:arc and emma:node elements.
602 [3.4.1] YY SUB CONSTRAINT: The value of the initial attribute on emma:lattice MUST be the value of the from attribute on at least one of the emma:arc elements that it contains.
603 [3.4.1] YY SUB CONSTRAINT: The value of each space separated number within the final attribute on emma:lattice MUST be the value of the to attribute on at least one of the emma:arc elements that it contains.
604 [3.4.1] YY SUB CONSTRAINT: The value of a to attribute on emma:arc MUST be the value of a from attribute on at least one of its emma:arc siblings or be one of the final values on the parent emma:lattice element.
605 [3.4.1] YY SUB CONSTRAINT: The value of a from attribute on emma:arc MUST be the value of a to attribute on at least one of its emma:arc siblings or be the initial value on the parent emma:lattice element.
606 [3.4.1] YY SUB CONSTRAINT: Any epsilon transitions in the lattice MUST be represented as emma:arc elements with no content other than emma:info.
607 [3.4.1] YY SUB CONSTRAINT: The content of the emma:arc element MUST be either an application namespace XML element, well-formed character content, an application namespace element and an emma:info element, or well-formed character content and an emma:info element.
608 [3.4.1] YY SUB CONSTRAINT: There MUST be no more than one emma:node specification for each numbered node in the lattice.
609 [3.4.1] YY SUB CONSTRAINT: For each emma:node specification, the value of the node-number attribute MUST appear as the value of at some from or to attribute in one of the sibling emma:arc elements.
700 emma:literal [3.5] YN String literal semantic interpretations MUST be wrapped with the tag emma:literal.
Annotation Elements
800 emma:model [4.1.1] NN The data model of the application semantic markup MAY appear within the emma:model element.
810 [4.2.16] YY SUB CONSTRAINT: If emma:model appears as a child of emma:emma, there MUST be emma:one-of or emma:interpretation elements within emma:emma with emma:model-ref attributes that refer to the id of the emma:model child of emma:emma.
811 [4.2.16] YY SUB CONSTRAINT: If emma:model-ref appears on an emma:interpretation or emma:one-of, there MUST be an emma:model child of the dominating emma:emma whose id value equals the value of the emma:model-ref attribute.
901 emma:derived-from [4.1.2] NN The interpretation of all but the first stage of processing MAY contain an emma:derived-from element with an attribute resource which refers to the interpretation from which that interpretation was derived.
904 [4.1.2] YY SUB CONSTRAINT:If there is more than one emma:derived-from within an emma:interpretation or emma:one-of, then all of the emma:derived-from elements MUST have the attribute composite="true".
910 emma:derivation [4.1.2] YN The interpretation resulting of the most recent processing step MUST appear as a child of emma:emma.
911 emma:derivation [4.1.2] NN If interpretations resulting from earlier processing steps are included they SHOULD appear as children of a single emma:derivation element within emma:emma.
1000 emma:grammar [4.1.3] NN The grammar used MAY be referenced using an emma:grammar element within emma:emma.
1001 [4.2.15] YY SUB CONSTRAINT: If an emma:interpretation or emma:one-of is annotated with the emma:grammar-ref attribute, then the value of the emma:grammar-ref MUST reference the id of an emma:grammar element child of emma:emma in the same document.
1002 [4.2.15] YY SUB CONSTRAINT: If an emma:grammar element appears as a child of emma:emma, then there MUST be an emma:interpretation or emma:one-of element dominated by that emma:emma, whose emma:grammar-ref attribute references the id of the emma:grammar element.
1100 emma:info [4.1.4] NN Application and vendor specific annotations SHOULD appear within emma:info.
Annotation attributes
1201 emma:tokens [4.2.1] NN emma:tokens MAY be used to indicate the sequence of tokens recognized in the input by the recognizer.
1300 emma:process [4.2.2] NN emma:process MAY be used to identify the process used to assign the interpretation.
1400 emma:no-input [4.2.3] YN emma:no-input="true" MUST be used to designate lack of expected input.
1500 emma:uninterpreted [4.2.4] YN Input for which the processor is unable to assign an interpretation MUST be marked as emma:uninterpreted="true".
1600 emma:lang [4.2.5] NN emma:lang MAY be used to designate the human language of input
1700 emma:signal [4.2.6] NN emma:signal (with a URI value) MAY be used to designate the location of the signal processed by an EMMA processor.
1701 emma:signal-size [4.2.6] NN emma:signal-size MAY be used to indicate the file size in 8-bit octets of the input signal.
1800 emma:media-type [4.2.7] NN emma:media-type MAY be used to designate the MIME type of the processed signal.
1900 emma:confidence [4.2.8] NN emma:confidence MAY be used to designate the confidence score of an input.
2000 emma:source [4.2.9] NN emma:source (with a URI value) MAY be used to designate the source device.
2100 emma:start [] NN emma:start with a millisecond value MAY be used to indicate the absolute start time of an input.
2101 emma:end [] NN emma:end with a millisecond value MAY be used to indicate the absolute end time of an input.
2201 emma:time-ref-uri [] NN The emma:time-ref-uri attribute MAY be used to indicate the URI used as a reference point for relative timestamps.
2202 emma:time-ref-anchor [] YY SUB CONSTRAINT: If the emma:time-ref-uri points to an interval, emma:time-ref-anchor MUST appear on the interpretation with a value of start or end, indicating whether to use the start or end of that interval as the reference point.
2203 emma:offset-to-start [] NN The value of emma:offset-to-start MAY be used to indicate the number of milliseconds from the reference point to the start of the current input in a relative timestamp.
2204 emma:duration [] NN The emma:duration attribute value MAY be used to indicate the duration in milliseconds of the current input.
2300 emma:medium [4.2.11] YN emma:medium MUST be included on all EMMA inputs, indicating the medium of input.
2301 emma:medium [4.1.2] YY SUBCONSTRAINT: When EMMA inputs from different modes are combined to make a composite input, if the combining inputs have different values for emma:medium, then the value on the result MUST be a space separated list of the emma:medium values from the combining modes.
2310 emma:mode [4.2.11] YN emma:mode MUST be included on all EMMA inputs, indicating the mode of input.
2311 emma:mode [4.1.2] YY SUBCONSTRAINT: When EMMA inputs from different modes are combined to make a composite input, the value of emma:mode on the result MUST be a space separated list of the values on the combining inputs.
2320 emma:function [4.2.11] NN emma:function MAY be included on EMMA inputs, providing a classification of the function of the input.
2330 emma:verbal [4.2.11] NN emma:verbal MAY be included on EMMA inputs, indicating whether the input is verbal or non-verbal.
2401 emma:hook [4.2.12] NN emma:hook MAY be used on application namespace markup to indicate where content from another mode needs to be integrated.
2500 emma:cost [4.2.13] NN emma:cost MAY be used to indicate the cost or weight of an interpretation or lattice arc.
2510 emma:dialog-turn [4.2.17] NN emma:dialog-turn MAY be used to indicate an identifier for the current dialog turn.
Scope and inheritance of annotations
2600 emma:one-of [3.3.1] YN EMMA annotations appearing on emma:one-of MUST be true of all of the container elements (emma:one-of,emma:interpretation,emma:sequence,emma:group) within the emma:one-of.
2601 emma:derived-from [4.3] YN Each EMMA annotation from the previous stage of a sequential derivation (indicated using emma:derived-from) MUST be true of the interpretations in the current stage of the derivation, unless the annotation is explicitly re-stated.
Endpoint elements and attributes
2700 emma:endpoint-info [4.1.5] NN The emma:endpoint-info element MAY be used to contain emma:endpoint elements describing the characteristics of the endpoints.
2701 emma:endpoint [4.1.5] NN The emma:endpoint element MAY be used to provide annotations regarding a communication endpoint.
2710 emma:endpoint-role [4.2.14] NN The emma:endpoint-role attribute MAY be used to indicate the role of an endpoint.
2711 emma:endpoint-address [4.2.14] NN The emma:endpoint-address attribute MAY be used to indicate the address of an endpoint.
2713 emma:port-type [4.2.14] NN The emma:port-type attribute MAY be used to indicate the type of port of an endpoint.
2714 emma:port-num [4.2.14] NN The emma:port-num attribute MAY be used to indicate the port number of an endpoint.
2715 emma:message-id [4.2.14] NN The emma:message-id attribute MAY be used to indicate the message id of an endpoint.
2716 emma:service-name [4.2.14] NN The emma:service-name attribute MAY be used to indicate name of service that the system provides.
2717 emma:endpoint-pair-ref [4.2.14] NN The emma:endpoint-pair-ref attribute MAY be used to indicate the pairing between sink and source endpoints.
2718 emma:endpoint-info-ref [4.2.14] NN The emma:endpoint-info-ref attribute MAY be used to reference the endpoint-info element associated with an interpretation.


Appendix A - Implementation report submission format

The following XML should be used as a template for providing an implementation report.

<system-report name="YOUR-SYSTEM-NAME-HERE">
	<assert id="100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="200" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="301" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="310" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="400" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="402" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="602" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="603" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="604" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="605" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="606" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="607" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="608" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="609" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="800" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="810" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="811" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="901" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="904" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="910" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="911" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1000" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1001" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1002" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1400" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1701" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1800" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1900" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2000" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2101" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2202" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2203" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2204" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2301" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2310" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2311" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2320" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2330" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2401" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2510" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2601" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2701" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2710" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2711" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2713" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2714" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2715" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2716" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2717" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2718" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>

Appendix B - Acknowledgements

The Multimodal Working Group would like to acknowledge the contributions of several individuals: