EMMA: Extensible MultiModal Annotation 1.0:
Implementation Report Plan

23 September 2008

Editor:

Michael Johnston, AT&T

Authors:

Paolo Baggia, Loquendo

Patrizio Bergallo, Loquendo

Daniel C. Burnett, Nuance

Jerry Carter, Nuance

Deborah Dahl, Invited Expert

Copyright © 2008 W3C ^® ( MIT , ERCIM , Keio), All Rights Reserved. W3C liability, trademark, document use rules apply.

---

Table of Contents

• 1. Introduction
  • 1.1 Implementation report objectives
  • 1.2 Implementation report non-objectives
• 2. Work during the Candidate Recommendation period
• 3. Participating in the implementation report
• 4. Entrance criteria for the Proposed Recommendation phase
• 5. Implementation report requirements
  • 5.1 Detailed requirements for implementation report
  • 5.2 Notes on testing
  • 5.3 Out of scope
• 6. Systems
• 7. Test assertions
  • 7.1 Classification of test assertions
  • 7.2 EMMA XML Schema conformance
  • 7.3 EMMA test assertions
• Appendices
  • Appendix A. Implementation report submission format
  • Appendix B. Acknowledgements

1. Introduction

The EMMA Specification entered the Candidate Recommendation period on 14 April 2008.

Preparation of an Implementation Report is a key criterion for moving beyond the Candidate Recommendation phase. This document describes the requirements for the Implementation Report and the process that the Multimodal Working Group will follow in preparing the report.

Note: This Implementation Report Plan was modified as follows:

22 January 2008

• The Candidate Recommendation period is modified from "?? 2007" to "14 April 2008" as defined in the Candidate Recommendation.

23 September 2008

• Test assertions 801, 902, 903 and 1501 have been removed, because they are actually not described in the EMMA specification.

1.1 Implementation report objectives

1. Must verify that the specification is implementable.

1.2 Implementation report non-objectives

1. The IR does not attempt conformance testing of implementations.

2. Work during the Candidate Recommendation period

During the CR period, the Working Group will carry out the following activities:

1. Clarification and improvement of the exposition of the specification.
2. Disposing of comments that are communicated to the WG during the CR period.
3. Preparation of an Implementation Report meeting the criteria outlined in this document.

3. Participating in the implementation report

You are invited to contribute to the assessment of the W3C EMMA 1.0 Specification by participating in the Implementation Report process.

• Deadline for submission of a EMMA Implementation Report is February 29th 2008.
• All the test assertions are reported in the table below.
• Comments on this document or requests for further information should be sent to the Working Group's public mailing list www-multimodal@w3.org (archive). If an updated version of this document or the test assertion set is published, a notification will be sent to this mailing list.

4. Entrance criteria for the Proposed Recommendation phase

The Multimodal Working Group established the following entrance criteria for the Proposed Recommendation phase in the Request for CR:

1. Sufficient reports of implementation experience have been gathered to demonstrate that EMMA processors based on the specification are implementable and have compatible behavior.
2. Specific Implementation Report Requirements (outlined below) have been met.
3. The Working Group has formally addressed and responded to all public comments received by the Working Group.

5. Implementation report requirements

5.1 Detailed requirements for implementation report

1. Testimonials from implementers will be included in the IR when provided to document the utility and implementability of the specification.
2. IR must cover all specified features in the specification. For each feature the IR should indicate:
   • Feature status: required, optional, other.
   • Feature utility/usefulness based on feedback from implementers.
   • Implementability of the feature specification.
3. Feature status is a factor in test coverage in the report:
   • Required specification features must have at least two implementations. Implementations that do not implement a required specification feature must document the reason for not implementing the feature.
   • Optional specification features must have at least one implementation. Implementations that do not implement an optional specification feature should document the reason for not implementing the feature.

5.3 Out of scope

6. Systems

Note: The testimonials with pink background from "Acme Labs" and "Beta Inc." are just examples and will be replaced with the actually submitted testimonials.

7. Test assertions

The aim of this section is to describe the range of test assertions developed for the EMMA 1.0 Specification. The table lists all the assertions that were derived from the EMMA 1.0 Specification.

The Assert ID column uniquely identifies the assertion. The Feature column indicates the specific elements or attributes which the test assertion applies to. The Spec column identifies the section of the EMMA 1.0 Specification from which the assertion was derived. The REQ column is a Y/N value indicating whether the test assertion is for a feature which is required. The SUB column is a Y/N value indicating whether the test assertion is a subconstraint which is dependent on the implementation of the preceding non subconstraint feature test assertion. The Semantics column specifies the semantics of the feature or the constraint which must be met. The Result column will be annotated with the number of 'pass', 'fail', and 'not implemented' (P/F/NI) in the set of implementation reports.

7.1 Classification of test assertions

Test assertions are classified into two types, basic test assertions which test for the presence of each feature, and sub constraints which only apply if that particular feature is implemented. Generally, sub constraints encode structural constraints that could not be expressed in the EMMA schema. Sub constraints are marked with 'SUB CONSTRAINT:' in the Semantics field.

7.2 EMMA XML Schema conformance

The most fundamental test of a conforming EMMA implementation is that the EMMA documents it utilizes must successfully validate with respect to the EMMA XML Schema.

7.3 EMMA Test assertions

Appendices

Appendix A - Implementation report submission format

The following XML should be used as a template for providing an implementation report.

<system-report name="YOUR-SYSTEM-NAME-HERE">
	<testimonial>YOUR-WELL-FORMED-TESTIMOMIAL-CONTENT-HERE</testimonial>
	<assert id="100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="200" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="301" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="310" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="400" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="402" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="602" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="603" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="604" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="605" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="606" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="607" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="608" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="609" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="800" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="810" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="811" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="901" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="904" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="910" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="911" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1000" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1001" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1002" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1400" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1701" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1800" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="1900" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2000" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2101" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2202" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2203" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2204" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2301" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2310" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2311" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2320" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2330" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2401" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2510" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2601" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2701" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2710" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2711" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2713" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2714" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2715" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2716" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2717" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
	<assert id="2718" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
</system-report>

Appendix B - Acknowledgements

The Multimodal Working Group would like to acknowledge the contributions of several individuals:

• Kazuyuki Ashimura, W3C
• Jin Liu, T-Systems
• Dave Raggett, Volantis, W3C
• Massimo Romanelli, DFKI