PLS 1.0 Implementation Report Plan

Version: July 10, 2008

Editor:

Patrizio Bergallo, Loquendo

Authors:

Daniel Burnett, Voxeo

Paolo Baggia, Loquendo

Paul Bagshaw, France Telecom

Copyright ©2008 W3C^® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.

---

Table of Contents

• 1. Introduction
  • 1.1 Implementation Report Objectives
  • 1.2 Implementation Report Non-objectives
• 2. Work During the Candidate Recommendation Period
• 3. Participating in Implementation Report
• 4. Entrance Criteria to PR
• 5. Implementation Report Requirements
  • 5.1 Detailed requirements for Implementation Report
  • 5.2 Notes on Testing
  • 5.3 Out of Scope
• 6. Systems
• 7. Test Results
• Appendices
  • Appendix A. PLS 1.0 Conformance Language Definition
    • A.1 <test> Element
    • A.2 <instructions> Element
    • A.3 Input/Output Data
      • A.3.1 <input> Element
      • A.3.2 <output> Element
        • A.3.2.1 <item> Element
    • A.4 <lexicon> Element
    • A.5 Schema for PLS Conformance Language
    • A.6 Examples
  • Appendix B. Downloading Tests
    • B.1 The Manifest
    • B.2 The Report Submission Template
    • B.3 The Schema for PLS Conformance Language
    • B.4 The DTD for the manifest
    • B.5 The PLS 1.0 Implementation Report document
  • Appendix C. Contributors

1. Introduction

The PLS 1.0 Specification entered the Candidate Recommendation period on December 12, 2007.

The planned date for entering Proposed Recommendation is July 15, 2008. Preparation of an Implementation Report is a key criteria for moving beyond the Candidate Recommendation. This document describes the requirements for the Implementation Report and the process that the Voice Browser Working Group will follow in preparing the report.

Note: This Implementation Report Plan was modified as follows:

18 December 2007

• the title of this document (Overview.html) is modified to "Implementation Report Plan".
• The content of the test file for Assertion 79 is modified to check the assertion properly.

10 July 2008

• The abstract of the Assertion 50 is modified to better handle "xml:base" attribute.
• The content of the test file for Assertion 63 is modified on the test result to address the assertion properly.
• The pls-ir-20080710.zip archive is also regenerated to include above modified files.

1.1 Implementation Report Objectives

1. Must verify that the specification is implementable.
2. Must demonstrate interoperability of implementations of the specification.

1.2 Implementation Report Non-objectives

1. The IR does not attempt conformance testing of implementations.

2. Work During the Candidate Recommendation Period

During the CR period, the Working Group will carry out the following activities:

1. Clarification and improvement of the exposition of the specification.
2. Disposing of Comments that were communicated to the WG during the CR period.
3. Preparation of an Implementation Report meeting the criteria outlined in this document.

3. Participating in Implementation Report

You are invited to contribute to the assessment of the W3C Pronunciation Lexicon Specification 1.0 Specification by participating in the Implementation Report process.

• Deadline for submission of a report is July 15, 2008.
• All the tests reported in table below and the report submission format are provided with this document. The whole test suite including the report submission format is also contained in pls-ir-20080710.zip
• Comments on this document and the test suite or requests for further information should be sent to the Working Group's public mailing list www-voice@w3.org (archive). If an updated version of this document or the test set is published, a notification will be sent to this mailing list.

4. Entrance Criteria to PR

The VBWG established the following entrance criteria to Proposed Recommendation in the Request for CR:

1. Sufficient reports of implementation experience have been gathered to demonstrate that PLS processors based on the specification are implementable and have compatible behavior.
2. Specific Implementation Report requirements (outlined below) have been met.
3. Formal responses to all comments received by the Working Group.

5. Implementation Report Requirements

5.1 Detailed requirements for Implementation Report

1. Testimonials from implementers will be included in the IR when provided to document the utility and implementability of the specification.
2. IR must cover all specified features in the specification. For each feature the IR should indicate:
   • Feature status: required, optional, other
   • Feature utility/usefulness based on feedback from implementers
   • Implementability of the feature specification
   • Interoperability of multiple implementations of the feature
3. Feature status is a factor in test coverage in the report:
   • Required specification features must have at least two implementations. Implementations that do not implement a required specification feature must document the reason for not implementing.
   • Optional specification features must have at least 1 implementation. Implementations that do not implement an optional specification feature should document the reason for not implementing.

5.2 Notes on Testing

1. A test report must indicate the outcome of each test. Possible outcomes are pass, fail or not-impl.
2. A test report may contain an additional comment for each test. If a test fails a comment should be added (see also Detailed requirements for Implementation Report).
3. A tester is free to adapt the value of the xml:lang attribute, "ipa" pronunciations and <grapheme> orthographies to a language claimed to be supported by the processor. Such adaptation must be limited so that it does not change the logic or character of the assertion test.
4. Some tests contain notes that should be read before executing them. These notes are contained in the instructions inside the tests. See Appendix A for a detailed description of the coding rules.
5. Tests containing non-conformant PLS documents are not intended to test the conformance claim of a PLS processor. They are there for testing things like schema and may be used by implementers who may wish to test (optional) rejection of non-conformant PLS 1.0 documents by their processor.

5.3 Out of Scope

1. IR will not cover SSML processor conformance.
2. IR will not cover SRGS/SISR processor conformance.

6. Systems

Note: The testimonials with yellow background from "Acme Labs" and "Beta Inc." are just examples and will be replaced with the actually submitted testimonials.

7. Test Results

The following table lists the assertions that were gleaned from the Pronunciation Lexicon 1.0 Specification.

• The ID column uniquely identifies the assertion and links to a corresponding test.
• The Spec column identifies the section of the specification from which the assertion was derived.
• The Conformance column indicates whether or not the specification requires the PLS processor to implement the feature described by the assertion. See Detailed requirements for Implementation Report for details on test coverage.
• The Test Type column indicates whether or not the associated test can be automated or requires manual execution.
• The Category column indicates the category of the assertion; an assertion may belong to one of four possible categories:
  • general: syntactical tests;
  • both: functional tests in tts and asr context;
  • tts: functional tests in tts context only;
  • asr: functional tests in asr context only.
• The Customize column indicates whether or not the associated test needs some manual customization before execution.
• The Abstract column describes the assertion.
• The Results column tabulates the results submitted by the contributors enumerated in Systems section.

Appendices

Appendix A - PLS 1.0 Conformance Language Definition

This appendix describes a framework for authoring PLS tests.

The framework allows specifying the expected behavior of a test in a formal manner, enabling automation of testing. It provides the way to represent test input and excepted output as well as the possibility to specify instructions to execute the test. This knowledge is contained in a separated file that refers to the actual PLS test files.

In the following a description of the language elements, the related XML schema and an example of use are given.

A.1 <test> Element

The root element of the PLS conformance language is the <test> element. A <test> element contains an ordered sequence of elements consisting of an OPTIONAL <instructions> element, followed by zero or more OPTIONAL ordered sequences of input/output data, followed by a REQUIRED <lexicon> element.

The namespace URI for PLS conformance is "http://www.w3.org/2007/01/pls-conformance". All PLS conformance markup MUST be associated with the PLS conformance namespace.

A.2 <instructions> Element

The <instructions> element SHOULD be specified for tests that may be customized. The <instructions> element contains text that informs testers what may be modified in the test.

A.3 Input/Output Data

Input/Output data SHOULD be specified in the test of an assertion that is categorized as applying to either tts, asr or both. Input/Output data is specified by an ordered sequence of elements consisting of a REQUIRED <input> element, followed by one or two REQUIRED <output> elements. Note that a test contains zero or more such sequences.

A.3.1 <input> Element

The <input> element contains the orthography of the input to the PLS processor.

The <input> element MAY have a role attribute (see Section 4.4 of the Pronunciation Lexicon Specification) that is to be used by a PLS processor when selecting the most relevant pronunciations. The role attribute takes as its value one or more white-space separated QNames.

A.3.2 <output> Element

The <output> element MUST have a category attribute. Possible values are "tts" for output expected in a TTS context and "asr" for output expected in an ASR context. If an input/output data sequence contains two <output> elements then the value of the category attribute MUST be unique to each of the elements. An assertion that is categorized as applying to "both" contexts SHOULD have two <output> elements in the input/output data.

The <output> element MUST contain one or more <item> elements. When the value of the category attribute is tts, any one <item> element describes a valid output of a compliant PLS processor in a TTS context. When the value of the category attribute is asr, the output of a compliant PLS processor in an ASR context is expected to exactly match that described by the complete set of <item> elements. Note that the order of the <item> elements has no significance.

A.3.2.1 <item> Element

The <item> element contains a choice of one or more <outalias> and <outphoneme> elements. The order of these elements is significant as it is the order of the reading of the input orthography or of the content of an <alias> element.

• <outalias> Element

  The <outalias> contains the pronunciation obtained from (or partially from) an <alias> element in a PLS document.

• <outphoneme> Element

  The <outphoneme> element MUST have an alphabet attribute. Possible values are described in Section 2 of Pronunciation Lexicon Specification.

  The <outphoneme> element contains the pronunciation obtained from a <phoneme> element in a PLS document.

A.4 <lexicon> Element

The <lexicon> element MUST have a uri attribute specifying the URI that identifies the location of a PLS document with media type application/pls+xml .

The REQUIRED conformant attribute indicates whether or not the referenced PLS document is a conformant PLS document. Possible values are true for a conformant document or false for a non-conformant document. Note that it is inappropriate to specify input/output data in a test that references a non-conformant PLS document.

A.5 Schema for PLS Conformance Language

The following XML Schema defines the PLS conformance language.

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:pc="http://www.w3.org/2007/01/pls-conformance"
    xmlns:pls="http://www.w3.org/2005/01/pronunciation-lexicon"
    targetNamespace="http://www.w3.org/2007/01/pls-conformance"
    elementFormDefault="qualified"
	  version="1.0">
  <xs:import namespace="http://www.w3.org/2005/01/pronunciation-lexicon" 
    schemaLocation="http://www.w3.org/TR/2007/CR-pronunciation-lexicon-20071212/pls.xsd"/>
  <xs:attribute name="category">
    <xs:simpleType>
      <xs:restriction base="xs:token">
        <xs:enumeration value="tts"/>
        <xs:enumeration value="asr"/>
      </xs:restriction>
    </xs:simpleType>
  </xs:attribute>
  <xs:element name="instructions" type="xs:string">
    <xs:annotation>
      <xs:documentation>
        Instruction for a test that may be customized
      </xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="input">
    <xs:annotation>
      <xs:documentation>Input grapheme</xs:documentation>
    </xs:annotation>
    <xs:complexType mixed="true">
      <xs:attribute name="role" type="pls:role.datatype" use="optional"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="output">
    <xs:annotation>
      <xs:documentation>Output required from PLS processor</xs:documentation>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence maxOccurs="unbounded">
        <xs:element name="item">
          <xs:complexType>
            <xs:choice maxOccurs="unbounded">
              <xs:element name="outphoneme">
                <xs:annotation>
                  <xs:documentation>pronunciation</xs:documentation>
                </xs:annotation>
                <xs:complexType mixed="true">
                  <xs:attribute name="alphabet" type="pls:alphabet.datatype" 
                    use="required"/>
                </xs:complexType>
              </xs:element>
              <xs:element name="outalias" type="xs:string">
                <xs:annotation>
                  <xs:documentation>text</xs:documentation>
                </xs:annotation>
              </xs:element>
            </xs:choice>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
      <xs:attribute ref="pc:category" use="required"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="test">
    <xs:annotation>
      <xs:documentation>PLS conformance test</xs:documentation>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="pc:instructions" minOccurs="0"/>
        <xs:sequence minOccurs="0" maxOccurs="unbounded">
          <xs:element ref="pc:input"/>
          <xs:element ref="pc:output" maxOccurs="2"/>
        </xs:sequence>
        <xs:element name="lexicon">
          <xs:annotation>
            <xs:documentation>PLS lexicon resource</xs:documentation>
          </xs:annotation>
          <xs:complexType>
            <xs:attribute name="uri" type="xs:anyURI" use="required"/>
            <xs:attribute name="conformant" type="xs:boolean" use="required"/>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

A.6 Examples

The following examples illustrate the use of the PLS 1.0 conformance language. Each example is made up of the conformant language document (.txml) and the related PLS test document (.pls).

Example 1

The following example describes the expected pronunciation of the word theater both in tts and asr contexts. Given that pronunciation is expressed by the phoneme element in the PLS document (see next), the outphoneme element is used. The category attribute is used to represent each context.

<?xml version="1.0" encoding="utf-8"?>
<conf:test xmlns:conf="http://www.w3.org/2007/01/pls-conformance"
  xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.w3.org/2007/01/pls-conformance
    http://www.w3.org/Voice/2007/pls-irp/pls-conformance.xsd">
  <conf:input>theater</conf:input>
  <conf:output conf:category="tts">
    <conf:item>
      <conf:outphoneme alphabet="ipa">ˈθɪətər</conf:outphoneme>
    </conf:item>
  </conf:output>
  <conf:output conf:category="asr">
    <conf:item>
      <conf:outphoneme alphabet="ipa">ˈθɪətər</conf:outphoneme>
    </conf:item>
    <conf:item>
      <conf:outphoneme alphabet="ipa">ˈθiːjətər</conf:outphoneme>
    </conf:item>
  </conf:output>
  <conf:lexicon uri="example1.pls" conformant="true"/>
</conf:test>

Here is the PLS document that defines the pronunciation for the word theater.

<?xml version="1.0" encoding="utf-8"?>
<lexicon version="1.0" xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.w3.org/2005/01/pronunciation-lexicon
    http://www.w3.org/TR/2007/CR-pronunciation-lexicon-20071212/pls.xsd"
  alphabet="ipa" xml:lang="en">
  <lexeme>
    <grapheme>theater</grapheme>
    <phoneme prefer="true">ˈθɪətər</phoneme>
    <phoneme>ˈθiːjətər</phoneme>
  </lexeme>
</lexicon>

Example 2

The following example describes the expected pronunciation of the word GNU both in tts and asr contexts. Given that pronunciation is expressed in terms of alias and phoneme elements in the PLS document (see next), the outalias and outphone elements are used. The category attribute is used to represent each context.

<?xml version="1.0" encoding="utf-8"?>
<conf:test xmlns:conf="http://www.w3.org/2007/01/pls-conformance"
  xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.w3.org/2007/01/pls-conformance
    http://www.w3.org/Voice/2007/pls-irp/pls-conformance.xsd">
  <conf:input>GNU</conf:input>
  <conf:output conf:category="tts">
    <conf:item>
      <conf:outalias>GNU is Not </conf:outalias>
      <conf:outphoneme alphabet="ipa">ˈjuːnɪks</conf:outphoneme>
    </conf:item>
  </conf:output>
  <conf:output conf:category="asr">
    <conf:item>
      <conf:outalias>GNU is Not </conf:outalias>
      <conf:outphoneme alphabet="ipa">ˈjuːnɪks</conf:outphoneme>
    </conf:item>
  </conf:output>
  <conf:lexicon uri="example2.pls" conformant="true"/>
</conf:test>

Here is the PLS document that defines the pronunciation for the word GNU.

<?xml version="1.0" encoding="utf-8"?>
<lexicon version="1.0" xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.w3.org/2005/01/pronunciation-lexicon
    http://www.w3.org/TR/2007/CR-pronunciation-lexicon-20071212/pls.xsd"
  alphabet="ipa" xml:lang="en">
  <lexeme>
    <grapheme>GNU</grapheme>
    <alias>GNU is Not Unix</alias>
  </lexeme>
  <lexeme>
    <grapheme>Unix</grapheme>
    <alias>a multiplexed information and computing service</alias>
    <phoneme>ˈjuːnɪks</phoneme>
  </lexeme>
</lexicon>

Example 3

The following example sets the conformant attribute to false to refer to a non-conformant test.

<?xml version="1.0" encoding="utf-8"?>
<conf:test xmlns:conf="http://www.w3.org/2007/01/pls-conformance"
  xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"  
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.w3.org/2007/01/pls-conformance
    http://www.w3.org/Voice/2007/pls-irp/pls-conformance.xsd">
  <conf:lexicon uri="example3.pls" conformant="false"/>
</conf:test>

Here is an example of non-conformant PLS document: it doesn't contain any pronunciation.

<?xml version="1.0" encoding="utf-8"??>
<lexicon version="1.0" xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.w3.org/2005/01/pronunciation-lexicon
    http://www.w3.org/TR/2007/CR-pronunciation-lexicon-20071212/pls.xsd"
  alphabet="ipa" xml:lang="en">
<lexeme>
  <grapheme>theater</grapheme>
</lexeme>
</lexicon>

Appendix B - Downloading Tests

The pls-ir-20080710.zip archive contains a number of resources. The PLS tests are ordered by test assertion id and are organized into folders where the folder name corresponds to the assertion id. In addition the archive includes the following:

• B.1 The Manifest
• B.2 The Report Submission Template
• B.3 The Schema for PLS Conformance Language
• B.4 The DTD for the manifest
• B.5 The PLS 1.0 Implementation Report document

B.1 The Manifest

manifest.xml is a file containing the complete information about test assertions written in the PLS Implementation Report project. The structure of the Manifest presents a root element called <tests> ; this is the container of all the Test Assertions. Every Test Assertion is represented by an <assertion> element containing CDATA that represents the description of the test assertion. At the end of the file is the <contribs> element; this lists all the people who have contributed to the Implementation Report preparation. The <assertion> element must contain a <start> element that references the main test file and may optionally contain several <dep> element that identify the other tests useful to complete the test case. Here's the DTD for the Manifest:

<!ELEMENT assertion (#PCDATA)>
<!ATTLIST assertion
    id CDATA #REQUIRED
    spec CDATA #REQUIRED
    conformance (Required | Optional) #REQUIRED
    test-type (Manual | Auto) #REQUIRED
    category (General | Both | TTS | ASR) #REQUIRED
    customize (Yes | No) #REQUIRED
>
<!ELEMENT contrib EMPTY>
<!ATTLIST contrib
    usr_fname CDATA #REQUIRED
    usr_lname CDATA #REQUIRED
    usr_comp_name CDATA #REQUIRED
    editor CDATA #IMPLIED
>
<!ELEMENT contribs (contrib+)>
<!ELEMENT dep EMPTY>
<!ATTLIST dep
    uri CDATA #REQUIRED
    type CDATA #REQUIRED
>
<!ELEMENT start EMPTY>
<!ATTLIST start
    uri CDATA #REQUIRED
    type CDATA #REQUIRED
>
<!ELEMENT test (assertion, start, dep*)>
<!ELEMENT tests (test+, contribs)>

Test assertion typology is defined by several attributes on the <assertion> element. These attributes allow for a more complete identification of the nature of the current assertion and an idea of related tests structure.

• id : Test Assertion Identifier;
• spec: reference to paragraph belonging to the PLS 1.0 Specification;
• conformance: information about the required or optional nature of the current test assertion; it can assume one of the possible values: “Required”, “Optional”
• test-type: indicates whether or not the associated test can be automated or requires manual execution; it can assume one of the following possible values: Manual, Auto;
• category: indicates the category of the assertion; it can assume one of the following possible values: General, Both, TTS, ASR;
• customize: indicates whether or not the associated test need some manual customization before execution; it can assume one of the following possible values: Yes, No;

<start> and <dep> elements are characterized by the following attributes:

• uri: relative URI that links the referenced file;
• type: referenced file mime type.

For instance here’s a fragment of the manifest.xml document:

<tests>
[…]
 <test>
  <assertion id="8"   
        spec="S4.1" 
        conformance="Required" 
        test-type="Auto"
        category="General"
        customize="No">
    <![CDATA[ <lexicon> element: The values of the 'alphabet' 
    attribute are described in Section 2. (Section 2) A valid value 
    for the 'alphabet' attribute is "ipa". ]]>     
  </assertion>
  <start uri="8/8.txml" type="text/xml"/>
  <dep uri="8/8.pls" type="application/pls+xml"/>
 <test>
[…]
</tests>

The <tests> element contains the <contribs> element too: here you can find the generalities of all the contributors that have participated in the Implementation Report activity.

B.2 The Report Submission Template

The template has to be filled by the company following the rules described in Section 3. An excerpt of the Template is shown below.

<system-report name="YOUR-SYSTEM-NAME-HERE">
<testimonial> YOUR-WELL-FORMED-TESTIMOMIAL-CONTENT-HERE</testimonial>
<assert id="Id1" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
        [...]
<assert id="Idn" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert>
</system-report>

B.3 The Schema for PLS Conformance Language

pls-conformance.xsd is a file containing the schema for PLS Conformace Language, as described in the related section.

B.4 The DTD for the manifest

manifest.dtd is a file containing the DTD for the manifest, as described in the related section.

B.5 The PLS 1.0 Implementation Report document

The archive also contains the present document.

Appendix C - Contributors

The PLS 1.0 Implementation Report includes 78 assertions and corresponding tests. The Voice Browser Working Group would like to further acknowledge the contributions of several individuals for authoring and reviewing assertions and for reviewing tests:

• Jim Larson, Larson Technical Services
• Kazuyuki Ashimura, W3C
• Matt Womer, W3C