W3C

Speech Synthesis Markup Requirements
for Voice Markup Languages

W3C Working Draft 23 December 1999

This version:
http://www.w3.org/TR/1999/WD-voice-tts-reqs-19991223
Latest version:
http://www.w3.org/TR/voice-tts-reqs
Previous version:
http://www.w3.org/Voice/Group/1999/tts-reqs-19991118
Editor:
Andrew Hunt

Abstract

The W3C Voice Browser working group aims to develop specifications to enable access to the Web using spoken interaction. This document is part of a set of requirements studies for voice browsers, and provides details of the requirements for markup used for speech synthesis.

Status of this document

This document describes the requirements for markup used for speech synthesis, as a precursor to starting work on specifications. Related requirement drafts are linked from the introduction. The requirements are being released as working drafts but are not intended to become proposed recommendations.

This specification is a Working Draft of the Voice Browser working group for review by W3C members and other interested parties. This is the first public version of this document. It is a draft document and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress".

Publication as a Working Draft does not imply endorsement by the W3C membership, nor of members of the Voice Browser working groups. This is still a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite W3C Working Drafts as other than "work in progress."

This document has been produced as part of the W3C Voice Browser Activity, following the procedures set out for the W3C Process. The authors of this document are members of the Voice Browser Working Group. This document is for public review. Comments should be sent to the public mailing list <www-voice@w3.org> (archive) by 14th January 2000.

A list of current W3C Recommendations and other technical documents can be found at http://www.w3.org/TR.

0. Introduction

The main goal of this subgroup is to establish a prioritized list of requirements for speech synthesis markup which any proposed markup language should address. This document addresses both procedure and requirements for the specification development. The requirements are addressed in separate sections on Design Criteria, Architecture and Integration, Text Content, Speech-Specific Rendering, and Miscellaneous followed by links to Further Reading Material.

0.1 Process

The specification development process will consist of the following steps:

  1. Collect requirements on speech synthesis markup and prioritize those requirements.
  2. Distribute requirements to, and take feedback from, relevant groups working on specific markup languages for speech synthesis.
  3. Develop a specification based on the requirements for delivery to the W3C Voice Browser Working Group.

1. Design Criteria

The markup language for speech synthesis will be developed within the following broad design criteria. They are ordered from higher to lower priority. In the event that two goals conflict, the higher priority goal takes precedence. Specific technical requirements are addressed in the following sections.

  1. The markup language for speech synthesis will enable consistent control of voice output by speech synthesizers for use in voice browsing and in other contexts. Consistent rendering of speech synthesis markup should be possible across multiple platforms and multiple speech synthesis implementations.
  2. The markup language for speech synthesis will be an XML Application and shall be interoperable with relevant W3C specifications (see the interoperability requirements for details).
  3. The markup language for speech synthesis should be appropriate for speech output from a wide range of computer applications with varying speech content (see the application requirements for details).
  4. The markup language for speech synthesis will be internationalized to enable speech output of a large number of languages (see the mono-lingual and multi-lingual requirements).
  5. It should be easy to automatically generate, author by hand and process documents using the markup language for speech synthesis.
  6. All features of the markup language for speech synthesis should be implementable with existing, generally available technology. Anticipated capabilities should be considered to ensure future extensibility (but are not required to be covered in the specification).
  7. The number of optional features in the markup language for speech synthesis will be kept to an absolute minimum, ideally zero. For optional features, it is highly desirable that a reasonable rendering behavior be available when not implemented fully by a speech synthesizer. (See also the compliance requirement.)
  8. Documents written in the markup language for speech synthesis should be human-legible and reasonably clear and the specification should avoid unnecessary terseness.
  9. The element set should avoid unnecessary differences with HTML, XHTML, ACSS and other relevant specifications. (See also the interoperability requirements.)
  10. The markup language for speech synthesis specification should be prepared quickly, where appropriate deriving from existing, applied specifications.

2. Architecture and Integration

2.1 Speech generation from HTML, XHTML, DOM, XSL, ACSS etc. (must have)

It must be practical to generate speech synthesis output from a wide range of existing document representations. Most importantly speech output from HTML, HTML plus ACSS/CSS, XHTML, XML plus XSL, and DOM must be possible.

2.2 Speech generation from application (must have)

It must be practical for a wide range of applications to automatically generate speech synthesis output. Key examples include voice browsers, email readers, web browsers, and accessibility applications.

2.3 Integration with other Voice Markup (must have)

The speech synthesis markup must be interoperable with other relevant specifications developed by the W3C Voice Browser Working Group. It must be possible to embed speech synthesis markup into the dialog markup for prompt generation and other spoken output. It must be possible to utilize pronunciations defined in a standard pronunciation format. It must be possible to utilize speech synthesis markup for universal access. (See also 5.2 Event generation.)

2.4 Mono-modal output (must have)

The speech synthesis markup must be appropriate in the context of an audio-output-only (mono-modal) user interaction.

2.5 Multi-modal output (must have)

The speech synthesis markup must be appropriate in the context of an multi-modal system output, most importantly, in combination with visual output. Where appropriate, synchronization of speech and other output should be supported with SMIL or a related standard. (Also see 5.2 Event generation.)

3. Text Content

3.1 Document Structure (must have)

The speech synthesis markup must support the ability to indicate document structure in a way that is instructive to a speech synthesizer for rendering the document. The specification must provide a well-defined set of document structure elements. At a minimum, it must be possible to mark paragraph and sentence structures. Dialog types and other structural elements with distinctive spoken style will be considered in the specification process.

3.2 Mono-lingual document (must have)

The speech synthesis markup must support the ability to incorporate and render text of a single language in a single document and to mark the language content appropriately.

3.3 Multi-lingual document (should have)

The speech synthesis markup may support the ability to incorporate and render text of more than one language in a single document where those languages are supported by the speech synthesizer. The levels in document structure in which language change is permitted would be determined during the specification process as the definition of the speech synthesis document structure emerges.

3.4 Phonemic pronunciations (must have)

The speech synthesis markup must provide the ability to specify pronunciation entities as sequences of phonemes. Phonetic pronunciation models may also be considered.

3.5 Reference to externally defined pronunciations (should have)

The speech synthesis markup may support the ability to reference extenally defined pronunciation or lexicon documents. In particular, if the Voice Browser Working Group defines a lexicon format it must be possible to reference it from the speech synthesis markup. [In the absence of a Working Group proposal, there are no obvious candidates for standard externally referencable lexicon formats.]

3.6 Out-of-vocabulary handling (Nice to have)

The speech synthesis markup may support a mechanism to request particular handling of out-of-vocabulary text or other unpronunciable text. [This may instead be an API design issue and out of the scope of these Speech Synthesis Markup Requirements.]

3.7 Acoustic-phonetic sequences (should have)

The speech synthesis markup may provide a mechanism to exactly specify the desired acoustic-phonetic rendering of a given text segment. This may be accomplished with a sequence of high-level phonetic and phonemic symbols, accompanied by a detailed acoustic information for rendering the phonetic and phonemic symbols such as duration, pitch movement, intensity, etc.

3.8 Special text constructs (must have)

The speech synthesis markup must provide the ability to mark a set of common text constructs that require special handling by speech synthesizers. The list should include dates, times, numbers, phone numbers, currency amounts, URLs, postal addresses and measures. A mechanism should also be provided to indicate locale or other information that enables a speech synthesizer to incorporate dates and other locale-sensitive constructs.

3.9 Spelling - literal output (must have)

The speech synthesis markup must provide the ability to mark regions of text for "spelled" or literal output, as appropriate to the text language.

3.10 Non-speech output (must have)

The speech synthesis markup must provide the ability to incorporate non-speech audio output. This may include references to audio files (e.g. wave and MIDI files) that are linked inline. This may also include generation of a set of defined audio samples such as touch-tone or other commonly used prompt sounds.

3.11 Within document natural language generation (future revision)

The speech synthesis markup may provide a mechanism to allow on-the-fly generation/modification of output text. For example, based on dialog context or based on what a user said to a dialog system, the speech output may choose the appropriate verb/noun to echo the user's spoken words. Another example could be the use of style sheets to apply style rules to control how things like dates are transformed before being spoken.

4. Speech-Specific Rendering

4.1 Speaking voice control (must have)

The speech synthesis markup must provide the ability to indicate a speaking voice for a document or for regions of text within a document. A set of common speaking voice (or speech font) characteristics must be defined and may include gender, age, name and instance selection (where multiple voices have common characteristics; e.g. two male voices).

4.2 Emphasis (must have)

The speech synthesis markup must provide the ability to mark words and other regions of text for spoken emphasis (also referred to as prominence or stress).

4.3 Intonation control (should have)

The speech synthesis markup may provide the ability to mark words and other regions of text with intonational characteristics including boundary tones (rise or fall at sentence/phrase end) and sentential intonation (movements across phrases/sentences).

4.4 Acoustic prosodics (must have)

The speech synthesis markup must provide the ability to mark regions of text with acoustic characteristics such as pitch, pitch range, speaking rate and volume.

4.5 Synchronized facial animation (nice to have)

The speech synthesis markup may provide the ability to mark text with features that enhance synchronized facial animation. Features may include positions of physical facial features (e.g. lip rounding, jaw position, eye brow movements), timing data, and expressions (e.g. smile).

4.6 Spatial audio (nice to have)

The speech synthesis markup may provide a mechanism for generating spatial audio (also known as 3D audio). For instance, this could request that the voice output be in the upper-right quadrant forward of the listener. It may also allow the voice location to shift over time.

5. Miscellaneous

5.1 Compliance Definition (must have)

The specification must address the issue of compliance by defining the sets of features that must be implemented for a system to be considered compliant with the specification. Where appropriate, compliance criteria may be defined with variants for different contexts or environments.

5.2 Event generation (must have)

The speech synthesis markup may provide methods to mark points in text output or segments of text output to generate callbacks, event notification or other information that can be used to track progress of text output, that determine timing and location of barge-in for an appropriate resume, that can be used to trigger other activities, or that can be used to synchronize speech output with other output modalities. The mechanisms by which event notifications are issues are outside the scope of the speech synthesis markup specification. (See also, Integration with SMIL).

5.3 Pause/resume behavior (should have)

The speech synthesis markup specification may define behavior of implementations with respect to pausing and resuming audio output. Beyond the typical instant stop/start model (a tape player paradigm) some consideration could be given to specifying word boundaries or other locations where pausing is reasonable for a listener. Similarly, the markup may enable a mechanism to indicate appropriate locations to resume output that may be different from the pause location. [These capabilities may be more of an environment of API issue than a markup issue.]

5.4 Comments (must have)

The speech synthesis markup must support a mechanism for inline comments. [Presumably the parent markup language, e.g. XML, will provide such a mechanism.]

5.5 Engine extensibility (should have)

The speech synthesis markup may need to define a mechanism by which specific speech synthesizer implementations can provide enhancements or non-standard extensions without affecting the core specification behavior.

6. Further Reading Material

The following resources are related to the Speech Synthesis Markup Language requirements and specification.

SABLE
(http://www.research.att.com/~rws/Sable.v1_0.htm)
SABLE is a markup language for controlling text to speech engines. It has evolved out of work on combining three existing text to speech languages: SSML, STML and JSML. Implementations are available for the Bell Labs synthesizer and in the Festvial speech synthesizer. The following are two of the papers written about SABLE and its applications:


Java Speech API Markup Language
(http://java.sun.com/products/java-media/speech/forDevelopers/JSML/index.html)
JSML is an XML specification for controlling text-to-speech engines. Implementations are available from IBM, Lernout & Hauspie and in the Festival speech synthesis platform and in other implementations of the Java Speech API.

Spoken Text Markup Language
(http://www.cstr.ed.ac.uk/publications/1997/Sproat_1997_a.ps)
STML is an SGML language for controlling text to speech engines developed jointly by Bell Laboratories and by the Centre for Speech Technology Research, Edinburgh University.

Microsoft Speech API Control Codes
(http://www.microsoft.com/iit/)
SAPI defines a set of inline control codes for manipulating speech output by SAPI speech synthesizers.

VoiceXML Prompts
(http://www.voicexml.com/)
The Voice XML specification for dialog systems development includes a set of prompt elements for generating speech synthesis and other audio output that are very similar to elements of JSML and SABLE.