Status of this document
This Note was produced by the W3C SYMM Interest Group. It describes a snapshot
of certain aspects of the discussions on future work on SYMM and SMIL. Following
the approach spearheaded by the W3C HTML Working group, the Note describes
how SMIL 1.0 functionality and future SYMM functionality can be represented
as a set of markup modules. The intention was to study the feasibility of
such an approach, and the details of the proposed modularisation are preliminary
only. Please send comments to the public mailing list
www-smil@w3.org.
This document is a NOTE made available by W3C for discussion only. This indicates
no endorsement of its content, nor that W3C has, is, or will be allocating
any resources to the issues addressed by the NOTE.
Table of Contents
-
Abstract
-
1. Introduction
-
2. Synchronized Multimedia Modules
-
3. Module Integration
-
4. Multimedia Profiles
-
5. Issues
-
References
This document describes a proposed modularization of synchronized multimedia
functionality, based upon the SMIL 1.0 Recommendation. The proposed
modules complement HTML/CSS modules to support the definition of multimedia
profiles that combine SYMM and HTML/CSS modules, as well as other XML-based
languages. A set of example profiles for multimedia use is described.
The document is intended to form a basis for work by a future SYMM Working
Group.
The first W3C Working Group on Synchronized Multimedia (SYMM) developed
SMIL - Synchronized Multimedia Integration Language.
This XML-based language is used to express synchronization relationships
among media elements. SMIL 1.0 documents describe multimedia presentations
that can be played in SMIL-conformant viewers.
SMIL 1.0 provides limited means for integrating with HTML, CSS or other XML
languages. This integration has been an area of considerable interest
within the W3C as well as other external organizations (such as the
ATSC T3/S17 specialist group). The
W3C SYMM Interest Group has been
tracking these issues in preparation for future work on the SMIL language..
The W3C HTML Working Group is working
towards a modularization of HTML. The
W3C CSS Working Group has had
similar discussions. One reason that the HTML Working Group is pursuing
modularization is to support the use of HTML in a wider range of applications.
In support of this activity, the HTML Working Group and the SYMM Interest
Group have been discussing strategies for integrating multimedia functionality
into HTML. Several proposals have been submitted and are being discussed
(HTML+TIME and BHTML).
One approach to solving this problem is to modularize synchronized multimedia
functionality, and make these modules available to other languages.
1.1. Goals and Requirements
There are four goals for modularizing synchronized multimedia functionality:
-
G1: provide a means for the W3C and third parties to integrate timing
models into other languages (such as HTML).
-
G2: provide a means for the W3C to extend SMIL with new or optional
features.
-
G3: provide a means for third parties to extend SMIL with
domain-specific features.
-
G4: provide a means for third parties to integrate other languages
into SMIL applications.
In essence, modularizing SMIL is the first step towards the integration of
timing and other key aspects of SMIL with other Web languages. This
modularization provides a means for other languages, such as HTML and an
XML-based vector-graphics format, to benefit from timing and synchronization
functionality while reusing the lessons learned and work embodied in SMIL.
This becomes especially important for the development of cross-language tools
which can readily transcode syntax but generally have a difficult time porting
content to different timing and content models.
This Note describes the current approach to this problem by the SYMM Interest
Group. The modules described herein satisfy the following design requirements:
-
R1: modularize SMIL 1.0 without meaningful changes to the SMIL 1.0
content model.
-
R2: modularize SMIL 1.0 without meaningful changes to the SMIL 1.0
timing model.
-
R3: specify SMIL 1.0 modules that complement HTML modules.
-
R4: adopt new W3C recommendations when appropriate and not in conflict
with other requirements.
Several additional requirements will inform the work of a future SYMM working
group:
-
R5: All of the modules should have integrated support for the document
object model. This facilitates additional control via script and custom
user agents (e.g. for playback controls). Support for the respective
object model is optional for all profiles. In designing the object
model for facilitating script control, it should be investigated whether
declarative solutions can also be provided, and, where possible, be specified.
-
R6: Interfaces should be defined that provide means for
integration between 3rd party player/renderers and browser engines.
Time-based behaviors of browsers should also be available to 3rd party
player/renderers. For example, applications may interface with a HTML renderer
and with an independent audio and video renderer. The result should
be tightly integrated multimedia presentations.
-
R7: Where practical, modules should be defined isomorphic with modules
from other W3C recommendations. This will assist module sharing across
profiles. Targetting for isomorphic modules requires bilateral
cooperation. Alignment of overlap in functionality concerns in particular
the HTML module definitions.
-
R8: It must be possible to define a profile that is 100% backwards
compatible with SMIL 1.0.
-
R9: For the purpose of integration with other profiles, modules
may take another syntax. However, it is required that the semantics are
maintained.
The purpose of the requirements is to ensure that the modularization makes
no significant changes to the SMIL 1.0 language while laying a foundation
for integration with HTML, CSS, and other Web Languages. This Note provides
a discussion of current integration strategies and proposals.
The SYMM Interest Group recommends that the results presented in this Note
form the basis for some of the work of a future SYMM Working Group.
1.2. Modules and Profiles
Modularization of synchronized multimedia functionality provides a building
block for language, application, and platform developers. This Note recognizes
that in addition to functional modules, there exists the notion of profiles
and platforms.
For the purposes of this Note, we define the terms module,
profile, and platform:
-
module
-
A module is a collection of elements. Each element is in one and only one
module.
-
profile
-
A profile is a collection of modules particular to an application domain
or language. For example: The SMIL profile corresponds to the collection
of modules that make up the SMIL language. A enhanced television profile
would correspond to the collection of modules for media-enhancement of broadcast
television.
-
platform
-
A platform is a collection of profiles, non-Web technologies, policies, resource
requirements, etc.
This Note focuses on modules and profiles, and is not concerned with specifying
or characterizing platforms. Nevertheless, it is worth noting the distinction
between a profile and a platform: a profile generally describes a document
type and its usage, whereas a platform encompasses the entire
environment. (The term "profile" as defined by the HTML WG is likely
to bear a wider scope than used in this Note.)
This Note proposes a set of modules for SYMM functionality. The first
section describes the modules associated with functionality defined in SMIL
1.0. A second section describes proposed extensions to SMIL 1.0.
2.1. SMIL 1.0 Modules
-
Structure
-
Includes the <smil>, <head> and <body> elements.
These elements will be used in documents based upon traditional SMIL profiles.
-
-
Meta
-
Includes the <meta> element. The elements contain information describing
the document, either to inform the human user or to assist some automation.
The element will be used in documents based upon traditional SMIL profiles,
possibly in concert with meta data modules available from other efforts.
-
-
Layout
-
Includes the <layout>, <root-layout> and <region> elements,
and related attributes. This module supports the layout mechanism described
in SMIL 1.0. These elements will all be used in documents based upon
the SMIL profile.
-
-
Timing and Synchronization markup
-
This module includes timing structure (e.g. sequence and parallel timelines)
and timing control properties (e.g. begin, end, and repeat) that describe
the timing relationships among elements. This module also includes
mechanisms to control the performed synchronization of the document
(described in SMIL 1.0 as soft and hard).
This is separated from the media so that it can be applied to HTML and XML
applications independent of support for media.
Note that in some integration profiles, the specific syntax for
the markup may vary slightly with the profile (e.g. the SMIL 1.0
<seq> element may be represented as a sequence attribute if a CSS
style-based integration is used). These differences notwithstanding,
the module semantics and the timing model will be consistent
across all profiles.
The specific syntax and mechanism of integration with HTML and XML applications
is under review.
-
-
Media Elements
-
This includes the media declaration elements <ref>,
<animation>, <audio>, <img>, <video>, <text>
and <textstream>. The media declaration may relate to the object
element in HTML (this needs attention).
-
-
Linking
-
Includes the <a> and <anchor> elements.
The relation of this module to other linking modules, particularly XLink,
XPointer and HTML linking, is under review. For example when integrating
with HTML, timing applied to the (HTML) <a> and <area> elements
could provide much or all of the SMIL 1.0 defined functionality.
-
-
Content control
-
This includes the <switch> element and test attributes introduced in
SMIL 1.0. This is of general utility to authors, independent of the
multimedia functionality.
2.2. Proposed Extensions
-
Layout (extensions)
-
Additional functionality will likely be added to this module as part of ongoing
work. Relationship of this Layout module to other Layout mechanisms,
including CSS, is to be considered.
-
-
Timing and Synchronization markup (extensions)
-
Additional functionality will likely be added to the module as part of ongoing
work. Proposed extensions include event-based timing constructs, e.g.
to support user-interaction and stream events, and more detailed mechanisms
to control the performed synchronization of the document.
-
-
Transitions
-
This module includes new functionality to support transition effects on
elements. These may be media elements, or they may be traditional HTML
(or XML) elements. A CSS-style syntax under consideration might represent
transitions as in this HTML-based example:
<html>
<head>
<title>Powerpoint Animation in HTML</title>
<style>
p.caption { transition-style: wipe; font-size: big }
li { transition-style: dissolve }
</style>
</head>
<body>
<p class="caption">Multimedia Requirements</p>
<ul>
<li>Timing the Display of Bullet Items</li>
<li>Transition Effects</li>
</ul>
</body>
</html>
Specifics of the module syntax is under review. There is a desire to
present a simple syntax for typical use-cases, and also to support more complex
functionality by modeling transitions as filters with timing support. The
various proposals will be considered and refined as work proceeds.
-
Animation
-
This module includes functionality to support common simple animation like
motion paths. Where Transitions apply rendering filters, Animation
behaviors manipulate Object Model properties like position, color, size or
rotation (e.g. in an XML-based vector-graphics format).
-
-
Resource Management
-
This module includes support for resource (e.g. bandwidth) management
hints. These can facilitate user agent optimizations, and can control
the degradation strategy of the user agents. For example, the author
may assign priorities to elements to inform a user-agent that will dynamically
balance network bandwidth. Other proposed markup includes hints for cueing
or preloading media and a means of specifying that an individual media element
is delivered as a substream within an aggregated media stream (for large
streamed presentations).
The HTML Working Group is still reviewing formal mechanisms to define or
specify profiles. Several approaches are under consideration including
DTDs, XML Schemas, and RDF-based mechanisms. The SYMM Activity will
provide associated requirements and use-cases to the HTML Working Group,
and will follow the recommendation of the HTML Working Group for profile
specification.
Several mechanisms for integrating SYMM functionality with other languages
(especially HTML) have been discussed. It is possible that a given
module may be represented with different syntax depending upon the profile
and integration. Nevertheless, the SYMM modules will define consistent
semantics for all profiles, regardless of syntax variants.
The current variants and issues for integrating synchronized multimedia
functionality with HTML and XML languages are:
-
Defining tagsets for SMIL elements. This is a yes/no choice, and combines
with the others. I.e. A profile can either:
-
Define XML tags for SMIL elements together with SMIL attributes (as specified
with one of the mechanisms below), or
-
Define only attributes. All SMIL elements would be represented as attributes
(of some sort) applied to HTML elements.
At issue in particular are the SMIL timing structure elements <par>
and <seq>, which could be expressed as an attribute on HTML (or XML)
container elements. While this is expressed as a dichotomy, it really defines
a range of possibilities. Profiles may map some elements to
attributes and leave others as elements.
-
Attribute Syntax variants
-
XML Namespace qualified attributes
-
This has the advantage of general applicability to any XML language. It has
the disadvantage that it is impossible to set the value of an XMLNS attribute
via CSS stylesheets without defining a CSS property that corresponds to the
XMLNS attribute.
-
CSS properties
-
This has the advantage that stylesheets can be used to define timing,
transitions, etc. It the disadvantage that it requires CSS, which may
not be appropriate for some XML languages or for some profiles and platforms.
-
XSL properties
-
This is a variant on CSS properties, but would support a broader range of
XML applications.
A range of profiles have been discussed that would make use of different
sets of modules. Below is an initial list of profiles that could integrate
SYMM and HTML functionality in different ways. This is not intended
to be a final statement, but rather an indication of how modules would be
used. It provides a backdrop for the discussion of modularization.
In addition, some specific proposals can be described in terms of the relevant
modules:
-
SMIL 1.0 would be defined as the 1.0 versions
of the Structure, Meta, Layout , Timing, Media, Linking, and Content control
modules. SMIL 2.0 would include all of the new versions of these modules,
as well as some or all of the new modules proposed.
-
HTML+TIME proposes a profile that extends an
HTML 4.0 profile to include Timing, Media, Linking and Content control
modules.
-
BHTML proposes a profile that similarly extends
HTML to include Timing, Media and Linking, as well as Transition support.
4.1. Example Profiles
-
Leightweight presentations
-
A basic profile would handle simple presentations, supporting timing of text
content. The simplest version of this could be used to sequence stock quotes
or headlines on restricted platforms like palmtop devices or smart
phones. The Mobile Access group
is looking at related profiles. An HTML-based profile would combine
HTML modules (probably a subset of HTML) with the timing module. It might
include the transitions as well.
-
-
Timeline-Centric Multimedia presentations
-
This describes the profile associated most closely with SMIL 1.0. This
does not integrate HTML, and uses SMIL layout rather than the flow layout
model associated with HTML . Typical use-cases for this profile are
multimedia presentations that include time-based and streaming media. This
would include most or all of the SMIL modules.
-
-
HTML integrated Multimedia presentations
-
This is a variant on the Traditional Multimedia profile that mixes multimedia
and HTML. It mixes HTML layout (i.e. flow layout) and CSS
positioning. Use-cases for this profile are presentations that use
HTML as a media type in concert with time-based and streaming media (e.g.
PowerPoint-like presentations with time-based media). This profile would
include (most or all of) the HTML modules, the Timing, Media, Transition,
Animation, Resource Management and Content control modules (i.e. all but
the Structure, Meta and Layout modules). The linking functionality
would overlap between the Linking module and a similar HTML module.
-
-
Web Enhancement of Broadcast, On-Demand Video and DVD
-
This describes support for web presentations tied to broadcast and/or on
demand media. The primary media will often define the main timeline.
The main additional requirements here are for stream event support and the
ability to tie the page clock to the primary media. Stream event support
requires event-based timing of elements (and timelines), and a means of
associating events with document-external (stream) triggers. E.g. where the
stream could be a TV-broadcast, it would be possible to refer to the events
by referencing the TV broadcast stream or player-object. It must be
possible to ensure that the multimedia (enhancement) timeline is strictly
synced to or driven by the primary (broadcast) media.
Note that where "webcasting" may associate Web-pages to a TV-broadcast, in
"enhanced television" additional media form an integral part with the main
video presentation, providing enhancement of and interaction with the program.
While the typical use-cases may be dominated with the primary media (e.g.
streaming video broadcast or DVD video), the authoring model is still closely
tied to the Web and HTML. This profile would likely use the same modules
as the HTML integrated Multimedia presentations profile, but with the additional
timing and synchronization requirements described.
Syntax
The details of the elements and properties still need to be worked out.
In particular, there are open issues on the syntax for timing markup, and
the mechanism for integration with HTML and XML applications, and stylesheet
tools like CSS and XSL.
Accessibility syntax
An additional issue that must be coordinated with representatives of the
Web Accessibility Initiative (WAI) concerns accessibility support.
This includes elements and attributes which assist the accessibility of content
declared in the presentation. In SMIL 1.0, this concerns the title, alt and
longdesc attributes.
It is not clear whether this should be an additional module, or additional
support in specific modules. In particular, the intent is not to define
a new set of Accessibility mechanisms, but to integrate current work of expert
groups in this area.
-
Broadcast
HTML
-
A Modular Hypertext Markup Language for Broadcast Applications. HTML working
group contributed paper. Ted Wugofski. This is available at:
http://toocan.philabs.research.philips.com/misc/atsc/bhtml/
-
HTML+TIME
-
Timed Interactive Multimedia Extensions for HTML (HTML+TIME) Note 18 September
1998, Patrick Schmitz, Jin Yu, Peter Santangeli. This is available at:
http://www.w3.org/TR/NOTE-HTMLplusTIME
-
Reformulating
HTML in XML
-
W3C Working Draft 5 December 1998, Dave Raggett, Frank Boumphrey, Murray
Altheim, Ted Wugofski. This is available at:
http://www.w3.org/TR/1998/WD-html-in-xml-19981205
-
SMIL 1.0
-
Synchronized Multimedia Integration Language (SMIL) 1.0 Specification 15
June 1998, Philipp Hoschka. This is available at: http://www.w3.org/TR/REC-smil