Delivery Context in MPEG-21

Anthony Vetro1 and Sylvain Devillers2
Chairman of Ad-Hoc Group on MPEG-21 Digital Item Adaptation
1 MERL - Mitsubishi Electric Research Laboratories, Murray Hill, NJ, USA
2 Philips Research France, Suresnes, France


The purpose of this paper is to provide an overview of the activity with the MPEG-21 community regarding delivery context. In December 2001, MPEG issued a Preliminary Call for Proposals on Digital Item Adaptation, which includes an extensive set of requirements related to delivery context. The final Call will be made in March 2002, and responses to the Call are due in May 2002. While the activities of MPEG are centered on various technologies for multimedia content and the activities of theW3C focus on web-centric technology, there is an increasing amount overlap between the industries that each body serves. Therefore, we hope that a strong collaboration through formal liaisons and mutual participation among the members of relevant standards bodies can be established with regards to delivery context and capability exchange.

1. MPEG-21 Overview

Moving Picture Experts Group (MPEG) is a working group of ISO/IEC in charge of the development of standards for coded representation of digital audio and video. Established in 1988, the group has produced MPEG-1, the standard on which such products as Video CD and MP3 are based, MPEG-2, the standard on which such products as Digital Television set top boxes and DVD are based, MPEG-4, the standard for multimedia for the fixed and mobile web and MPEG-7, the standard for description and search of audio and visual content. Work on the new standard MPEG-21 "Multimedia Framework" has started in June 2000. So far a Technical Report has been produced [1] and the formal approval process has already begun for several normative parts of the standard.

Today, many elements exist to build an infrastructure for the delivery and consumption of multimedia content. There is, however, no 'big picture' to describe how these elements, either in existence or under development, relate to each other. The aim for MPEG-21 is to describe how these various elements fit together. Where gaps exist, MPEG-21 will recommend which new standards are required. ISO/IEC JTC 1/SC 29/WG 11 (MPEG) will then develop new standards as appropriate while other relevant standards may be developed by other bodies. These specifications will be integrated into the multimedia framework through collaboration between MPEG and these bodies.

The result is an open framework for multimedia delivery and consumption, with both the content creator and content consumer as focal points. This open framework provides content creators and service providers with equal opportunities in the MPEG-21 enabled open market. This will also be to the benefit of the content consumer providing them access to a large variety of contents in an interoperable manner.

The vision for MPEG-21 is to define a multimedia framework to enable transparent and augmented use of multimedia resources across a wide range of networks and devices used by different communities. MPEG-21 introduces the concept of Digital Item, which is an abstraction for a multimedia content, including the various types of relating data. For example, a musical album may consist of a collection of songs, provided in several encoding formats to suit the capabilities (bit-rate, CPU...) of the device on which they will be played. Furthermore, the album may provide the lyrics, some bibliographical information about the musicians and compositors, a picture of the album, information about the rights associated to the songs and pointers to a web site where other related material can be purchased. All this aggregation of content is considered by MPEG-21 as a Digital Item, where Descriptors (e.g. the lyrics) are associated to the Resources, i.e. the songs themselves.

Conceptually, a Digital Item is defined as a structured digital object with a standard representation, identification and description. This entity is also the fundamental unit of distribution and transaction within this framework.

Seven key elements within the MPEG-21 framework have been defined. One of these elements is Terminals and Networks. Details on the other elements in this framework can be found in [1], [2]. The goal of the MPEG-21 Terminals and Networks work item is to achieve interoperable transparent access to advanced multimedia content by shielding users from network and terminal installation, management and implementation issues. This will enable the provision of network and terminal resources on demand to form user communities where multimedia content can be created and shared, always with the agreed/contracted quality, reliability and flexibility, allowing the multimedia applications to connect diverse sets of Users, such that the quality of the user experience will be guaranteed.

2. Digital Item Adaptation

Universal Multimedia Access is concerned with the access to any multimedia content from any type of terminal or network and thus it is closely related to the target mentioned above of "achieving interoperable transparent access to (distributed) advanced multimedia content". Toward this goal, and in the context of MPEG-21, we target the adaptation of Digital Items. This concept is illustrated in Figure 1. As shown in this conceptual architecture, a Digital Item is subject to a resource adaptation engine, as well as a descriptor adaptation engine, which together produce the modified Digital Item.

Concept of Digital Item Adaptation
Figure1: Concept of Digital Item Adaptation

The adaptation engines themselves are non-normative tools of Digital Item Adaptation. However, tools that provide support for Digital Item Adaptation in terms of resource adaptation, descriptor adaptation, and/or Quality of Service management are within the scope of the requirements.

With this goal in mind, it is essential to have available not only the description of the content but also a description of its format and of the usage environment in order that content adaptation may be performed to provide the user the best content experience for the content requested with the conditions available. The dimensions of the usage environment are defined in more detail below, but it essentially includes the description of terminal and networks resources, as well as user preferences and characteristics of the natural environment. While the content description problem has been addressed by MPEG-7, the description of content format and usage environments has not been addressed and it is now the target of MPEG-21 Digital Item Adaptation.

3. Requirements

In [3], high-level requirements for adaptation, and specific requirements for usage environment description and content format description have been specified. The usage environment description shall consider all the dimensions that may influence the transparent access to the multimedia content, notably terminals, networks, users and the natural environment where users and terminals are located. Additionally, the MPEG-21 Digital Item Adaptation tool-set may include MPEG-7 or any external standard tools by reference whenever these tools fulfill requirements that have been specified in this document. The content format description shall use generic tools to assist the adaptation engines.

General requirements on descriptions have been developed. These include generic properties of the description itself, such as compactness, extensibility, human readability and low-complexity. Additionally, the ability to efficiently access partial descriptions and update existing description has been targeted. With regards to Digital Rights Management, there is also a requirement to support associations of Rights Data and Expressions with descriptions for Digital Item Adaptation.

The specific requirements on usage environment description aim to describe a variety of dimensions, including terminal, network, delivery, user and natural environment capabilities. Terminal capabilities include hardware properties, such as processor speed and memory capacity, software properties such as operating system, display properties such as screen resolution, and device profiles, which may indicate the media formats supported, e.g., MPEG profile/level. Physical network conditions specify delay characteristics, such as end-to-end delay, one-way delay or delay variation, error characteristics, such as bit-error rate, packet loss or burstiness, and bandwidth characteristics, such as amount of available bandwidth or bandwidth variation. Delivery capabilities specify the type of transport protocols supported, such as MPEG-2 Systems, TCP/IP and RTP, as well as the types of connections supported, e.g., broadcast, unicast, multicast. User preferences include filtering and search preferences, browsing preferences, display preferences and QoS preferences, as well as demographic information, such as gender and age. Natural environment characteristics include location, such as GPS coordinates and locale, the type of location, e.g., indoor, outdoor, home or office, the velocity of a user or terminal, as well as the illumination properties affecting a user or terminal.

In addition to the above, the digital item adaptation descriptions may also specify service capabilities. Service capabilities include a particular users role, e.g., content creator, service provider, rights owner, billing party or end consumer, as well as the type of service that particular user provides, such as content creation, rights negotiation, billing, content adaptation and transcoding, use of the network and content consumption. Assuming that a particular user is the rights owner or content creator, digital item adaptation descriptions may also include the permissible types of adaptations that are allowed, e.g., the bit-rate should not be less that 2Mb/sec or spatial resolution of a video should not be reduced by more than a factor of two.

One important requirement regarding the usage environment description is that MPEG-21 shall provide mechanisms that support vocabularies defined by established capability description frameworks and be semantically compatible with such vocabularies. This requires MPEG-21 to be semantically compatible with, CC/PP defined by W3C, UAProf defined by WAP, CONNEG defined by IETF and any emerging works.

Beyond the usage environment description, the content format description shall provide information on how the resource itself can be adapted in a generic way. For example, several multimedia coding formats such as JPEG2000 for still images or MPEG-4 for videos feature scalable properties. By retrieving and/or decoding a part of the bitstream, it is then possible to render a degraded version of the content in terms of SNR quality, frame-rate or size without having to decode and re-encode it. The content format description shall enable generic tools to work in the compressed domain, and hence facilitate the adaptation of scalable multimedia content to terminal and network capabilities.

4. Concluding Remarks

In December 2001, MPEG issued a Preliminary Call for Proposals on Digital Item Adaptation [4], which aims at satisfying an extensive set of requirements related to delivery context [3]. The final Call will be made in March 2002, and responses to the Call are due in May 2002. A Committee Draft that will begin the formal approval process is scheduled for December 2002. A Final Draft of the International Standard is scheduled for July 2003.

Parties interested in submitting a response to the Call, or obtaining further information about the Call, should refer to [4]. Discussions on this topic are also taking place on the mailing list, which can be subscribed to by sending email to We do hope that collaboration on delivery context and capability exchange, not only with W3C, but with all related bodies, can be established.


Below references are available at under 'Hot News' or 'Working Documents'.

  1. "Information Technology - Multimedia Framework: Vision, Technologies and Strategies," ISO/IEC TR 21000-1, September 2001.
  2. "MPEG-21 Overview v.3," ISO/IEC JTC1/SC29/WG11 N4511, December 2001.
  3. "MPEG-21 Requirements on Digital Item Adaptation," ISO/IEC JTC1/SC29/WG11 N4515, December 2001.
  4. "Preliminary Call for Proposals on Digital Item Adaptation," ISO/IEC JTC1/SC29/WG11 N4514, December 2001.