Video on the Web

W3C Architecture Domain Leader,
Co-Chair of the W3C Workshop on Video on the Web

These slides at http://www.w3.org/2008/03/video-on-the-web

What's on the Internet tonight?

The Internet is becoming consumers' primary entertainment source […]
— Saul Berman, IBM Media & Entertainment Strategy and Change

Marjorie "Marge" Simpson using the Web (© Fox Broadcasting Company)
© Fox Broadcasting Company

There is a decline of TV as Primary Media Device.
Nearly 75% of US Internet users watched online video
Video producers are looking at ways to increase their reach.
Broadcast TV advertisement is a $60B market, Internet (not counting search) is $10B. $4.3B spend in online video advertising over the next 4 years. Online video ad revenues are growing 55.5% per year.

Video Forecast

Cisco's Global Consumer Internet Traffic Forecast

Cisco's Global Consumer Internet Traffic Forecast
Source: Cisco, 2007

Video is driving IP traffic growth
thus consumer bandwidth will increase (but never enough)
Increase in volume of video content
In 2011, nearly 30% is Internet Video, additional 30% due to Video P2P

Increase in Video Online Content

User Generated Video Uploads per day

User-Generated Video Uploads per Day
Source: IDC, 2007

So, what does all this tell us?

YouTube was Just the Beginning: launched in 2005, take 4% of the US Internet Traffic by the end of 2006.
While TV brought us information and entertainment, online video adds the social aspect dimension, it's a means of expression. YouTube and MySpace are good examples. The value is the combination of communications and content.
Multiple and fragmentation of multiple video platforms
Drive towards higher quality, ideally HD. Camcorders and HDTV are getting deployed.
User Generated Content helps drives the number

The Long Tail… of Video

The future of entertainment is in the millions of niche markets at the shallow end of the bitstream.
— Chris Anderson, Wired magazine

The number of viewers is inversely proportional to its popularity rank
With online and mobile video, million of titles are viewed by a relatively small audience
Notion is that the total amount of viewing of less popular content is large than the total amount of viewing of popular content.
But, it needs to be available, for a small price, and you need to help consumers find it. (see Amazon, Netflix, Rhapsody)

Video at W3C

We do video related work in several areas, each more or less affecting different video issues

Synchronized Multimedia Activity: Timed Text (closed captioning), SMIL (video element/API, closed captioning)
HyperText Activity: HTML 5 (video element/API)
Graphics Activity: SVG (video element/API)
Semantic Web Activity: SWD, SWEO
Architecture Activity: EXI (container formats)
WAI Activity: Protocols and Formats (closed captioning), Guidelines
Mobile Web Initiative (device independence)
Rich Web Applications
Privacy Languages Interest Group
Multimedia Semantics XG (closed)

Lots of activities can potentially impact video

W3C Workshop (1/2)

W3C Video on the Web Workshop

12-13 December 2007, San Jose, California and Brussels, Belgium

Hosted by Cisco Systems

W3C Video Workshop Telepresence
Photo credit: Chris Lilley

Make video a first-class object on the Web

W3C Workshop Participants

Accenture, Adobe Systems, Annodex Association, Apple, CBS Interactive, CableLabs, Canon, Capgemini, CWI, Cisco Systems, Dailymotion, Hitachi, Hot Potato, IBBT, IJS, MTV Networks, Meraka Institute, Motorola, Move Networks, Mozilla Foundation, Nemours Foundation, Nokia Corporation, RealNetworks, Samsung Electronics, Sony, Sun Microsystems, Thomson, Turner Broadcasting System, University Of California at Santa Cruz, University of Wollongong, VRT medialab, Via Licensing, Walt Disney Internet Group, Web3D Consortium, Wright State University, YouTube, thePlatform

Steve Bratt makes a point in Black and White
Photo credit: Robert Freund

Laundry list

Codecs and formats
Metadata
Content protection
Spatio-temporal fragment
Accessibility
Tagging
Searchability
Synchronization, Temporal properties for styling
Distribution: cost, QoS, tracking, mobile, etc.
Playlist files
<video> HTML 5
…

Accessibility is still an issue

HTML 5 <video>

<video src='MyVideo'
        id='vid'
        controls
        autoplay
>

vid.play();
vid.pause();
vid.muted = true;
vid.addCueRange("ranges", 10, 20, false, enterCallBack, exitCallBack);

Flash Video examples

Flash 6: flv container

Stream #0: Video: flv
Stream #1: Audio: mp3

Flash 8: flv container

Stream #0: Video: vp6
Stream #1: Audio: mp3

Flash 9: MPEG-4 container

Stream #0: Video: h264/avc
Stream #1: Audio: he-aac

Video codecs

H.264/AVC: supported by Apple/Quicktime/Safari, Adobe/Flash, Mobile industry (3GPP), Joost, HD DVD/Blue Ray, DVB, etc.
VC-1: supported by Microsoft/Silverlight, HD DVD/Blue Ray.
Ogg/Theora: Supported by Mozilla, Opera, and the KDE team.
Dirac: Developed by the BBC. I-frame only will become VC-2.
H.263: Older codec but still widely spread.

Video codecs are still evolving: H264/AVC SVC (2007), H.265 (2012?), …

HTML 5 Requirements for a Video codec

Known to not require per-unit or per-distributor licensing

Compatible with the open source development model

Of sufficient quality as to be usable

Not an additional submarine patent risk for large companies

None of the codecs fit the requirements for HTML 5…

Metadata

Many standards: MPEG 7, SMIL, iTunes XML, Yahoo! MediaRSS, CableLabs VOD Metadata Content, etc.
Different sources: Content Creator, Publisher, Professional Content, Users.
Survivability: allowing the metadata to survive along the delivery chain (transcoding, republication, syndication, etc.).
Ability to add metadata without touching the data

Goals:

Search
Discover

Content protection

Pushed by the content owners: monetizing commercial content
Enforce usage rights
Monitor content usage
DRM, watermarking, fingerprinting

Spatial Fragments (1/2)

Beth, the audience, Belgium, her pitch, and herself
Photo credit: Robert Freund

Spatial Fragments (2/2)

Can use SVG or MPEG-7, but require indirection

<svg:rect id="beth"
       x="14.64" y="15.73" width="146.98" height="147.48"
       style="opacity:1;fill:none;fill-opacity:1;
              fill-rule:nonzero;stroke:#ff0000;stroke-opacity:1"
/>

<SpatialDecomposition>
  <StillRegion id="beth">
    <SpatialMask>
      <SubRegion>
        <Box>14.64 15.73 161.62 163,21</Box>
      <SubRegion>
    </SpatialMask>
  </StillRegion>
</SpatialDecomposition>

Temporal Fragments (1/2)

http://dig.csail.mit.edu/2007/09/OneWebDay

Temporal addressing

Motivations: Playlists of video highlights, Search results, Mashups, Caching
Addressing: offset, segment, named event

Temporal Fragments (2/2)

SMIL and MPEG-7 require an indirection

<video src="http://dig.csail.mit.edu/2007/09/OneWebDay"
         region="r1" clipBegin="34.45"/>

TemporalURI and MPEG-21 approaches extend the URI syntax

http://dig.csail.mit.edu/2007/09/OneWebDay?clipBegin=34.45

http://dig.csail.mit.edu/2007/09/OneWebDay#clipBegin(34.45)

Issue with ? and #
Issue with # and RFC3986

Next steps

Codecs and formats: is it possible to meet the HTML 5 requirements?
New Video Activity: 5 Working Groups
1. Media Fragments Working Group (video, audio, and images)
  http://www.w3.org/2008/01/media-fragments-wg
2. Media Annotation Working Group: Develop an ontology and map it to existing formats
3. Guidelines for Media Content on the Web (see next slide)
4. Finishing the efforts of SYMM and TimedText Working Groups (SMIL and DXFP)

Those are immediate steps

Guidelines for Media Content on the Web

Scope is deployment, use, and manipulation of media content.
Targeted towards user agents, authoring tools, protocols, and formats.

Items on the list:

Baseline codecs and encapsulation format to use
Functionalities needed for Video API
Accessibility recommendations
Copyright and licensing information