ISSUE-7: codec support and the <video> element

video-codecs

codec support and the <video> element

State:
CLOSED
Product:
HTML 5 spec
Raised by:
Sam Ruby
Opened on:
2007-11-08
Description:
## Editor's position, 30 Jun 2009
http://www.w3.org/mid/Pine.LNX.4.62.0906300444010.1648@hixie.dreamhostps.com

[[
After an inordinate amount of discussions, both in public and privately,
on the situation regarding codecs for <video> and <audio> in HTML5, I have
reluctantly come to the conclusion that we have no way to get consensus on
this issue. In particular, I am not aware of any codec that every browser
vendor is willing to implement and that video publishers are interested in
using to publish their videos.

I have therefore removed the two subsections in the HTML5 spec in which
codecs would have been required, and have instead left the matter
undefined, as has in the past been done with other features like <img> and
image formats, <embed> and plugin APIs, or Web fonts and font formats.
]]



## Research

* Opera support for and examples of <video> - http://people.opera.com/howcome/2007/video/

## Emails, IRC

* Doug Schepers "Mandated Video Format" - http://lists.w3.org/Archives/Public/public-html/2007Apr/0252.html
* HÃ¥kon Wium Lie, Opera "RE: Video proposals" - http://lists.w3.org/Archives/Public/public-html/2007JanMar/0232.html
* Ian Hickson Re: <video> element feedback - http://lists.w3.org/Archives/Public/public-html/2007Oct/0082.html
* Chris Wilson on IRC: "Our legal people do not believe that Ogg Theora has much of any chance of being free from patent issues." - http://krijnhoetmer.nl/irc-logs/html-wg/20071108#l-1549


## Preamble

The HTML5 specification contains new elements to allow the embedding of audio and video, similar to the way that images have historically been embedded in HTML. In contrast to today's behavior, using object, where the behavior can vary based on both the type of the object and the browser, this allows for consistent attributes, DOM behavior, accessibility management, and so on. It also can handle the time-based nature of audio and video in a consistent way.

However, interoperability at the markup level does not ensure interoperability for the user, unless there are commonly supported formats for the video and audio encodings, and the file format wrapper. For images there is no mandated format, but the widely deployed solutions (PNG, JPEG/JFIF, GIF) mean that interoperability is, in fact, achieved.

## Licensing

The problem is complicated by the IPR situation around audio and video coding, combined with the W3C patent policy <http://www.w3.org/Consortium/Patent-Policy-20040205/>. "W3C seeks to issue Recommendations that can be implemented on a Royalty-Free (RF) basis." Note that much of the rest of the policy may not apply (as it concerns the specifications developed at the W3C, not those that are normatively referenced). However, it's clear that at least RF-decode is needed.

## Candidates

There are, of course, a number of codecs and formats that can be considered. A non-exhaustive list might include a variety of 'public' codecs, as well, of course, as proprietary ones:

1. open-source projects: the ogg family (vorbis, theora), and the BBC Dirac video codec project
2. Current ISO/IEC (MPEG) standard codecs, notably the MPEG-4 family: AVC (14496-10, jointly published with the ITU as H.264), AAC (part of 14496-3)
3. Older MPEG codecs, notably MPEG-2 layer 3 (aka MP3), MPEG-2 layer 1 and 2 audio, and maybe MPEG-4 part 2 video (14496-2)
4. Current standard codecs from other bodies; SMPTE VC-1, for example
5. Older standards from other bodies: ITU recommendations H.263 (with or without its many enhancement annexes) or even H.261
6. Very old standard codecs, formats, or industry practices; notably the common format for video from digital still cameras (Motion JPEG with uncompressed audio in an AVI wrapper)
7. Proprietary codecs, such as Dolby AC-3 audio

## Candidate concerns

There are concerns or issues with all of these:
1. a number of large companies are concerned about the possible unintended entanglements of the open-source codecs; a 'deep pockets' company deploying them may be subject to risk here;
2. the current MPEG codecs are currently licensed on a royalty-bearing basis;
3. this is also true of the older MPEG codecs; though their age suggests examining the lifetime of the patents;
4. and also SMPTE VC-1
5. H.263 and H.261 both have patent declarations at the ITU. However, it is probably worth examining the non-assert status of these, which parts of the specifications they apply to (e.g. H.263 baseline or its enhancement annexes), and the age of the patents and their potential expiry.
6. This probably doesn't have significant IPR risk, as its wide deployment in systems should have exposed any risk by now; but it hardly represents competitive compression.
7. Most proprietary codecs are licensed for payment, as that is the business of the companies who develop them.

## Other licensing concerns

It's also possible that there are other issues around licensing:
1. variations in licensing depending on filed patents in various geographies
2. restrictions on usage, or fees on usage, other than the fees on implementation (e.g. usage fees on content sold for remuneration).

It's not entirely clear, also, whether 'implementing' HTML means the ability to decode and display, or whether encoding is also included. Including encoding in the equation might significantly complicate matters.

## Possible action

The members of the WG are engineers, not IPR experts. There is general consensus that a solution is desirable, but also that engineers are not well placed to find it:
1. they are not experts in the IPR and licensing field;
2. many of them are discouraged by their employers from reading patents or discussing IPR.

It's clear that the December workshop cannot be silent on this subject. There must be recognition of the issue and evidence of at least efforts to solve it, and preferably signs of progress.

It is probable that this is best handled in parallel with the technical work, and headed by someone 'technically neutral' and qualified, such as W3C technical and legal staff. A good start would be to:
1. examine the declaration, licensing, and patent expiry situation for various codecs;
2. contact the licensing authorities for various codecs to determine their level of interest and flexibility, and possibly invite them to the December workshop.
3. analyze the open-source codecs for their risk level, and possibly seek statements from patent owners if that is deemed prudent.

HTML5-SPEC-SECTIONS [video]
Related Actions Items:
Related emails:
  1. Re: Google and MPEG LA Announce Agreement Covering VP8 Video Format (from silviapfeiffer1@gmail.com on 2013-03-09)
  2. Re: Google and MPEG LA Announce Agreement Covering VP8 Video Format (from rubys@intertwingly.net on 2013-03-08)
  3. Re: Change Proposals, objections, and the Decision Policy (from fielding@gbiv.com on 2010-06-14)
  4. Re: ISSUE-7: video-codecs - Chairs Solicit Proposals (from rubys@intertwingly.net on 2009-12-09)
  5. Minutes HTML WG 3 December 2009 (from cooper@w3.org on 2009-12-03)
  6. {agenda} HTML WG telcon 2009-12-3: issues, action items, calls for consensus/proposals, task force reports (from mjs@apple.com on 2009-12-02)
  7. Re: Change Proposal status and deadlines (from mjs@apple.com on 2009-11-25)
  8. Re: {agenda} HTML WG telcon 2009-11-19: issues, action items, calls for consensus/proposals, task force reports (from oradnio@gmail.com on 2009-11-19)
  9. Re: {agenda} HTML WG telcon 2009-11-19: issues, action items, calls for consensus/proposals, task force reports (from laura.lee.carlson@gmail.com on 2009-11-19)
  10. {agenda} HTML WG telcon 2009-11-19: issues, action items, calls for consensus/proposals, task force reports (from Paul.Cotton@microsoft.com on 2009-11-18)
  11. {agenda} HTML WG telcon 2009-11-19: issues, action items, calls for consensus/proposals, task force reports (from Paul.Cotton@microsoft.com on 2009-11-18)
  12. Re: Topics for 'Video' Breakout Group (from silviapfeiffer1@gmail.com on 2009-11-06)
  13. Re: Topics for 'Video' Breakout Group (from laura.lee.carlson@gmail.com on 2009-11-06)
  14. Re: Topics for 'Video' Breakout Group (from silviapfeiffer1@gmail.com on 2009-11-05)
  15. Topics for 'Video' Breakout Group (from silviapfeiffer1@gmail.com on 2009-11-05)
  16. minutes Re: {agenda} HTML WG telcon 2009-10-29: issues, action items, calls for consensus/proposals, task forces, F2F... (from cooper@w3.org on 2009-10-29)
  17. Re: {agenda} HTML WG telcon 2009-10-29: issues, action items, calls for consensus/proposals, task forces, F2F... (from singer@apple.com on 2009-10-29)
  18. {agenda} HTML WG telcon 2009-10-29: issues, action items, calls for consensus/proposals, task forces, F2F... (from rubys@intertwingly.net on 2009-10-28)
  19. RE: Clarification on old issues and rationale -- Change Proposal vis a vis Issue-7. (from david.dailey@sru.edu on 2009-10-28)
  20. Re: ISSUE-7: video-codecs - Chairs Solicit Proposals (from silviapfeiffer1@gmail.com on 2009-10-28)
  21. Chairs Solicit Proposals or recommend closing for issues: 4, 7, 14, 30, 63 (from rubys@intertwingly.net on 2009-10-26)
  22. ISSUE-7: video-codecs - Chairs Solicit Proposals (from rubys@intertwingly.net on 2009-10-26)
  23. [VER 2] {agenda} HTML WG telcon 2009-10-15: AIs, Task Forces, F2F meeting plans (from Paul.Cotton@microsoft.com on 2009-10-15)
  24. Re: {agenda} HTML WG telcon 2009-10-15: AIs, Task Forces, F2F meeting plans (from singer@apple.com on 2009-10-14)
  25. {agenda} HTML WG telcon 2009-10-15: AIs, Task Forces, F2F meeting plans (from Paul.Cotton@microsoft.com on 2009-10-14)
  26. minutes of 2009-10-01 telcon (from mike@w3.org on 2009-10-02)
  27. Re: Agenda for HTML WG telcon 2009-09-24 - CfCs, Accessibility TF, Testing TF, etc. (from mjs@apple.com on 2009-09-30)
  28. Re: Agenda for HTML WG telcon 2009-09-24 - CfCs, Accessibility TF, Testing TF, etc. (from singer@apple.com on 2009-09-30)
  29. Agenda for HTML WG telcon 2009-09-24 - CfCs, Accessibility TF, Testing TF, etc. (from mjs@apple.com on 2009-09-30)
  30. Re: Spec with issue markers [was: Re: HTML5-warnings - request to publish as next heartbeat WD] (from rubys@intertwingly.net on 2009-08-24)
  31. State of HTML WG Unresolved Issues (from mjs@apple.com on 2009-08-20)
  32. Re: Consolidated issues that may qualify as 'controversial' (from rubys@intertwingly.net on 2009-08-12)
  33. Re: Consolidated issues that may qualify as 'controversial' (from shelleyp@burningbird.net on 2009-08-12)
  34. Consolidated issues that may qualify as 'controversial' (from mjs@apple.com on 2009-08-12)
  35. Re: Feedback on the current editor's draft (from shelleyp@burningbird.net on 2009-08-11)
  36. Re: Feedback on the current editor's draft (from rubys@intertwingly.net on 2009-08-11)
  37. Re: Feedback on the current editor's draft (from shelleyp@burningbird.net on 2009-08-11)
  38. Re: Feedback on the current editor's draft (from rubys@intertwingly.net on 2009-08-11)
  39. controversial issue candidates in issues 1-15 (from mjs@apple.com on 2009-08-10)
  40. RE: <video>RE: Codecs for <video> and <audio> (survey of positions on ISSUE-7 video-codecs) (from jfoliot@stanford.edu on 2009-07-28)
  41. Re: <video>RE: Codecs for <video> and <audio> (survey of positions on ISSUE-7 video-codecs) (from singer@apple.com on 2009-07-28)
  42. Re: Codecs for <video> and <audio> (survey of positions on ISSUE-7 video-codecs) (from connolly@w3.org on 2009-07-28)
  43. Re: Codecs for <video> and <audio> (survey of positions on ISSUE-7 video-codecs) (from rubys@intertwingly.net on 2009-07-28)
  44. Re: Codecs for <video> and <audio> (survey of positions on ISSUE-7 video-codecs) (from connolly@w3.org on 2009-07-28)
  45. Re: <audio> and <video> Support - formal objection to one vendor/one vote (from joedwil@earthlink.net on 2009-07-13)
  46. Re: formal objection to one vendor/one vote (from joedwil@earthlink.net on 2009-07-13)
  47. Re: formal objection to one vendor/one vote (from singer@apple.com on 2009-07-13)
  48. Re: formal objection to one vendor/one vote (from rubys@intertwingly.net on 2009-07-13)
  49. Re: formal objection to one vendor/one vote (from singer@apple.com on 2009-07-13)
  50. Re: formal objection to one vendor/one vote (from rubys@intertwingly.net on 2009-07-13)
  51. Re: formal objection to one vendor/one vote (from singer@apple.com on 2009-07-13)
  52. Re: formal objection to one vendor/one vote (from rubys@intertwingly.net on 2009-07-13)
  53. Re: formal objection to one vendor/one vote (from faulkner.steve@gmail.com on 2009-07-13)
  54. Re: formal objection to one vendor/one vote (from rubys@intertwingly.net on 2009-07-11)
  55. Re: formal objection to one vendor/one vote (and group decision process for, e.g. ISSUE-7 video-codecs) (from shelley.just@gmail.com on 2009-07-09)
  56. minutes HTML weekly 9 July for review (from connolly@w3.org on 2009-07-09)
  57. Re: formal objection to one vendor/one vote (and group decision process for, e.g. ISSUE-7 video-codecs) (from connolly@w3.org on 2009-07-09)
  58. Re: FYI: Firefox 3.1 Will Support Ogg (ISSUE-7 video-codecs) (from sam.kuper@uclmail.net on 2008-07-31)
  59. Re: FYI: Firefox 3.1 Will Support Ogg (ISSUE-7 video-codecs) (from shadow2531@gmail.com on 2008-07-30)
  60. FYI: Firefox 3.1 Will Support Ogg (ISSUE-7 video-codecs) (from connolly@w3.org on 2008-07-30)
  61. {minutes} HTML WG issue-tracking telcon 2008-06-19 (from mike@w3.org on 2008-06-25)
  62. {minutes} HTML WG teleconference 2007-12-20 (from mike@w3.org on 2007-12-21)
  63. Re: Video codec requirements changed [ISSUE-7 video-codecs] (from connolly@w3.org on 2007-12-12)
  64. ISSUE-7 (video-codecs): <video> codec [HTML 5 spec] (from sysbot+tracker@w3.org on 2007-11-08)

Related notes:

note Proposal from Sayer 07 Jul 2009
http://www.w3.org/mid/4A53CD71.10707@mozilla.com

Dan Connolly, 9 Jul 2009, 14:46:40

Seems worthwhile to link http://www.w3.org/Bugs/Public/show_bug.cgi?id=7010 "Making <applet> non-conforming may hinder royalty-free <video> adoption"

Shawn Medero, 30 Jul 2009, 16:48:20

Display change log ATOM feed


Paul Cotton <Paul.Cotton@microsoft.com>, Maciej Stachowiak <mjs@apple.com>, Sam Ruby <rubys@intertwingly.net>, Chairs, Michael[tm] Smith <mike@w3.org>, Staff Contact
Tracker: documentation, (configuration for this group), originally developed by Dean Jackson, is developed and maintained by the Systems Team <w3t-sys@w3.org>.
$Id: index.php,v 1.325 2014-09-10 21:42:02 ted Exp $