Questions & Answers blog

Here's a hint: binary blobs are not acceptable, even as a stop-gap. Fedora, which is a nice baseline for these sorts of things as it's strictly FOSS and its parent, RedHat is extremely patent averse, doesn't ship the no-cost GStreamer binary MP3 playback plugin, so the chances of it shipping a binary H.264 blob is zero.

Also, aren't the P3P and Eolas cases instances of submarine patents? If so, surely that means you could use the "full force of the W3C" to address the patent situation of Ogg Theora.

I'm particularly sympathetic to comments such as this one about the rich network of support for H.264:

...There's far more FOSS support for MPEG-4 and H.264 than for Ogg/Theora ...

FOSS support for H264 is illegal, because the patent holders prohibit distributing software which implements the algorithms under terms compatible with FLOSS licensing.

Just because small time infringers are not rigorously pursued, as in some other industries, does not make the situation acceptable. For example, Microsoft does not pursue users of unlicensed Windows as hard as the MPAA pursues infringers of music copyrights, but that does not make an unlicensed copy of Windows software any more legal than an unlicensed implementation of the H264 codec.

It is unacceptable to exclude FOSS software from music/video on the Internet.

Yes, we can investigate the patent situation around Ogg Theora, but if we can clear the hurdles around H.264, it's not clear what critical need Theora would fill.

p.s. James, have we met? Do you keep a blog or homepage or the like?

The first and last terms of Ian Hickson's wishlist are too hard to satisfy and misses the point. Actually, what we need is not a codec that "do not require a per-unit or per-distributor licensing", the license just has to be permissive enough. Xiph's license could hardly be more permissive.

All innovation is an "additional submarine patent risk". Rather than wish for a miracle, we should seek and support, at least suggest the least patented soulution. Submarines are only a part of the patent problem, focusing only on those misses the point. It is particularly unfair to disqualify Vorbis/Theora on this point. Think of it, all competitors are more patented.

I think H.264 would be a good codec to use in the HTML 5 standard. The patent issues need to be resolved first though. Also the W3C could use it's full force to resolve the second issue. Actually it's not like Microsoft has always been implementing standards the way they should, so why should anyone actually care about their point of view?

Not wanting to delve into the CODEC can of worms (but wishing they'd just use OGG/Theora), can anyone tell me why <video> is preferable to using <object> ? And if it is preferable why introduce a new (non-backward-compatible) tag rather than an attribute to specify media type on the object element?

It would be nice to see W3C working actively in this area to try and open the patent minefield. Although I hope with more success than in the EOLAS case - although that patent still seems entirely without merit, the end result of all the work is apparently that it is available on discriminatory royalty-bearing terms, which is about as bad as it was at the beginning.

H.264 is currently not compatible with the open source model, so it is currently not a valid option. If MPEG-LA could lax the terms a little so Open Source decoders (not encoders) could be implemented without any licensing, it would be a very strong contender.

I think the best thing would be for W3C to put official force behind an outreach and plea to MPEG-LA to revise the licensing terms so at least H.264 and perhaps even the whole of MPEG-4 can be implemented license- and royalty-free in Open Source.

I think too many people are focusing on the H.264 standard's commercial codec implementations, as there are FLOSS alternatives out there: http://www.videolan.org/developers/x264.html

However, I am not familiar with the patent landscape surrounding the H.264 standard, so I will refrain from commenting upon that. It is, however, well worth investigating this venue.

If you look the Anime fansub-scene, which seems to be in the future when it comes to adopting standards that turns out to be commonplace, then you'll see that the combination MKV/x264 seems to be the second most common combination. Only beaten by AVI/Xvid, and that is only used for NON-HDTV/low-quality content.

Both MKV and x264 are FLOSS, and as such must be excellent candidates for W3c?

Another idea, what could be worth thinking about, is using system codecs. Would make it much easier to keep the codecs up to date, remove a whole lot of wasps nests, and decrease the complexity neccesary to implement the video playback (by not having codecs implementet directly into the browser). Not that I've kept myself up-to-date on the HTML4 WD, so my apologies if this has already been discussed.

This report I wrote last year may be of interest " FOSS Codecs for Online Video: Usability, Uptake and Development". It is a review of the best available tools for the creation, playback and embedding of online video using Free and Open Source Software video codecs, and a set of recommendations for development to enhance their adoption by social change video projects on the web.

< a href="http://wiki.transmission.cc/index.php/FOSSCodecsForOnlineVideo:UsabilityUptakeandDevelopment_1.2"> Report Viewable Online

or available as a PDF here:

Download PDF

You say "I have been studying digital media enough to be confident I could produce Ogg/Theora or Dirac; but I'm not sure my audience can consume it." I understand your hesitation here. We authored this report to identify the areas for improvement to get Ogg Theora to a stage where it "just works".

But as with all digital distribution strategies you must think about who your audience is, and what technology they are able to use. Eg. the excellent VLC media player (VideoLan Client) plays Ogg in addition to just about any other format/codec you can name, and is available cross-platform. GUIs are now available to transcode to Ogg on Windows, Mac and Linux. Options for embedded playback of Ogg are being developed by projects such as Metavid and Wikipedia.

You might need to help your audience a little and educate them as you go along, and there are gaps in usability that do need to be filled. But the future is looking brighter for Ogg. I for one will very much enjoy the ability to play back Theora video files in FF without the need for a plugin - in fact this will be a big step-forward for usability that users will no doubt rally behind.

You state that the Ogg codecs (Theora/Vorbis/Flac/Speex) have the risk of a submarine patent claim. However, this is the case with all codecs. Every conceivable piece of technology could be patented without you knowing about it. There is no reason to trust any codec to be free of patent claims.

Theora is based on On2's VP3 codec. On2 has granted a universal royalty-free license on the patented technology. This means that On2 won't sue you. But another company might if they claim that you infringe their patent.

H.264 is an MPEG4 codec. MPEG-LA has not granted a royalty-free license. They will definitely sue you. If you convince MPEG-LA to grant a royalty-free license, still another company might claim that you infringe their patent, and therefore sue you.

So, there is NO difference, except that Theora is free as far as we know it, and H.264 is NOT. Even if you convince MPEG-LA to relinquish their patents, it might still be subject to submarine patents, like any other technology.

I guess Apple and NOKIA should really pack their stuff and go home while they still can, because they might be sued for infringing the patent on:

"An apparatus that looks like HTML5, but can't play video to save its life."

h.264 all the way.

ogg is nice but too much politics

I think HTML5 should support open standard all the way. It is not just a matter of ideology, it's a matter of compatibility, portability and non-discriminating choices for every user.

Regarding the audio I don't think there's any discussion: Vorbis is the only feasible solution.

The fact that there are some open implementation of h.264 (x264 by VLC developers and ffh264 by the ffmpeg team) does not justify the our support for it. It may very well be that tomorrow we'll not be able to play those files, or that Apple decides to make its codec a little bit different for commercial or DRM realted purposes... who knows.

The point is: it should be accessible from any device, running under any OS, any architecture, and through any browser that supports the HTML5 specs. Period. Somehow I don't feel like we are going to achieve that with h.264.

So either we find a way to build the implementation around x264 or libavcodec (ffh264) or we take Theora.

Theora is in final beta stage before the official release of version 1.0. Xiph.org have stated that the performance characteristics of the current Theora reference implementation are mostly dominated by implementation issues inherited from the original VP3 code base. An internal document exists, which lists the known implementation problems and gives an example of how improving one aspect of the encoder can lead to visibly improved quality. Current work on Theora is focused on an experimental version, which targets correcting aspects of the encoder which were identified in that paper as being suboptimal. This experimental version is supposed to replace the current encoder in a future Theora release.

In big point in favour of Ogg (vorbis+theora) is that a HUGE website like Wikipedia (~#7 most visited in the world) choose to adopt Ogg for all a/v files. If the HTML5 supported really open standards it would make the companies change the way the think products and consumers.

Open standars please, for the sake of everyone.

Hi Federico,

for small devices, we have to keep in mind that the issue is not only software related, but also materials. Basically, small devices have to include video processors to not drain the batteries. So deployment and updates are a bigger issue.

I can certainly understand the worries about submarine patents, but there is no way of avoiding the problem. Imagine, for example, that H.264 was adopted as a mandatory codec. All the big browser players negotiate a deal with MPEG-LA which allows them to ship a decoder. At this point in time, MPEG-LA has an incentive to offer a deal at a reasonable price. If they asked for silly amounts of money, the community would not adopt H.264 at all.

Now suppose that, unknown to you, I have a patent on some essential aspect of H.264. I could license my patent to MPEG-LA and collect a small share of the royalties. On the other hand, I might make more money by holding on to it and keeping quiet. Once H.264 is used universally on the web, I come out of hiding. Now I can ask for $50M or $100M for licensing, because everyone is committed to supporting H.264.

It's tempting to think you can sign a license with MPEG-LA, and then implement H.264 risk free. In fact, you have no guarantee that all the IP owners are participating in that group. If I really did own IP in H.264 then there would be arguments both ways. Participating would give me a low risk income, and staying out would be higher risk while giving me the chance of a huge payoff.

Once you've given your money to MPEG-LA, you have licensed all known patents. Similarly, when you've downloaded Theora for free, you have licensed all known patents. The situation is identical, but in one case you have paid money and prevented legal open source implementations. We can conclude that Theora is better in some ways (free licensing of known patents) and identical in all other ways (submarine patents). No one can guarantee that any codec is free of submarine patent claims, even in theory; Theora is as good as it gets. IMHO this means it is the only reasonable candidate for the mandatory algorithm.

It is possible that MPEG-LA would agree to free licensing of decoders, while charging for encoders. This would make H.264 a better choice, but still not as good as Theora, which is completely free. The ability to do open source web development is important, just as it's important to have a standards-compliant open source browser.

There does indeed seem to be a consensus behind H.264, as mentioned in the quotation. The difference is that the web is designed to be open in a way that a set-top box, for example, is not. H.264 is a reasonable choice for a closed system like a set-top box, but on the web, it doesn't achieve the essential objectives of openness and non-discrimination. Following consensus only makes sense if the original design decisions can be justified in the new situation, and in this case, they can't be.

Finally, we could have no mandatory codec at all. This would be better than mandating a non-free codec, but it would harm interoperability. Interoperability is, of course, the point of a specification like HTML 5, so it would be a shame if things went this way.

Why doesn't the W3C call for the abolition of software patents? They're clearly impeding standardization efforts. This seems like a socially useful function for the W3C to perform.