Today

Everything about Video!

• HTML
• Basic APIs
• Fragment URI
• CSS
• Graphics: Canvas and SVG
• Capture
• Streaming
• Real Time Communication
• Content protection
• Captioning
• Accessibility
• Performance

The World Wide Web Consortium (W3C)

• Lead Web to its Full Potential
• 385 Members
• Web ecosystem: users, developers, browsers, etc.

How does W3C work?

• Focus is to create standards that lead to public and commercial benefits
• W3C accepts inputs from all and provides outputs to all
• Each participation brings their expertise, perspective
• 65 Groups. Each has engineers from member companies and W3C staff support
• Liaisons with many organizations: IDPF, IETF, OMA, DLNA, ISO JTC1, ITU-T, Open IPTV Forum, SMPTE, EBU, more.

How can you participate?

• Community Groups:
  build a community around your idea for the Web
• WebPlatform.org:
  Your Web, documented
• Test the Web Forward:
  Better tests for a better web

Let's dive

• HTML
• Accessibility
• Basic APIs
• Fragment URI
• CSS
• Graphics: Canvas and SVG
• Capture
• Streaming
• Real Time Communication
• Content protection
• Captioning
• Performance

HTML5 <video>

<video controls preload="auto" poster="poster.png">
 <source src="trailer.mp4" type='video/mp4'>
 <source src="trailer.webm" type='video/webm'>
 <track kind='subtitles' src='brave.en'
           srclang='en' label="English">
 <track kind='captions' src=brave_bis.en.vtt'
           srclang='en' label="English for the Hard of Hearing">
 <track kind='subtitles' src='brave.fr.vtt'
           srclang='fr' label="Français">
 <p>Your user agent does not support the HTML5 Video element.

</video>

Basic APIs

videoElement.currentTime
videoElement.play()
videoElement.pause()
videoElement.addEventListener("ended", ...);
video.audioTracks[1].enabled = true;

See HTML5 Video Events and API

HTML Media Groups

link multiple media elements together

  <video src="movie.vid#track=Video&amp;track=English"
            mediagroup=movie controls id=v1>
  </video>
  <video src="movie.vid#track=sign"
            mediagroup=movie>
  </video>
  <script>
   v1.controller.play();
  </script>

Fragment URI

http://www.example.com/example.mp4#xywh=160,120,320,240
http://www.example.com/example.webm?t=10,20

Codecs and formats

CSS

CSS properties: opacity, background (multiple), resize, border-radius

Canvas: ctx2D.drawImage(video, 0, 0)

See also ctx3D.texImage(target, level, internalformat, format, type, video)

Captioning

• HTML has support for external text tracks for media elements, such as subtitles, captions, descriptions, chapters, metadata
• HTML does not define or recommend a text track format
• Two specifications for timed text formats: Timed Text Markup Language (TTML) 1.0, WebVTT
• Browser support varies

Timed Text (TTML)

• Produced by the W3C Timed Text Working Group, Group created in January 2003
• TTML 1.0 became a W3C Standard in November 2010
• Handles various features: styling, regions, animations
• Based on XML and XSL Formating Objects (XSL FO)
• Originally intended for interchange among authoring systems
• Only implemented in Desktop IE

WebVTT

• Started in WHATWG, stabilized by W3C Web Media Text Tracks Community Group
• Based on SRT and CSS
• W3C is investigating whether to standardize the format
• Intended for user-generated content market
• Desktop support in Chrome, Safari, and IE

Captioning Considerations

• SMPTE-TT, a superset of TTML 1.0, has broad support in video industry
• WebVTT has general "support" from the browser vendors
• FCC rule
  
  • Require closed captioning of IP-delivered video programming (previously on TV)
  • Establish requirements for captioning features
  • Declare SMPTE-TT as safe harbor

Fullscreen

API and CSS pseudo selectors for fullscreen support

myElement.requestFullscreen();

document.exitFullscreen();
document.fullscreenEnabled;

section:fullscreen {
  border: none;
}

Media Accessibility

User Requirements

• By type of disability: blindness, atypical color perception, deafness, deaf-blind, …
• By alternatives: described video, clean audio, sign translation, …
• By System Requirements: time-scale, discovery, access to controls, …

Media Capture

Local audio and video access

 function start() {
   navigator.getUserMedia({audio:true, video:true}, gotStream);
   startBtn.disabled = true;
 }
 function gotStream(stream) {
    video.src = URL.createObjectURL(stream);
    video.onerror = function () {
      stream.stop();
    };
   stream.onended = function () {
     startBtn.disabled = false;
   };
 }

See getUserMedia example

Media Source Extensions

• Fine-grained control: app developer constructs media stream from a series of video or audio chunks.
• Use cases
  • Adaptive streaming (when various resolutions available from server)
  • Ad-insertion
  • Time-shifting
  • Video editing
• Even when single stream, can improve performance by allowing parallel downloads of chunks that are recombined
• Also works with live streams
• We expect to see innovation in adaptive algorithms.

Encrypted Media Extensions

• W3C has a proposal for an API to control playback of protected content.
  • Connects Content Management System (CMS) to content provider serving encrypted media
• Challenges to this approach:
  • Opacity of existing CMS
  • Assumes interoperability between CMS and service delivering the key.
  • Relationship with others is work in progress:
    • Canvas, CSS
    • Assistive technology

Real Time Communications

• Peer Connection
  • P2P Channel for video, audio, data
  • Codec control
  • Encryption
  • Bandwidth management
• Session setup (signaling)
• DataChannel
• ICE, SDP

Performance

• See Steve if you're using too much video on your page :-)
• Measurements
• Saving power

Navigation Timing

client-side latency measurements within applications

function onPlay() {
  var now = new Date().getTime();
  alert("User-perceived autoplay loading time: "
        + now - performance.timing.navigationStart);
}
video.addEventListener("play", onPlay, false);

fetchStart, requestStart, responseEnd, etc.

Page Visibility

Determine the current visibility of a page

document.hidden
document.addEventListener('visibilitychange', ...);

function onvisibilitychange() {
  if (document.hidden)
   video.pause();
  else
   video.play();
}

requestAnimationFrame

Script-based animations where the user agent is in control of limiting the update rate of the animation

function animate(time) {
  if (!document.hidden) {
    // if visible, draw
    canvas.drawImage(myVideoElement, 0, 0);
  }
  window.requestAnimationFrame(animate);
}
function start() {
  window.requestAnimationFrame(animate);
}

Questions?

• HTML
• Basic APIs
• Fragment URI
• CSS
• Graphics: Canvas and SVG
• Capture
• Streaming
• Real Time Communication
• Content protection
• Captioning
• Accessibility
• Performance