HTML Media Capture

Table of Contents

• 1. Introduction
• 2. Conformance
• 3. Security and Privacy Considerations
• 4. Capture aware file-select control
• 5. The capture parameter
• 6. WebIDL interfaces
  • 6.1 Example
  • 6.2 MediaFileData interface
    • 6.2.1 Attributes
  • 6.3 MediaFile interface
    • 6.3.1 Methods
  • 6.4 MediaFileDataSuccessCallback interface
    • 6.4.1 Methods
  • 6.5 MediaFileDataErrorCallback interface
    • 6.5.1 Methods
  • 6.6 MediaFileDataError interface
    • 6.6.1 Attributes
    • 6.6.2 Constants
• A. User Interface Examples
• B. References
  • B.1 Normative references
  • B.2 Informative references

1. Introduction

The HTML Form Based Media Capturing specification defines a new interface for media files, a new parameter for the accept attribute of the HTML input element in file upload state, and recommendations for providing optimized access to the microphone and camera of a hosting device.

Providing streaming access to these capabilities is outside of the scope of this specification.

The Working Group is investigating the opportunity to specify streaming access via the proposed <device> element.

2. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words must, must not, required, should, should not, recommended, may, and optional in this specification are to be interpreted as described in [RFC2119].

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [WEBIDL], as this specification uses that specification and terminology.

3. Security and Privacy Considerations

This specification builds upon the security and privacy protections provided by the [HTML5] <input type="file"> and the [FILE-API] specifications; in particular, it is expected that any offer to start capturing content from the user’s device would require a specific user interaction on an HTML element that is entirely controlled by the user agent.

In addition to the requirements already highlighted in the [HTML5] and [FILE-API] specifications, implementors should take care of additional leakage of privacy-sensitive data from captured media. For instance, embedding the user’s location in a captured media metadata (e.g. EXIF) might transmit more private data than the user might be expecting.

4. Capture aware file-select control

This section is normative.

[HTML5] links <input type="file"> to the File interface. This specification defines a refined MediaFile interface to be used when the accept attribute take certain values — this will require coordination with the HTML5 Working Group.

If an input element in the File Upload state [HTML5] contains accept attribute with values image/*, sound/*, or video/*, the user agent can invoke a file picker that allows respectively the user to take a picture, record a sound file, or record a video in addition to selecting an existing file from the file system.

See the User Interface Examples appendix for the illustration.

In case the user chooses to capture video, audio, or image content, the user agent creates media files on the fly as specified in [HTML5].

If the user selects files of whose MIME types match image/*, sound/*, or video/* (on the filesystem or via a successful media capture), the relevant files in the files attribute [HTML5] must implement the MediaFile interface.

<input type="file" accept="image/*" id="capture"> 

5. The capture parameter

This section is normative.

The capture parameter may be specified on the media type values of the accept attribute to provide user agents with a hint of that by the default a file picker should be in media capturing mode.

[HTML5] defines the accept attribute to take no parameters on MIME types. This specification proposes to use a MIME type parameter — this will require coordination with the HTML5 Working Group.

The capture parameter can take one of the following values: camera, camcorder, microphone, filesystem. These values indicate which source the file picker interface should preferably present to the user by default.

The values and their exact meaning are still very much in flux.

For example, the following code indicates that the user is expected to upload an image from the device camera:

<input type="file" accept="image/*;capture=camera" id="capture"> 

A possible rendering of a file picker taking this parameter into account is offered in the User Interface Examples appendix.

6. WebIDL interfaces

var captureInput = document.getElementById('capture');
// Accessing the file object from the input element with id capture
var file = captureInput.files[0];
if (file) {
  // getting format data asynchronously
  file.getFormatData(displayFormatData, errorHandler);
}

// success callback when getting format data
function displayFormatData(formatData) {  
  var mainType = file.type.split("/")[0]; // "image", "video" or "audio"
  var mediaDescriptionNode = document.createElement("p");
  if (mainType === "image") {
    mediaDescriptionNode.appendChild(document.createTextNode("This is an image of dimensions " + formatData.width + "x" + formatData.height);
  } else {
    mediaDescriptionNode.appendChild(document.createTextNode("Duration: " + formatData.duration  + "s");
  }
  captureInput.parentNode.insertBefore(mediaDescriptionNode, captureInput);
}

// error callback if getting format data fails
function errorHandler(error) {
  alert("Couldn’t retrieve format properties for the selected file (error code " + error.code + ")");
}

[NoInterfaceObject]
interface MediaFileData {
    attribute DOMString     codecs;
    attribute unsigned long bitrate;
    attribute unsigned long height;
    attribute unsigned long width;
    attribute float         duration;
};

[NoInterfaceObject]
interface MediaFile : File {
    void getFormatData (in MediaFileDataSuccessCallback successCallback, in optional MediaFileDataErrorCallback errorCallback);
};

[Callback=FunctionOnly, NoInterfaceObject]
interface MediaFileDataSuccessCallback {
    void onSuccess (in MediaFileData formatData);
};

[Callback=FunctionOnly, NoInterfaceObject]
interface MediaFileDataErrorCallback {
    void onError (in MediaFileDataError error);
};

[NoInterfaceObject]
interface MediaFileDataError {
    const unsigned short UNKNOWN_ERROR = 0;
    const unsigned short TIMEOUT_ERROR = 1;
    readonly attribute unsigned short code;
};