Region Capture

The cropping mechanism presented in this document (cropTo) relies on Crop-session Target rather than on direct node references. This serves a dual purpose.

• It allows cropping by one document to coordinates specified in another document.

• Tagging an Element as a potential crop-target allows the user agent to avoid unnecessary work on all other elements, like the calculation of bounding boxes and sending such coordinates cross-process.

CropTarget is an intentionally empty, opaque identifier. Its purpose is to be handed to cropTo as input.

WebIDL[Exposed=(Window,Worker), Serializable]
interface CropTarget {
  [Exposed=Window, SecureContext] static Promise<CropTarget> fromElement(Element element);
};

fromElement()

Calling fromElement with an Element of a supported type associates that Element with a CropTarget. This CropTarget may be used as input to cropTo. We define a valid CropTarget as one returned by a call to CropTarget.fromElement() in a document that is still active.

When fromElement is called with a given element, the user agent creates a CropTarget with element as input. The user agent MUST return a Promise p. The user agent MUST resolve p only after it has finished all the necessary internal propagation of state associated with the new CropTarget, at which point the user agent MUST be ready to receive the new CropTarget as a valid parameter to cropTo.

When cloning an Element on which fromElement was previously called, the clone is not associated with any CropTarget. If fromElement is later called on the clone, a new CropTarget will be assigned to it.

Note

There is no consensus yet on whether producing a CropTarget should be done by invoking an asynchronous method like CropTarget.fromElement(), or a CropTarget constructor that accepts an Element as input. This is further discussed on issue #17.

To create a CropTarget with element as input, run the following steps:

1. Let cropTarget be a new object of type CropTarget.

2. Let weakRef be a weak reference to element.

   Create cropTarget.[[Element]] initialized to weakRef.

   Note

   cropTarget keeps a weak reference to the element it represents. In other words, cropTarget will not prevent garbage collection of its element.

CropTarget objects are serializable. The serialization steps, given value, serialized, and a boolean forStorage, are:

1. If forStorage is true, throw with new DOMException object whose name attribute has the value "DataCloneError".

2. Set serialized.[[CropTargetElement]] to value.[[Element]].

The deserialization steps, given serialized and value are:

1. Set value.[[Element]] to serialized.[[CropTargetElement]].

Recall that, as per [SCREEN-CAPTURE], when getDisplayMedia() is called, it returns a Promise<MediaStream>, and that this MediaStream contains exactly one video track, whose type is MediaStreamTrack.

We specify that if the user chooses to capture a browser display-surface, the user agent MUST instantiate the video track as either MediaStreamTrack, or as some sub-class of MediaStreamTrack, and that cropTo MUST be exposed on this track. For simplicity's sake, this document assumes that a subclass called BrowserCaptureMediaStreamTrack is used by the user agent.

The track MUST be initially uncropped.

WebIDL[Exposed = Window]
interface BrowserCaptureMediaStreamTrack : MediaStreamTrack {
  Promise<undefined> cropTo(CropTarget? cropTarget);
  BrowserCaptureMediaStreamTrack clone();
};

cropTo()

Calls to this method instruct the user agent to start/stop cropping a video track to the bounding client rectangle of cropTarget.[[Element]]. Since the track is restricted to the visible viewport of the display-surface, the captured area will be the intersection of the visible viewport and the element bounding client rectangle. Whenever cropTo is invoked, the user agent MUST execute the following algorithm:

1. If cropTarget is neither a valid CropTarget nor null, the user agent MUST return a Promise rejected with an UnknownError.

2. Let p be a new Promise.

3. Run the following steps in parallel:

   1. If cropTarget is neither undefined nor a valid CropTarget, reject p with a NotAllowedError and abort these steps.

   2. If cropTarget is either undefined or a valid CropTarget, the user agent MUST update this video track's crop-state according to cropTarget:

      • If cropTarget is set to undefined, the user agent MUST stop cropping. This video track reverts to the uncropped state.
      • If cropTarget is a valid CropTarget, the user agent MUST start cropping this video track to the contours of the element referenced by this CropTarget. This means that for each new frame produced on the track, the user agent calculates the bounding box of the pixels belonging to the element, and crops the frame to the coordinates of this bounding box.

   3. Call the track's state before this method invocation PRE-STATE, and after this method invocation POST-STATE. The user agent MUST resolve p when it is guaranteed that no more frames cropped (or uncropped) according to PRE-STATE will be delivered to the application, and that any additional frames delivered to the application will therefore be cropped (or uncropped) according to either POST-STATE or a later state.

      Note

      The timing of the cropTo promise resolution and the timing of the actual cropping of video frames is observable to JavaScript through MediaStreamTrack transforms. It is expected that the first newly cropped video frame will be enqueued on the MediaStreamTrack ReadableStream just after the cropTo promise is resolved.

4. Return p.

clone()

When a BrowserCaptureMediaStreamTrack is cloned, the user agent MUST produce a track which is initially uncropped, regardless of the crop-state of the original track.