Bug 15861 - API for JS interaction with congestion control
API for JS interaction with congestion control
Status: REOPENED
Product: WebRTC Working Group
Classification: Unclassified
Component: WebRTC API
unspecified
All All
: P2 normal
: ---
Assigned To: Web RTC Working Group
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-02-03 05:00 UTC by Randell Jesup
Modified: 2014-04-16 17:51 UTC (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Randell Jesup 2012-02-03 05:00:07 UTC
I propose we add a callback on PeerConnection and some attributes on it
and MediaTracks:

{
...
    // allow app to limit total bitrate used for the PeerConnection
    attribute unsigned long maxBitrate;
    attribute readonly unsigned long currentBitrate;  // [Open issue] - how much smoothing?

    attribute Function? onBitrateChange;
    // don't notify until change since last notify is more than this
    attribute unsigned long minBitrateChangeToNotify;
    attribute Function? onResolutionChange;  // not strictly needed

    // These are global settings that can be overridden on a per-track basis
    attribute float minFramerate;
    attribute float maxFramerate; // might not be needed
    attribute boolean preferMotion;  // false means preference for Resolution
}


To each MediaTrack:

{
...
    // If the app bumps up the currentBitrate of one track and doesn't lower another track, the
    // system will make cuts to keep the total bitrate within bounds.
    attribute unsigned long currentBitrate;
    attribute short priority;  // for allocating bits

    // The rest of these are for video (is short guaranteed enough, always?  Probably)
    attribute unsigned short width;  // Let app override the values chosen automatically
    attribute unsigned short height;
    attribute float framerate;
    attribute float minFramerate;
    attribute float maxFramerate; // might not be needed
    attribute boolean preferMotion;  // false means preference for Resolution

    // typically derived from QP
    attribute readonly unsigned short videoQuality;
    // audioQuality?? (samplerate?)

}

Should some of these be echoed in MediaTrackLists or MediaStreams?
Comment 1 Cullen Jennings 2012-03-19 16:39:18 UTC
This probably needs a bunch of list discussion first
Comment 2 Cullen Jennings 2012-05-18 14:57:26 UTC
This has been OBE (Overcome by events) with the Constraints stuff.
Comment 3 Randell Jesup 2012-05-19 00:54:38 UTC
Strongly disagree; this has nothing at all to do with constraints and barely anything to do with getUserMedia.  This is in-call interaction with the congestion-control-based allocation of bits.

No matter where that issue ends up, we want to have this.  I don't see this as extremely contentious, though details may get modified.

Reopening for this reason.
Comment 4 Randell Jesup 2012-08-06 23:08:39 UTC
I'll note for completeness that closing this bug was apparently not intentional; it got confused with another bug somehow.
Comment 5 Cullen Jennings 2012-10-24 19:23:20 UTC
Any chance of progress on this at W3C meeting coming up ?
Comment 6 Gili 2013-09-04 03:46:46 UTC
I'd like to propose the following:

1. The user is allowed to specify "fence conditions" and a callback. An example of fence conditions are minimum or maximum bandwidth.
2. If WebRTC is forced to "cross" one of these boundaries, it fires the callback.
3. The callback should respond by altering the video properties and/or fence conditions and the process repeats ad infinitum.

For example, I set a minimum bandwidth of 50kbps, and maximum bandwidth of 1000kbps. If the minimum bandwidth is crossed, I decide to either increase the video resolution or (if the video is as big as I'd like) reduce the minimum bandwidth fence. If the maximum bandwidth is crossed, I respond by reducing the video resolution or (if the video is as small as I'd like) increase the maximum bandwidth fence.

How should bitrate, frame rate and resolution constraints interact with one another? Which should get priority over the other? This approach puts the application in control, because ultimately the application is the only one that knows the answer to that question.

Obviously you can layer timers on top of this. Meaning, application can decide to react only if the maximum bandwidth fence is crossed for over 3 seconds (and ignore it otherwise).

This approach is not specific to congestion control. It can be applied to all other constraints as well.
Comment 7 Adam Bergkvist 2013-09-04 05:14:23 UTC
(In reply to comment #6)
> [...]
> This approach is not specific to congestion control. It can be applied to
> all other constraints as well.

The general mechanism you describe here is pretty much how our current constraint approach work (as far as I understand it).

A "fence conditions" is represented by a MinMaxConstraint object [1]. The script will be notified when the browser cannot stay within the fence via the "overconstrained" event [2]. 

[1] http://dev.w3.org/2011/webrtc/editor/getusermedia.html#idl-def-MinMaxConstraint

[2] http://dev.w3.org/2011/webrtc/editor/getusermedia.html#event-mediastreamtrack-overconstrained
Comment 8 Kiran 2013-09-04 06:28:56 UTC
But timer for these constraints were not specified anywhere in the spec.
We had a discussion some time back on the same regarding the importance of timers [1], and folks agreed on it too [2].
And this type of timers may be used not only for the bit-rate or bandwidth fencing but also for other cases where a similar kind of problem may arise.

[1] http://lists.w3.org/Archives/Public/public-webrtc/2013Jul/0593.html

[2] http://lists.w3.org/Archives/Public/public-webrtc/2013Jul/0600.html

Thanks,
Kiran.


(In reply to comment #7)
> (In reply to comment #6)
> > [...]
> > This approach is not specific to congestion control. It can be applied to
> > all other constraints as well.
> 
> The general mechanism you describe here is pretty much how our current
> constraint approach work (as far as I understand it).
> 
> A "fence conditions" is represented by a MinMaxConstraint object [1]. The
> script will be notified when the browser cannot stay within the fence via
> the "overconstrained" event [2]. 
> 
> [1]
> http://dev.w3.org/2011/webrtc/editor/getusermedia.html#idl-def-
> MinMaxConstraint
> 
> [2]
> http://dev.w3.org/2011/webrtc/editor/getusermedia.html#event-
> mediastreamtrack-overconstrained
Comment 9 Gili 2013-09-04 13:03:29 UTC
(In reply to comment #7)
> The general mechanism you describe here is pretty much how our current
> constraint approach work (as far as I understand it).

One difference I note is the existence of mandatory constraints in the spec. In my proposal, all constraints are optional and the application decides what to do when a constraint cannot be honored.

Another thing I wanted to clarify, when given conflicting constraints (resolution, frame-rate, bitrate) does the browser get to decide which gets priority over the other or does it fail-fast and notify the callback immediately that the constraints cannot be honored as-is? In my proposal, the latter occurs.

I'm trying to push as much of the decision-making out of the browser and into the application's hand.
Comment 10 Adam Bergkvist 2013-09-05 04:54:49 UTC
(In reply to comment #8)
> But timer for these constraints were not specified anywhere in the spec.
> We had a discussion some time back on the same regarding the importance of
> timers [1], and folks agreed on it too [2].

Do we have a concrete proposal on this?

A few of our options (from the top of my head):
* Define a browser default timeout.
* Add a timeout property to MinMaxConstraint (if we want to keep the type without a timer as well we could derive a new type TimedMinMaxConstraint)
Comment 11 Adam Bergkvist 2013-09-05 05:01:03 UTC
(In reply to comment #9)
> Another thing I wanted to clarify, when given conflicting constraints
> (resolution, frame-rate, bitrate) does the browser get to decide which gets
> priority over the other or does it fail-fast and notify the callback
> immediately that the constraints cannot be honored as-is? In my proposal,
> the latter occurs.

It depends. Mandatory constraints are unordered, so when one fails the script gets notified. Optional constrains are ordered, so the script lists them in the order of importance.
Comment 12 Adam Bergkvist 2013-09-05 05:03:25 UTC
Btw, we should probably spawn separate email threads for these more specific constraints discussions.
Comment 13 Kiran 2013-09-05 05:43:05 UTC
1. I have raised this proposal, as an addition to callbacks raised by Gili, to avoid congestion of callbacks. If required (based on the suggestions) I am ready to propose it in a new thread for discussing it in webrtc-mailing lists.

2. IMHO, there should be two default values, one for minimum and the other for maximum. (To avoid the abnormal/unusual values, for both minimum bandwidth/bitrate and maximum bandwidth/bitrate).

Analysing this further more, timers should behave like this.

#. First it should wait for the specified time before firing the first call back.

#. If browser fired a callback and still, if there is no action/response from the application, it should continue to send callbacks after that specified time.

#. Either if the state becomes proper before firing the callback (for some specific time), or if the browser receives the action response from the application, then there should not be any callback.

Regarding "Add timeout property" or "TimedMinMaxConstraint" -- can you please explain this more in brief.

(In reply to comment #10)
> (In reply to comment #8)
> > But timer for these constraints were not specified anywhere in the spec.
> > We had a discussion some time back on the same regarding the importance of
> > timers [1], and folks agreed on it too [2].
> 
> Do we have a concrete proposal on this?
> 
> A few of our options (from the top of my head):
> * Define a browser default timeout.
> * Add a timeout property to MinMaxConstraint (if we want to keep the type
> without a timer as well we could derive a new type TimedMinMaxConstraint)