This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 25298 - How to represent VideoFacingMode if browser is not able to detect the FacingMode of the camera.
Summary: How to represent VideoFacingMode if browser is not able to detect the FacingM...
Status: RESOLVED FIXED
Alias: None
Product: WebRTC Working Group
Classification: Unclassified
Component: Media Capture and Streams (show other bugs)
Version: unspecified
Hardware: All All
: P2 normal
Target Milestone: ---
Assignee: Cullen Jennings
QA Contact:
URL:
Whiteboard:
Keywords:
: 25793 (view as bug list)
Depends on:
Blocks: 25794 25810 26111
  Show dependency treegraph
 
Reported: 2014-04-09 17:02 UTC by Kiran
Modified: 2014-10-02 13:31 UTC (History)
7 users (show)

See Also:


Attachments

Description Kiran 2014-04-09 17:02:56 UTC
In certain scenarios, for example if an external webcam is connected through USB cable or bluetooth, it may not be possible for the browser to identify the exact facing direction of the camera.

In such scenarios,(if browser is not able to identify the facing direction of the camera), there is no attribute in the VideoFacindMode enum to represent/initialize.

I prefer to add new "unknown" attribute to videoFacingMode enum, so that if browser is not able to detect the videoFacingMode, then it must be set to "unknown".

The new enum might be like this

enum VideoFacingModeEnum {
    "user",
    "environment",
    "left",
    "right",
    "unknown"
};
Comment 1 Martin Thomson 2014-04-09 17:29:44 UTC
Sounds reasonable.  Presumably, this would not be a useful input to gUM as a constraint.  Do you believe that this needs to be addressed somehow?  Could we reasonably forbid its use as a constraint?
Comment 2 Paul Neave 2014-04-09 17:32:13 UTC
Agree, but perhaps rather than "unknown" this case should simply be JS's undefined property?
Comment 3 Kiran 2014-04-09 17:48:49 UTC
I suggested this by considering the following scenario.

If MediaStreamTrack.states() is invoked from the JS, it will return "MediaSourceStates", which internally contains "VideoFacingMode".
This attribute should be filled inside the browser. 

If browser don't know the exact direction facing of the camera, then spec is not providing any option to fill in. Then browser has to fill in either assumed or wrong default value or fill it with a NULL-String ("").

AFAIK Instead of returning NULL-String to JS, it is good to fill some predefined value like "unknown".
Comment 4 Martin Thomson 2014-04-09 17:58:47 UTC
I actually like Paul's suggestion.  If you don't know the value, don't provide one.
Comment 5 Kiran 2014-04-09 18:07:51 UTC
I feel, if the value is not known, then browser should convey the same to JS. Other wise spec should explicitly specify that browser should return a NULL string in case of unknown facing mode.
Comment 6 Jan-Ivar Bruaroey [:jib] 2014-04-09 18:13:14 UTC
(In reply to Kiran from comment #5)
> I feel, if the value is not known, then browser should convey the same to
> JS. Other wise spec should explicitly specify that browser should return a
> NULL string in case of unknown facing mode.

All members in dictionaries are optional - and constraints/settings/capabilities work like dictionaries in most respects - so I believe the browser can omit the facingMode member entirely rather than fill it with an empty string (which would be an illegal enum member currently).

Similarly, I would expect getCapabilities() to only contain facingMode when it is known.

However we achieve it, I agree this needs clarification.

I would also like clarification on whether the following excludes a "camera of unknown direction":

{video:{mandatory:{facingMode:['user','environment','left','right']}}}

I assume yes.
Comment 7 Martin Thomson 2014-04-09 18:17:34 UTC
(In reply to Jan-Ivar Bruaroey [:jib] from comment #6)
> However we achieve it, I agree this needs clarification.

Sure.
 
> I would also like clarification on whether the following excludes a "camera
> of unknown direction":
> 
> {video:{mandatory:{facingMode:['user','environment','left','right']}}}
> 
> I assume yes.

Likewise.  Perhaps the reason that the camera is not given a facing mode from the selected set is that it *is* known, but not one of the four provided options.  Unset wouldn't be strictly "I don't know" it would also encompass "none of the above".
Comment 8 Gili 2014-04-09 18:20:42 UTC
+1 for omitting unknown properties because there is no way to guarantee that the browser can even see all properties to begin with. Meaning, some properties are unknown because we can see them but don't understand their meaning. Other properties are not visible because a browser may query "basic" properties but omit querying for more "advanced" properties. Under OSX, for example, the functionalities exposed by QtKit and AVFoundation driver APIs are not identical so depending on which interface you use to query a device you will see different capabilities.
Comment 9 Jan-Ivar Bruaroey [:jib] 2014-04-09 18:21:34 UTC
Just to take both positions here, one argument FOR "unknown" would be to future-proof so we can add new enum values in the future. This would rely on Bug 19936 being addressed, which would let us write:

enum VideoFacingModeEnum {
    "user",
    "environment",
    "left",
    "right",
    "unknown" = default
};
Comment 10 Kiran 2014-04-09 18:24:28 UTC
(In reply to Jan-Ivar Bruaroey [:jib] from comment #9)
> Just to take both positions here, one argument FOR "unknown" would be to
> future-proof so we can add new enum values in the future. This would rely on
> Bug 19936 being addressed, which would let us write:
> 
> enum VideoFacingModeEnum {
>     "user",
>     "environment",
>     "left",
>     "right",
>     "unknown" = default
> };

This is what I expected it to be.
Comment 11 Adam Bergkvist 2014-04-16 06:59:19 UTC
R
Comment 12 Adam Bergkvist 2014-04-16 07:10:09 UTC
(Above: New personal record for shortest message before accidentally hitting the submit button :) )

Do we have to rely on the resolution of bug Bug 19936 to future-proof the enum? I noted that, for ECMAScript, assignments to enum attributes are ignored if the new value isn't a valid enum value, but an invalid enum value as a function argument generates an error. I would expect an assignment to a dictionary member to be similar to the attribute assignment case. Not sure though.
Comment 13 Jan-Ivar Bruaroey [:jib] 2014-04-16 15:20:47 UTC
(In reply to Adam Bergkvist from comment #12)
> Do we have to rely on the resolution of bug Bug 19936 to future-proof the
> enum?

I don't think so, I marked it as a dependency more to revive the other bug.

> I noted that, for ECMAScript, assignments to enum attributes are
> ignored if the new value isn't a valid enum value, but an invalid enum value
> as a function argument generates an error.

Yes, from http://www.w3.org/TR/WebIDL/#idl-enums

> I would expect an assignment to a dictionary member to be similar to the
> attribute assignment case. Not sure though.

Unfortunately, dictionaries are always function arguments (they can't be attributes), so invalid facingModes throw, hence Bug 19936.

I suggest two actions here:

1. Add "unknown" as Kiran suggests in comment 0.

2. Use DOMString for facingMode and refer to VideoFacingModeEnum in a
   comment as a workaround for now, as I suggest in Bug 19936 comment 5.
Comment 14 Jan-Ivar Bruaroey [:jib] 2014-04-16 15:38:06 UTC
(In reply to Paul Neave from comment #2)
> Agree, but perhaps rather than "unknown" this case should simply be JS's
> undefined property?

"unknown" = not left, right, user or environment, but something else.
"undefined" = unconstrained.

These are different. I think it would be quite fun actually to constrain on "unknown". Give me all your new and wonderfully odd cameras!

Also, since we don't have negative constraints, this lets us express things like "not environment" using exclusion ['user', 'left', 'right', 'unknown'].

Perhaps a better name would be "other"?
Comment 15 Adam Bergkvist 2014-04-17 05:51:37 UTC
(In reply to Jan-Ivar Bruaroey [:jib] from comment #13)
> (In reply to Adam Bergkvist from comment #12)
> Unfortunately, dictionaries are always function arguments (they can't be
> attributes), so invalid facingModes throw, hence Bug 19936.

I was referring to the value assignment of a dictionary member (not the entire dictionary).

foo.attr = "not-in-the-enum"; // ignored (from WebIDL spec example)

enumArgFunc("not-in-the-enum"); // error  (from WebIDL spec example)

var dict = {
  member = "not-in-the-enum" // this is the case: ignored or error?
};
Comment 16 Jan-Ivar Bruaroey [:jib] 2014-04-17 12:31:08 UTC
(In reply to Adam Bergkvist from comment #15)
> I was referring to the value assignment of a dictionary member (not the
> entire dictionary).
> 
> foo.attr = "not-in-the-enum"; // ignored (from WebIDL spec example)

As I understand it, WebIDL dictionaries returned to JS are not browser objects but POS (plain old JavaScript objects), so foo here is a POS and I wouldn't derive much from that.

> var dict = {
>   member = "not-in-the-enum" // this is the case: ignored or error?
> };

I think you mean: var dict = { member: "not-in-the-enum" ); and dict is also a POS.
Comment 17 Adam Bergkvist 2014-04-22 05:09:50 UTC
(In reply to Jan-Ivar Bruaroey [:jib] from comment #16)
> (In reply to Adam Bergkvist from comment #15)
> > var dict = {
> >   member = "not-in-the-enum" // this is the case: ignored or error?
> > };
> 
> I think you mean: var dict = { member: "not-in-the-enum" ); and dict is also
> a POS.

You're right (about the syntax too)
Comment 18 Kiran 2014-04-24 05:58:26 UTC
(In reply to Jan-Ivar Bruaroey [:jib] from comment #14)
> (In reply to Paul Neave from comment #2)
> > Agree, but perhaps rather than "unknown" this case should simply be JS's
> > undefined property?
> 
> "unknown" = not left, right, user or environment, but something else.
> "undefined" = unconstrained.
> 
> These are different. I think it would be quite fun actually to constrain on
> "unknown". Give me all your new and wonderfully odd cameras!
> 
> Also, since we don't have negative constraints, this lets us express things
> like "not environment" using exclusion ['user', 'left', 'right', 'unknown'].
> 
> Perhaps a better name would be "other"?

I don't have any problem with name, whether it is "unknown" or "other". I concentrated much on the feature.

The intention of proposing "unknown" is to cover the case where

1. Direction of facing is UNKNOWN in cases where we connect an external camera using USB / bluetooth (ofcourse user can change the direction in the middle of the session).
2. Rotating cameras apart from fixed top, down cameras.
Comment 19 Harald Alvestrand 2014-05-26 10:04:42 UTC
Discussed at the May 19 Media Capture TF meeting.

Conclusions:
- "Other/Unknown" will be represented as a missing value - that is, JS "undefined"
- Due to the issues with Enum in a dictionary, DOMString will be used
- Jan-Ivar took on the task of creating a text proposal
Comment 20 Jan-Ivar Bruaroey [:jib] 2014-06-25 14:52:39 UTC
Pull request: https://github.com/fluffy/webrtc-w3c/pull/34
Comment 21 Jan-Ivar Bruaroey [:jib] 2014-07-01 16:07:10 UTC
Note that pull request in comment 20 adds an "other" enum entry for UAs to coerce unknown strings into. This is useful for implementations, as it lets them translate incoming strings into an enum internally, as a point of parsing. As a consequence, JS users may see this value in constraints returned to them from APIs that return constraints, in place of the original string, which is not kept.
Comment 22 Harald Alvestrand 2014-07-01 18:36:14 UTC
*** Bug 25793 has been marked as a duplicate of this bug. ***
Comment 23 Stefan Hakansson LK 2014-09-19 07:22:19 UTC
Re-assigned. To be reviewed now that PRs #14 and #16 have been merged.
Comment 24 Cullen Jennings 2014-10-02 13:31:14 UTC
I think this was resolved with using JS undefined