Disposition of comments for the Media Annotations Working Group

API:

I find this spec a little confusing, though it's helped by the concrete
examples of use. You are right to identify local/internal and
remote/external sources for metadata, but the API doesn't feel very
Javascripty to me.

I think the approach of so many different specialized typed interfaces
may be too cumbersome for both implementers and authors. I would take a
more combinatorial approach, where the author uses the API to extract
sets of metadata via certain selection criteria, then further filter
those with selection criteria, or iterate through them generically and
programmatically.

It seems that you are considering the case of multiple instances of
orthogonal or even conflicting metadata, possibly in different formats,
being extracted from the same media resource (thus the returnset being
an array), but I couldn't see that explicitly described anywhere.

I realize that these are pretty high-level comments, but I'm happy to
join a telcon sometime to discuss them further, or expand on them via
email. Unless you have specific implementer feedback about these, I
think it might be fruitful to reexamine this approach before the CR phase.


[1] http://www.w3.org/TR/2010/WD-mediaont-10-20100309/
[2] http://www.w3.org/TR/2010/WD-mediaont-api-1.0-20100309/

Regards-
-Doug Schepers
W3C Team Contact, SVG and WebApps WGs

API for Media Resource questions:

Why is http://www.w3.org/TR/2010/WD-mediaont-api-1.0-20100608/#samplingrate-interface
the only audio parameter in that document?

Are there plans to include, for example, a quality parameter for
audio/x-speex?

Why does that section say "No exceptions"


> (8) Report of any Formal Objections....

I would like to formally object to the omission of an audio/x-speex
quality parameter from the API for Media Resource draft, and omission
of speex and vp8 from the Ontology for Media Resource draft.

Sincerely,
James Salsman
invited expert
Device API and Policy (DAP) Working Group

The RFC has the following specification for 501: "The server does not support the functionality required to fulfil the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource."

However the recent specification for API for Media Resources 1.0 ( http://www.w3.org/TR/2011/WD-mediaont-api-1.0-20110712/#api-status-codes ) uses it in the case when "only a subset of GET methods for properties implemented"..

What is the modern accepted usage of this response code? Will the use specified in Media Resources spec confuse any clients if they return a 501 for some GET requests but not others?

Hi,

I just noticed something, shouldn't it be API for Media Resource*s*? Same comment for the Ontology document.

Here are my notes on [0]:

ed.: "Thereby, the Media Ontology Core Properties will be used as a pivot vocabulary in the API." I don't think that the "thereby" is needed here. (same thing further down)

ed.: "the metadata formats in scope (1.2 Formats in scope) are stored in the Media Ontology in order to provide cross-community data integration" Instead of "are stored in the Media Ontology" I think you mean "are documented in the Ontology for Media Resources specification", no?

ed.: "The JavaScript examples in this document will only work if the API is implemented by the browser." That's probably not true, you could have an implementation of the API running in Javascript outside the browser. You might wish to say instead something like "The examples provided in this specification use Javascript, but they ought to be simple to translate to another language if there binding is implemented there."

ed.: "The API exists of a number of interfaces" -> The API has a number of interfaces

ed.: "Implementations of the API should provide objects implementing the different interfaces." That's not clear, I would expect that implementations MUST support these interfaces. How those are made available as objects ought to be clear from the WebIDL.

substantial: "Example on how to introduce this in HTML5 by making the HTMLMediaElement inherit from the MediaResource interface:" This shouldn't be an example. If a user agent is expected to expose this interface on media elements, then this has to be a MUST, possibly using an "implements" statement.

ed.: in getDiagnosis, "See Section 4" links to section 5.

substantial: Parameters that are optional in WebIDL need to be marked as "optional" instead of using the OPTIONAL 2119 keyword in the prose.

substantial: The general getProperty/getDiagnosis architecture seems skewed. I wonder if the API does not do more than what is actually required. I presume that when a getProperty request is made, all the metadata has already been loaded in memory and is available for immediate retrieval.

If that is not the case, the API *really* needs to be made asynchronous, otherwise it won't be usable in a browser context at all (or in any kind of performance-conscious environment, such as typically a web server).

If it is the case, then instead of specifying a complex filter in getProperty's parameters, it would be a lot simpler to just expose the property and leave it up to code to do the filtering. So instead of

props = mr.getProperty("title");
props = mr.getProperty("title", "#subchapter");
props = mr.getProperty("title", null, "tva");
props = mr.getProperty("title", null, null, null, "fr");

you would just have

props = mr.title;

and leave it up to code to perform any kind of filtering on its own terms. This can then be abstracted into a library. It seems clearer, and much more idiomatic.

substantial: getDiagnosis doesn't seem thread-safe, and I'm guessing that even in a JS context it could easily hit race conditions. Is conveying the exact reason why something was null really a strong use case in JS? If so, wouldn't an exception work? If not (which is what I suspect) this could be dropped. I doubt anyone would use it — in fact your examples never do!

substantial: I suggest exposing the properties directly on the object and removing getPropertyNamesWithValues.

ed.: "MAObject" is not a very descriptive name, 75% of it is dedicated to saying that it's an object, which one already knows from the fact that it has been retrieved as such. "MediaAnnotation" would be clearer.

substantial: Would it not be simpler to replace getSourceFormatsWithValues with a sourceFormats array that could be filtered in code if one so desires? Is it even an important use case in the browser to list which metadata formats are available in general?

substantial: I'm sorry but I don't understand what selectMAResource is supposed to do. Does it allow one to change the underlying resource which the current object is describing and load another one? If so, it should be asynchronous (and it should also be on another interface). What does the boolean which it returns mean? Or maybe it's loading additional external metadata? Again, if so, it really needs to be asynchronous (and probably somewhere else too).

ed.: It's probably worth indicating that the Language interface is meant to be inherited from, that it doesn't do much on its own.

substantial: It is not clear "plain text" is for unstructuredValue. Is this the raw binary for binary formats? A piece of XML for XML formats?

substantial: What is the syntax of sourceFormat? Of mappingType? Of fragmentIdentifier?

substantial: Why is there a value field on Identifier if it's holding a URI that could be in the uri field it inherits from MAObject? Same for Locator, Creator.

substantial: The "usage as a service" segments aren't precise enough and normative enough to describe a protocol properly. I think that this document should limit itself to describing an API using WebIDL, and then either help define a REST binding for WebIDL or another document should be created for the protocol.

substantial: It is quite counter-intuitive to have:

var lang = foo.getProperty("language").value;

when one could equally have:

var lang = foo.language;

The same applies to other properties, such as description.

substantial: "For the latter a number of suggested terms are defined". If there is to be a controlled vocabulary, it needs to be in the ontology, not in the API. And "suggested" isn't really good, it's a MUST, a SHOULD, etc.

substantial: The section CreateDate has just "Date" in the interface name. It should probably be "CreationDate" in both.

substantial: Why is it "createdate" in getProperty and createDate in the service URI? Are all those property names the local names from the ontology? I can't see that defined anywhere.

substantial: Your Location interface seems to overlap with equivalent interfaces defined by the Geolocation WG. I strongly recommend that you coordinate with them to return a similar interface.

substantial: (In Relation) "For the latter a number of suggested terms are defined:" but no list follows. And the same comments apply as above for "suggested" and controlled vocabularies.

substantial: In general it would be a lot easier to read if interface attributes linked to the definition of the property in the ontology — especially if it is supposed to have a specific syntax. I didn't review each interface in great detail because going back and forth between the two documents to check that they were properly related was too tedious.

ed.: For FrameSize, it says that the default unit is pixels, yet in the example float values are returned for unitless objects. Is that intentional?

substantial: It's unclear what to return for Compression. Is JPEG a compression? Something more specific? Is it case sensitive? Partially controlled?

ed.: Why is it Samplingrate instead of SamplingRate?

substantial: The relationship of section 5 to HTTP is unclear. Is this needed?

substantial: Section 7 should be beefed up. Notably, if I can access geographic information it could be used to access someone's location without them agreeing to it. Likewise for name and a number of other things. In general, you probably need to think about privacy.

note: integration with DAP's Media Capture is likely desirable.


[0]http://www.w3.org/2008/WebVideo/Annotations/drafts/API10/LC/Overview.html

Sujet: Re: [Reminder]: Last Call Working Drafts transition announcement
of the API and Ontology for Media Resource 1.0
Date : Sun, 18 Jul 2010 11:37:00 +0100
De : Jo Rabin <jo@linguafranca.org>
Pour : tmichel@w3.org

Catching up on email on a Sunday morning - BPWG won't be submitting
comments but I think I spotted a typo when I was (out of curiosity)
looking at the doc:

3.1.1

getOriginalData

This operation allows to retrieve the original metadata according to the
specified source format.
Parameter Type Nullable Optional Description
sourceFormat DOMString ✘ ✘ This argument identifies a specific metadata
format. If a metadata format is defined, only the metadata available in
the corresponding metadata format are retrieved. This parameter is optional.

(there is a disconnect between the ✘ under optional and the statement
that the parameter is optional)

Cheers
Jo

This is not an endorsement, merely editorial comments in response to a
call for comments.

http://www.w3.org/TR/2010/WD-mediaont-api-1.0-20100608/

> This specification defines a client-side API to access metadata
> information related to media resources on the Web. The overall
> purpose of the API is to provide developers with a convenient

I'd write "this API" instead of "the API" here and in the rest of this document.

> access to metadata information stored in different metadata formats.
> Thereby, the Media Ontology Core Properties will be used as a pivot

"Thereby" is an odd word. And given that you haven't introduced <MOCP>
until here, the entire sentence seems problematic. Typically I'd
expect someone to introduce briefly (one or two sentences) what
something (MOCP) is before saying that it will be used.

> vocabulary in the API. The description of relations between these
> core properties and the metadata formats in scope (1.2 Formats
> in scope) are stored in the Media Ontology in order to provide

Does "Media Ontology" here refer to the same thing as MOCP earlier?
if it does, then you should indicate that you intend to use that short
form when you introduced it, if not, then you just used an undefined
term.

> cross-community data integration. The API is described using the
> interface definition language [WEBIDL]. The decision to use Web IDL,
> which offers bindings for ECMAScript and Java, can be based on the
> Use Cases and Requirements for Ontology and API for Media Resource
> 1.0 [MEDIA-ANNOT-REQS]. This document clearly states that the focus
> for this API lies on multimedia services on the Web.

this should be "lies with" or "relies on"

> The API serves as a mediator between a developer and the underlying
> Ontology for Media Resource 1.0 [MEDIA-ANNOT-ONTOLOGY] with

drop "underlying", ontologies do not live below developers.

> the goal to support interoperability between metadata formats. It offers

change "to support" to "of supporting"

> operations to retrieve particular metadata informations represented in
> a certain metadata format related to media resources on the Web.

> This document is being published with the aspiration to gather wide

You don't want "aspiration", you could replace "published..to" with
"published in order to"

> feedback on the yet available API design.

This doesn't make sense.

> 1.1 Formats in scope
> Refers to the formats in scope of Ontology for Media Resource 1.0.

> 1.2 Formats out of scope
> Refers to the Formats out of scope of Ontology for Media Resource 1.0.

> 1.3 Terminology
> In this document the terms "Media Resource", "Property", "Mapping"
> and "Property value type" are to be interpreted as defined in Section
> 2 of Ontology for Media Resource 1.0.

> 2. Design consideration
> We consider two scenarios where the API could be implemented:
> either in a user agent (scenario 1) or as a web service (scenario 2).
> The two scenarios are shown in the figure.

There's no figure here. Please name it (the figure is 2 paragraphs away).

> Scenario 1: User agent
> The API is implemented in a user agent (e.g., browser or browser plugin)
> and exposed as a JavaScript API (using the WebIDL JavaScript binding).
> The user agent includes the components for metadata access (possibly

drop the "the" after "includes" and add an "and" before "possibly"

> extraction) and mappings for a supported set of formats. The metadata
> sources (the media resource and/or metadata document(s)) must be
> retrievable and access (establish connection, retrieval) of the metadata

change "of" to "to".

> sources is handled by the user agent.

Security considerations?

> Scenario 2: Web service
> The API is implemented as a Web service. Such an implementation would
> be typically used by a non-UI client, such as an agent harvesting metadata.
> However, the API could be also accessed from a user agent, and used the
> same way as described in scenario 1 by the help of a JavaScript library for

change "by" to "with"

> accessing the web service.


> At the back-end of the web service, this scenario

you don't usually write 'back-end of the web service'.
and i'm not sure what this means.

> also allows supporting a media repository (e.g. content provider's archive
> database, movie store) from which the user agent could directly retrieve
> metadata sources and which might have a custom metadata format not
> supported by a user agent.

This doesn't make sense and is incredibly long for a single "sentence".
How could a user agent *directly* retrieve formats it doesn't support?

> In contrast to an integrated component (see scenario 1), an implementation
> of the API in a web service could do more complex mappings on the fly as

replace "as" with "than"

> a component integrated in a user agent, and can be more flexible
> (e.g., supporting additional formats).

> Overview of different API options.

This alt text is woefully insufficient. It doesn't indicate that it's
a diagram. "JavaScript" is one word, not two.

> In both scenarios, the access to the metadata properties needs the following stack of components:

drop the first "the"

> * An implementation of the API for Media Resource (as defined in this
> document), which providers the actual getter methods for the properties.
> * An implementation of the mappings from a specific source format to the
> properties of the media ontology (as defined in Ontology for Media Resource 1.0).

Why isn't "Media Ontology" written as a proper noun here?

> * A format specific API to access the metadata. This can be an API for
> accessing a metadata document describing a media resource (e.g. an XML
> parser and a set of XPath statements) or an extractor to read metadata
> embedded in the media resource (e.g. a library to read EXIF information from
> JPEG images).

> In order to define the context on which the API for Media Resource is working,

change "on" to "with" and "working" to "applied"

> it is assumed that there is at least a unidirectional reference from the media

why are you assuming things? -- this is an editorial complaint, the
statement needs to be active "one needs at least ..."

> resource to the metadata document or vice versa. If this is not the case such a
> reference needs to be provided by the web application (scenario 1), web service
> (scenario 2) or media repository (scenario 2).

> 3. API Description
> The JavaScript examples in this document will only work if the API is implemented
> by the browser.

> The API exists of a number of interfaces, described using Web IDL.

change "exists" to "consists"

> Implementations of the API should provide objects implementing the different
> interfaces. The entire description can be found in Appendix A. The API is contained
> within the MediaResource interface within the mawg module.
> Objects implementing this interface provide the necessary methods to access
> metadata properties of a Media Resource. The object holds methods to identify
> the actual Media Resource and the metadata sources. All properties can be
> accessed through a specific operation getProperty.

... specific operation: getProperty.

> When an attempt to read a property fails, diagnostics information can be obtained
> using a diagnosis operation. Subtypes in the API are relevant for those properties
> mentioned in 4.1.3 Core properties of Ontology for Media Resource 1.0.

> Lastly, methods are available that allow to iterate through the available metadata.

change "to iterate" to "iteration"

> Next, the different interfaces and exposed methods are discussed.
> Finally, examples of the usage of the API can be found in section 4.

You can't use Lastly, Next and Finally as a set. -- And I'd suggest
that you avoid using "Lastly" entirely.

> 3.1 MediaResource interface
> The MediaResource interface offers a number of operations that allow
> accessing the metadata of a Media Resource.

... provides a number of operations to access the metadata ...

> Example on how to introduce this in HTML5 by making the HTMLMediaElement

change "on" to "of"

> inherit from the MediaResource interface:

> interface HTMLMediaElement : MediaResource, HTMLElement {...}

It seems like you're invading the HTML WG's namespace....

> As shown, the getElementsByTagName returns a HTMLMediaElement
> which inherits from the MediaResource interface. The getProperty
> method is part of this interface so the returned element has an
> implementation of this method. By calling the getProperty method
> with the argument "title" we can retrieve the title of the corresponding
> media resource.

I don't think that getProperty is a good name for something that might
go into HTML DOM.

> interface MediaResource {
> boolean selectMAResource (in DOMString mediaResource, in optional MetadataSource[] metadataSources);
> MAObject[] getProperty (in DOMString propertyName, in optional DOMString fragment, in optional DOMString sourceFormat, in optional DOMString subtype, in optional DOMString language);

getProperty is singular, it shouldn't be returning an array. Perhaps
getMAProperties()?

> DOMString[] getPropertyNamesWithValues (in optional DOMString sourceFormat,
> in optional DOMString language, in optional DOMString fragment);

This is in plural, but again, it's icky.

> DOMString[] getSourceFormatsWithValues (in optional DOMString language);

This is in plural, but again, it's icky.

> DOMString getDiagnosis ();

This is awful. Typically such things should be attributes. The
attribute name should be meaningful, this is not.

> DOMString[] getOriginalData (in DOMString sourceFormat);

Data is a plural, but this should probably be getMAOriginalData or
getOriginalMAData ...

> getDiagnosis
> This operation allows to retrieve the status code(e.g., the getProperty operation returning a null value). See Section 4 for details.

I believe you should be using "function" or "method" instead of
"operation" (global comment).
Change "to retrieve" to "retrieval of" (globally).
You're missing a space between "code" and "(". And splitting a return
value out of a function for the null case is not very DOMish,
typically we'd use an Exception with the information in the exception
object. Why aren't you?

> getProperty
> This operation allows to retrieve the value of a certain property. The specific
> property is passed as an argument and a list of objects is returned that hold the
> values according to the requested property. These objects implement the
> MAObject interface, described in (MAObject interface). Depending on the
> requested property, the returned objects implement a different sub-interface
> (inheriting from the MAObject interface).

> Requesting for the title gives back an array of MAObjects implementing the

This seems to be the first introduction of "title" (at least locally),
and it isn't marked up. It doesn't make sense. Perhaps:

Requesting "title" returns an array of MAObjects implementing the
[pre]Title[/] interface...

> Title interface,

> requesting the creator results in MAObjects implementing the Creator interface
> and so on. These interfaces are described in section 3.5 to 3.12.

> propertyName DOMString ✘ ✘
> This argument identifies the property for which the values need to be retrieved.
> Optional arguments allow to refine the request

replace "to refine" with "refining" and add a period to this "sentence"

> language DOMString ✘ ✔
> This argument allows to identify the language of the metadata.
> Only if the metadata is available in the specified language, the values are returned.

Values for the metadata will only be returned if it is available in
the specified language
> 3.5.1.1 Attributes
> type of type DOMString
> This attribute holds the type of the identifier, which can be filtered on in the
> getProperty operation. Possible values are " UMID" and "ISAN".

You have a space inside the quotes around " UMID" which seems odd.

> 3.5.2 Title interface
> Title interface is used as the specific return type of MediaResource.getProperty
> method which has "Title" as a value of propertyName parameter.

the earlier request seemed to be for 'title', not 'Title' ...
> 3.9.2.1 Attributes
> link of type DOMString
> This attribute holds a link to the license if it is externally available.

Why is this a string instead of a url type?

> organization of type DOMString
> This attribute identifies the organization that issued the license.

> type of type DOMString
> This attribute holds the actual type of the policy, which can be filtered on in the getProperty operation. Examples are "license", "access", and "privacy".

> value of type DOMString
> This attribute holds the description of the license.

> policy[0].value = "Attribution 2.5"
> policy[0].organization = "http://creativecommons.org/licenses/by/2.5"
> policy[0].type = "license"

Shouldn't organization be creative commons and the "link" property be
to the license?

> This section introduces a set of status codes for the defined API to indicate
> the system behaviour. It uses a subset of the HTML/1.1 [HTTP11] status

behaviour is the first word I can see which seems to be in en-GB
instead of en-US, I'd request that you consider whether your document
is in fact written in en-US and thus should use "behavior".

> codes for general informations (e.g., bad request), but also system specific

information

> ones (e.g., property not defined in source format). The specific status codes
> have been arranged in the HTML/1.1 [HTTP11] status codes categories.
> These codes can be retrieved by calling the getDiagnosis method.

> The set of status codes my be enlarged or changed in later versions of

Enlarged is odd.

Later versions of this document may include additional status codes or
other changes.

> this document. The numerical code range for the system specific status
> codes start with x62 (x = number of HTML/1.1 [HTTP11] status codes
> category and 62 = MA).

> Numerical Code Textual Description Example
> 200 Ok property delivered correctly

should be "OK"

> interface MediaResource {
> //Operation for retrieval of the reason of an error

this is really unhelpful.

> DOMString getDiagnosis();

> interface Rating: MAObject, Language {
> attribute DOMString issuer;
> attribute short value;
...
> attribute DOMString type;

random indentation

> interface Policy: MAObject, Language {
...
> attribute DOMString link;

random trailing whitespace

...
> interface FrameSize: MAObject {
...
> attribute DOMString unit;

random trailing whitespace

> Members of the Working Group are (at the time of writing, and by alphabetical order):

What about former members? :)

> Members of the Working Group are (at the time of writing, and by alphabetical order):

you already said that.