metadataInURI-31: Guide to Feedback and Outstanding Issues

Author: Noah Mendelsohn

Date: 28 March 2006

This is a page I prepared to help myself come up to speed on TAG issue metadataInURI-31. I've posted it in hopes it will be helpful to others, but this is not an official work product of the TAG. I'll be glad to consider suggestions for updates, but there's no guarantee that I'll be responsive to requests for changes.

Anyway, I'm trying to do three things here: (1) make it easy to find what I take to be the most important feedback that's been gathered in the years this issue's been alive; (2) summarize some options for how to move ahead; and (3) give some of my own opinions on what the finding should say.

Where to Find the Finding

The main URI for drafts on metadataInURI-31 is: http://www.w3.org/2001/tag/doc/metaDataInURI-31.html; the latest copy in date space as of today is the draft of 8 July 2003.

Earlier feedback and discussion

Here's a brief guide to discussion that's already been had in TAG meetings and in email.

Discussion of metadataURI-31 in TAG meetings

The issues list entry includes links to TAG minutes in which discussion occurred. As you'll see from the summary below, only a few contained substantive technical discussion. The table below should make those easier to find (I don't guarantee summaries are complete and unbiased—read the originals and draw your own conclusions):

Meeting Date	Summary of Discussion
2 Dec. 2002	References message from Ossi Nykänen. Ossi proposes that metadata be in a separate resource, but that a naming pattern in URIs would allow one to guess the name of the metadata resource for any given starting URI. E.g.: For a resource referred to by http://domain/path/filename the RDF description (declared by the resource) will be found at http://domain/path/filename/meta.rdf So, at the next level, there is metadata in the URI, because you infer a relationship between the two resources. Dave Orchard expresses an interest in versioning resources. Tim Bray says "Notion of encoding metadata in a URI is broken." Tim Berners-Lee encourages use of RDF :-). Resolved: Accept issue matadataInURI-NNN with note that TAG thinks the answer is "no" and will explain what to do instead.
6-7 Feb 2003 (Irvine)	Stuart Williams assigned to draft finding. No other action or discussion.
7 July 2003	This session appears to be commenting on a 4 July 2003 Member-only draft from Stuart. Dan: (hmm... no story atop 4 July 2003 draft of finding) Dan: I think the finding may be getting too long. I was hoping for three paras that say "Don't peek into URIs. People and software making use of URIs assigned outside of their own authority (i.e. observers) MUST NOT attempt to infer properties of the referenced resource except as licensed by relevant normative specifications or by URI assignment policies published by the relevant URI assignment authority."—Shorten 3 to "Don't peek inside the URI." It's always safe to not peek inside the URI. Dan: the W3C tech reports naming policy isn't based on the URI alone. there's no guarantee that http://www.w3.org/TR/2003/WD-snort-20040707 is a W3C working draft. If you know that it is a WD somehow, you can learn stuff from the URI. but you can never just peek into the URI alone. (FWIW: Noah observes several years later that this seems to be a key point. There's metadata you may be able to use in the URI if you have specific information from the resource owner, but that's very different from trying to follow your nose given some arbitrary URI and only well-known specs. with which to grok it.) NW: If I understand DC correctly, sounds like DC is saying that if I know XQuery is a Working Draft, then I can find meaning in WD in URI. If I saw a URI ending in "WD-".... could I infer that it's a WD? DC: No You have to look at the TR page as well. Once you have found something on the TR page, there's quite a bit you can go on. If you start from the TR page and find the URI, that's fine. But if you find the URI on the floor somewhere, you can't go on that alone. DO: Amazon publishes policy for query parameters. Action SW: Create new draft based on input today and send to www-tag. Action DO: Send rationale about why WSDL WG wants to peek inside the URI.
21-23 July 2003 (Vancouver)	These minutes appear to have the last detailed technical discussion of metadataInURI-31. They are lengthy and worth reading. Some highlights: SW: Is "don't peek inside" enough advice? TBray: I think happy medium between minimal approach (DC) and SW's current finding is: When you are processing a URI, there are normative specs (starting with [URI], then scheme specs); it's fine to peek inside URIs and interpret per normative specs. Beyond that it's inappropriate to make other assumptions by peeking in URI UNLESS you have private agreements (i.e., rules published by the URI authority). I think draft finding is too long, but I'm prepared to accept the assertion that SW's finding says what I mean. Do people agree that it's ok for the WSDL folks to say how to interpret their URIs? Later it was noted that some feedback in email had been in favor of keeping sections past sectin #1, somewhat contradicting the suggestion to make the finding shorter. DO: I think that we need to describe when it's ok to peek inside a URI. And when it's ok to write a spec that says it's ok to peek inside a URI. TBL: I have a criterion for this: suppose that someone has not come across the WSDL spec. Given a URI that was constructed according to the WSDL rules, can someone with such a URI (alone) determine that it's a WSDL URI? For example, do they get back a description (e.g., in a representation) of how to use the URI? Dan: I think I agree with Bray that replacing the finding by its 1st section would be forward progress. Interchange on fragids and content types: NW: You can't say squat about meaning of frag ids until you get the representation with the proper content type. TBL: I'm fine to talk about an imaginary document. But if you do retrieve it, you need to register a new content type or a frag id syntax. TimBL-YVR, you wanted to say that if you want a funny fragid syntax you need a new mime type DO: The WSDL WG wants to register a new content type. specs defining patterns for using structure in URI assignments to name abstract things, how do you establish a chain to establish authority to make such an assignment? 2396 delegates assignment authority to URI schemes which in turn delegate onwards ultimately to some specification or person or organisation. Such an authority could then explicitly state that they do in fact make assignments according to the specified pattern. Discussion of adding examples. Some sentiment for keeping only section 1, but adding examples. Suggestion that forms be one of the examples. Seeming agreement that it's good to discuss schemes in general (ftp, etc.), not just http. Discussion of pros and cons of fragids carrying metadata. Not sure I follow all the nuances just yet. Discussion of how creating specs that require metadata in URIs causes stress for server systems that have their own mappings from internal structures to URIs. There is repeated reference in the minutes to 19 June summary from DO; unfortunately, that link is a 404! Much more was discussed, but the above gives the flavor of the main areas considered.
6-8 October 2003 (Bristol)	metadataInURI-31: 8 July 2003 draft of "The use of Metadata in URIs" Action SW 2003/07/21: Produce a revision of this finding based on Vancouver ftf meeting discussion. Action DO 2003/07/07: Send rationale about why WSDL WG wants to peek inside the URI. See also TB email on Apple Music Store and use of URI schemes instead of headers See comments from Mark Nottingham and followup from Noah M.
12-14 May 2004 (Boston)	Stuart promises draft by mid-June. No technical discussion.
7 Feb 2005	Noah briefly mentions relationship to WS Addressing (I.e. if you don't use reference parameters, then you probably need to encode the equivalent in the URI.) Otherwise, no other technical discussion. Stuart agrees to pursue issue, with help from Noah.
21 Sept 2005 (Edinburgh)	Norm says we should keep working. Noah and Roy take ACTION to do it. No other technical discussion.
13 Dec 2005	No substantive technical discussion, except for a bit of review regarding the person who got in legal trouble for trying ../../.. in his browser.

Email discussion

Apparently, Stuart Williams went through a similar process of trying to consolidate input back in November of 2004. He wrote an email (member-only) which is basically an index to all the email he'd been through on the subject to that date. With Stuart's permission, I've included here a list that's heavily based on his input from that email:

Summary of comments on metadataInURI-31 draft finding:

Original question: "Should there be a standard way of embedding metadata
in URI"; Answer NO.
Norm: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0048.html

Embedding is ok... extraction may be more problematic:
Norm: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0048.html

Editorials:
Norm: http://lists.w3.org/Archives/Member/tag/2003Jul/0023.html

Technical comment on .ca domain example:
Mark Baker:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0041.html

Finding size:
  Says too much: 
     Tim Bray: http://lists.w3.org/Archives/Member/tag/2003Jul/0030.html
     Mark Nottingham:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0048.html
     Oliver Fehr:
http://lists.w3.org/Archives/Public/www-tag/2003Sep/0196.html
     TAG (someof):
http://www.w3.org/2003/07/21-tag-summary.html#metadataInURI-31

  Keep sections 2 and 3:
     Mark Baker:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0041.html
     Noah Mendelsohn:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0055.html
     TAG (someof):
http://www.w3.org/2003/07/21-tag-summary.html#metadataInURI-31

Bring some of the more important points forward:
Noah Mendelsohn:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0055.html
Mark Nottingham:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0056.html

Embed metadata, DO peek inside:
Michael Daconta:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0037.html
Michael Daconta:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0067.html

Don't Peek;
Patrick Stickler:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0039.html
Paul Prescod:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0043.html
Norm: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0065.html
Norm: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0068.html
Len Bullard:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0066.html
Patrick: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0071.html

Embed static characteristics only;
Patrick Stickler:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0039.html
Michael Daconta:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0045.html

There are better ways of getting resource metadata:
Patrick Stickler:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0039.html

REST "hypermedia as the engine of application state" of relevance:
Mark Baker:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0041.html
Mark Baker:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0099.html
Stuart (understands):
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0101.html

Some URI scheme specs do seem to try to constrain the sort of
resource referenced:
Stuart: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0091.html

Sort of resource unconstrained by URI scheme (indirect indentification):
Mark Baker:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0041.html

Just "Don't Peek" is too simplistic:
Stuart: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0040.html
Patrick: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0055.html
(but don't peek - generally)
Stuart: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0080.html
Partick (middle ground):
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0123.html
Mark Nottingham:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0048.html

Syntactic understanding is not peeking:
Norm: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0086.html

Truely opaque URI; syntactic conventions about schemes and #s would
require rework of existing specs.:
Jonathan Borden:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0133.html

Robustness prinicple applied to URI:
Norm: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0095.html
Stuart(some appeal):
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0100.html
Mark Baker (disagrees):
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0104.html

Identity and Identification...don't get confused:
Roy: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0102.html
Stuart: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0103.html

Indirect reference and context of use:
Roy: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0105.html
Stuart (agreeing):
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0108.html

Identifiers and Identification are related (inherently):
Len Bullard:
http://lists.w3.org/Archives/Public/www-tag/2003Jul/0106.html
Len: http://lists.w3.org/Archives/Public/www-tag/2003Jul/0107.html

Re: sec 3.3 ednote: Query responses rarely cached, URI treated as opaque
if they are:
Mark Nottingham:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0048.html

Target some of the comments at 'agent' implementers and spec writers.
Mark Nottingham:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0048.html
Mark Nottingham:
http://lists.w3.org/Archives/Public/www-tag/2003Aug/0056.html

Notion of Authority is 'overplayed' Most URI scheme defn's are
operational:
Larry Masinter:
http://lists.w3.org/Archives/Public/www-tag/2003Sep/0192.html
Stuart: http://lists.w3.org/Archives/Public/www-tag/2003Sep/0193.html

Assignment, interpretation on receipt and intention when making
reference are different things.
The finding is more about the last two which is separate from the
association of a resource with an identifier.
Larry: http://lists.w3.org/Archives/Public/www-tag/2003Sep/0213.html

Operationalise: Think operationally.
Larry (off-list correspondence).

Separate URI assignment from attributions of meaning:
Larry (off-list correspondence)

Forms as server suppled specifications for URI query construction.
Larry (offlist correspondence)
David Orchard (offlist correspondence).

TAG July F2F Comments:
http://www.w3.org/2003/07/21-tag-summary.html#metadataInURI-31
TB,DC,DO proposal: just section 1 plus examples inc. 
 - what format spec. designers can do. (DO)
 - Give an example of a relative normative spec that is providing
policies for looking within the URI.  (RF)
 - add a a mention that fragid needs mime type in 3.4 (TBL)
 - Section 1.1, part one - We need an example. (RF)
 - possibly merge sections 2 and 3 into a URI component centric view.
(SKW)
 - Need to be clear that 'authority' propagates through specs. (TBL)

Deciding how to move forward

Stuart's email makes clear that there's been quite a lot of discussion, and as someone who's just taking responsibility for this finding, it's clear that we need to strike a good balance between:

Exploring all of that feedback in detail, rereading every email, cataloging every issue raised, and making sure it's resolved satisfactorily.
Trying to identify some key points that we think we want to make, editing and republishing the draft finding to reflect those points, and encouraging anyone who (still) has a concern to (re)state that concern.

I think we need to do some of both, but my inclination is to pursue mainly the 2nd path. It's going to be very painful to debate each of the old email points in detail, though I will certainly read them and try to extract key ones. So, in the section below, I offer some strawperson proposals on what the key points of the finding should be.

Proposed key points of a finding on metadataInURI

The following in part reflect my own tentative opinions, and in part my distillation of what appeared to be emerging consensus in earlier discussions. So, I propose that the principle goal of the finding be to make the following points:

Those with authority over resources (what are the right words here about ownership?) MAY encode metadata in a URI for their own private purposes. For example, a site MAY establish a convention that URI's ending in .html do indeed invariably denote resources with HTML representations. When doing this, any standard mechanisms established by pertinent specifications SHOULD be exploited: for example, resources which form a hierarchy SHOULD be named using RFC 3986 hierarchical URI syntax (I.e. using "/" characters to set off components of the hierarchy.) Those with authority over resources may freely use such metadata in processing or serving the resource (e.g., it is appropriate for a .htaccess file to indicate that all URIs ending in .html should be served using media-type text/html.)
Quoting from the 8 July 2003 Draft Summary of Principles: "People and software making use of URIs assigned outside of their own authority (i.e. observers) MUST NOT attempt to infer properties of the referenced resource except as licensed by relevant normative specifications or by URI assignment policies published by the relevant URI assignment authority."
Again quoting from the 8 July 2003 Draft Summary of Principles: "A URI assignment authority MAY publish specifications detailing its URI assignment policies. Policies may detail the use of resource properties (version, creation date, author) in making URI assignments. Policies may detail the structure and semantics of URI query components served by resources under the control of the authority. People and software using URIs assigned by that authority (i.e. observers) MAY make use of such published information. For example the structure of URIs used to identify W3C technical reports is documented in [PUBRULES]. However, observers are cautioned that assignment policies are not generally subjected to standardization and may be changed by the relevant authority at any time. Software programmed on the basis of such a policy is at risk of becoming obsolete."
There is always a cost to peeking into a URI to get metadata. Even when Web architecture and this finding say that you MAY do so, you should be reluctant. Software that peeks is less likely to be work with arbitrary resources than softare that doesn't, so only peek if you have good reason. (I think the latest 8 July 2003 Draft makes this point, but not early or forcefully enough.)

I'd be happy if the finding made those four points, perhaps with some examples.

Change log

28 March 2006: [Noah Mendelsohn] Changed incorrect meeting date in table from 7 July 2003 to 2 December 2002.