POWDER Working Group Blog

POWDER is a W3C Recommendation

Phew! It's been a long haul – too long really – but we're there and POWDER is now a recognised Web Standard. We received good feedback from W3C members who reviewed the documents (thank you) and these lead to a couple of trivial – I mean trivial – corrections here and there. What next? The specs are done, the key tools are in place including the validator and two conformant processors (created and maintained by i-sieve Technologies and University of Insubria) as well as the toolkit for using POWDER within a framework like Jena and Pellet. I-sieve has also created a tool for generating mobileOK conformance claims for a Web site using POWDER that also adds in Dublin Core metadata and Creative Commons licences if desired. The QUATRO Plus project that helped to foster the development of POWDER ends this month but the partners are all actively seeking new ways to exploit the work, particularly with trustmarks, aiming to balance the wisdom of the crowds with the wisdom of the experts. Other members of the WG are set to exploit the technology in their own systems with focus on improved search and discoverability. The specs may be done, but there's still plenty of work to do.

POWDER is a Proposed Recommendation

The headline says it all!

See the announcement from W3C.

The W3C membership now has a chance to review the docs and approve (or not) the final transition to Recommendation. As far as the working group is concerned, the specification is complete. Work will continue, of course, on deployment and the development of new applications and community building. Although the specs are done, the work never stops...

Last Call closed – preparing for Proposed Recommendation

The third Last Call period ended at the start of this week and lead to what amounts to errors being corrected in the IRI canonicalisation sections of the Grouping of Resources document (which was the focus of the call, see implementation experience). A small number of extra tests was added to the test suite to cover this important aspect of matching IRIs against POWDER documents. No changes were made to the Description Resources document or Primer.

The Formal Semantics document was reviewed by the OWL WG and the informative section on expressing POWDER in OWL 2 was subsequently updated. At the time of writing, we're dealing with a substantive comment from Michael Schneider. This is likely to lead to a rephrasing of Section 4.3 but no substantive change that would impact on implementations.

So, once that issue is resolved, the next step is Proposed Recommendation. As flagged previously, we're skipping Candidate Rec stage as we already have plenty of implementation experience to hand.

So is the work on POWDER really over 'bar the shouting' as the phrase goes? i.e. have we done what we set out to do? Let's see.

The charter says: The POWDER Working Group is chartered to specify an RDF vocabulary for specifying authorship of and authentication of Description Resources, a specification for associating a Description Resource with a class of Web resources, predicates for declaring classes of resources based on string functions of the resource URIs, and a protocol for accessing Description Resources. This seems well covered!

The charter gives specific examples of statements that must be expressible in POWDER:

All resources on the travel.example.com domain are suitable for display on mobile devices.
All resources on broadcast.example.com/clips are video clips that are suitable for all ages.
All resources on example.com are accessible for all users and meet WAI AA guidelines except those on visual.example.com which are not suitable for users with impaired vision.
Web crawlers are welcome to explore all resources on example.com except those with a path beginning with 'private'.

The first two are exemplified in the documents published by the group (see mobileOK and ICRA examples). The other two could easily be encoded in similar fashion using suitable vocabularies.

The charter sets out specific deliverables:

A W3C Recommendation providing a normative encoding of description resources (we refer to it internally as 'the DR doc').
A W3C Recommendation describing methods of defining groups of resources (referred to internally as 'the grouping doc).
A W3C Recommendation providing an HTTP-based mechanism for locating and accessing description resources associated with a particular Web resource (see Section 4 of the DR doc).
Subject to available resources, the group will develop a description resource validation tool. Done! (along with several other tools).

The charter also calls on the group to resolve the open questions raised by the Web Content Label Incubator Group: a previous blog entry covers that, which just leaves the requirements derived from the use cases. An exhaustive re-listing of the requirements and how they're covered would make a very dull and largely pointless blog entry but I'll pull out a few for special mention.

3.1.8 Reference: It must be possible for a DR to refer to other DRs or other sources of data that support the claims and assertions made. This is possible through the certified and supportedby terms discussed in Sections 5.2 and 5.3 of the DR doc. This also covers 3.2.4 Link to Test Results where the requirement is to be able to link to test results (perhaps encoded in EARL).
3.1.9 Standard Vocabularies: There must be standard vocabularies for assertions about DRs. Actually we kept the vocabulary to a minimum (i.e. reused existing ones as far as possible). However, terms such as issuedby, issued, validfrom and validuntil were introduced. issuedby caused particular debate as it is so close to terms in both the FOAF and DC vocabularies – but which one to support? Can we not support both? Which is the 'stable standard?'. Cue a lot of e-mails and discussion!
3.1.10 Identity: DRs, their components and individual assertions should have unique and unambiguous identifiers. Actually we haven't done this as such. POWDER documents have URIs and you can give individual descriptorsets identifiers but, largely due to RDF's lack of (formal) support for named graphs, each component of a DR does not have an identifier. Furthermore, we deliberately made sure that classes in POWDER-S documents typically had node IDs that can't be referenced externally so as to avoid assertions being added to those classes from external sources.
3.2 Fitting in with Commercial or Other Large Scale Workflows: This requirement — actually a heading for a whole set of requirements — is at the heart of the operational/semantic split. POWDER has to be as easy to use in a commercial, practical environment as possible. Hence the XML encoding and, in particular, the ordered lists of DRs through which we meet requirement 3.2.3 Default Description.

Unless something goes wrong, we should be ready to begin the transition to Proposed Recommendation process next week (or the week after as a worst case).

We really are nearly there.

The What and Why of the Third Last Call

Since making our second Last Call for comments at the end of last year we've responded to all of them and have been completing work on the tools - the processors, the XSLTs and so on - all of which has now been done.

So why aren't we at PR yet? There are a couple of reasons for this. There has been an ongoing battle between (WG member) Stasinos Konstantopoulos and (W3C team member) Eric Prud'hommeaux over whether the POWDER Semantic extension applies to the data layer or the application layer. The two protagonists have agreed to differ whilst the document has gone forward.

The second issue that has arisen concerns the other area that has prompted a lot of comment - the canonicalisation section of the grouping doc. Despite many changes over the last year or so and input from the Internationalisation WG, (W3C team member) Thomas Roessler spotted more problems with this section.

After a lot of e-mail bouncing around on, I'm sorry to say, a team-only list, I had a long conversation with Thomas and (Semantic Web lead) Ivan Herman. Some context:

The Semantic Web (RDF, OWL, SPARQL etc.) is written solely about URIs, not IRIs. Furthermore, all URIs are opaque strings - they have no meaning. This goes back to Jeremy Carroll's useful distinction between "http://www.example.com" and <http://www.example.com>. In other words, for RDF etc. our issue of processing URIs/IRIs as meaningful strings doesn't arise.

IRIs - that is, URIs written in things like Cyrillic script, Japanese Kanji etc. - are going to be more important in the coming 12 - 18 months (according to Thomas who, among other things, represents W3C at ICANN and so has a good grasp of such things). For POWDER not to be useful now and in the near future across all parts of the Web would limit its potential usefulness.

As we have discovered, the encoding of IRIs and International Domain Names is less than clear. However, there is a recognised path to take in this area which involves converting IRIs into ASCII - essentially converting IRIs to URIs.

It is not the POWDER WG's job or wish to solve the whole issue, neither is it within our ability to do so.

The POWDER to POWDER-BASE transform XSLT doesn't do any of the canonicalisation stuff, and it would be hard, if not impossible for it to do so. Therefore, strictly speaking, some form of pre-processing is already necessary before the XSLT is applied (the Perl script does do the canonicalisation but now needs a bit of work).

Given all of the above, the group was faced with a choice between either making significant edits in all its documents to talk only about URIs or, as we have done, further amending the canonicalisation section to introduce the ToASCII function as defined in RFC 3490. It is this change that has prompted the new Last Call announcement, open until Monday 27 April. The documents published on 3 April to coincide with the Last Call reflect all the changes made following comments received in the previous calls and, with the exception of the highlighted sections of the Grouping of Resources document on canonicalisation, are considered stable.

One important, although editorial, change worth highlighting is in the Description Resources document. The definition of the describedby relationship type and the two POWDER Content Types is much improved. describedby is now included in the IANA registry of ATOM Link relationship types.

The Test Suite has been significantly updated in the 3 April version and is now unlikely to change significantly (we may add some new tests for the canonicalisation section). Perhaps rather perversely, the test suite does include a test for the informative section on HTTP Link. The Primer is considered stable and very unlikely to change further.

Minor adjustments to the POWDER Processors are in progress and, notwithstanding changes prompted by the last call, the group is expecting to go to Proposed Recommendation in May.

Situation Report 16 January 2009

The group's charter expired at the end of December so we're in something of a state of limbo (one might say unchartered territory). However, a lot of work is being done to get us ready for a request for transition to PR. The links below are all very much temporary but they are all up to date and reflect the true status of the group's work (the homepage and published documents are not being updated at the moment). Several tools are done and dusted:

The Validator. This is expected to move to validator.w3.org in due course
Perl-based processor. Again, this should move to a new home soon. There are minor improvements to make but it implements POWDER.
Semantic Processor. This is not a complete application, nor is it a conformant POWDER Processor, but is a command line-driven exemplar of POWDER-S. It uses Jena and Pellet to return descriptions of a given URI based on a given POWDER-S document.

Tools have also been produced that implement the transformations defined in the Formal document.

POWDER to POWDER-BASE. There are no known bugs in this XSLT.
POWDER to POWDER-BASE script. As the POWDER to POWDER-BASE XSLT uses XSLT 2, the validator was extended to include a transformation engine. It is accessible directly with documentation (such as is necessary) with the The Validator.
POWDER-BASE to POWDER-S. For most cases, this works fine. We're now throwing horrible examples at it that are revealing minor bugs but it's very nearly done.

An implementation report is being maintained. The POWDER XML Schema is also being debugged. Again, it's almost there. Actually, there are several interlinked schemata to handle POWDER and POWDER-BASE. The GRDDL transformation links will be included when we next make these available. A second, more sophisticated POWDER processor is close to completion. Based on PHP, it uses the schema to validate POWDER documents (the Perl-based processor uses the same code as the validator). Finally, most of the documents have been updated. It would be improper to link to them here but if you want to see updated versions of the DR, Grouping and Formal docs, plus the Primer and Test Suite, please get in touch and I'll send you the URIs. We're nearly there. Thankfully.