Evergreen standards and Registries at the W3C -- 24 Oct 2018

<nigel> scribe: nigel

dsinger: David Singer, Apple. Jeff Jaffe asked me to work on evergreen standards.

Melanie: Melanie Richards, Microsoft

dbaron: David Baron, Mozilla

<TabAtkins> Tab Atkins, Google

<melanierichards> Melanie Richards, Microsoft

<Mek> Marijn Kruisselbrink, Google

nigel: Nigel Megitt, BBC

Janina: Janina Sajka, chair APA WG

<tink> Léonie Watson, The Paciello Group (TPG)

<azaroth> Rob Sanderson, J Paul Getty Trust, co-chair of JSON-LD WG

Evergreen Standards

dsinger: "Screwthread" standards, work hard, publish, go to the bar, walk away.
... Simple characterisation.
... Sister org, where victory is never declared, continuous evolution, "Living Standard"
... W3C circumstances where more incremental, less big bang better
... 1. Registries, not big edits to each row, just adding new rows.
... 2. Spec maintenance, minor fixes, no obvious big bang moment to go to Rec
... Not obvious those two use cases match to W3C
... Want to get clear use cases where W3C approach is a poor fit and evergreen would be
... better.
... Challenge to W3C: IPR, snapshot process, value added by reviews.
... Other groups say that works, wide review with customers, standards bodies is good.
... Meshing that value with incremental development is a challenge.

<dsinger> https://www.w3.org/wiki/Continuous_Publications

dsinger: Wiki page
... Not a lot of conclusion. Need to work out what would work from the point of
... view of the producer of the standard, lawyers, reviewers etc to get evergreen standards
... that work.

ChristopherA: I have some examples.

dsinger: discussion on the wiki for how to ease into this.

ChristopherA: I'd like to offer what's happening outside within context of W3C.
... Credentials CG founded 6-7 years ago.
... Spinoff of web payments
... Verifiable Claims WG span out of that, will probably close in March.
... They discovered there are things that, because of limited time, have to be deferred to
... registries. First one was a status registry. CG's do not expire, and CCG has been active,
... that WG charged the CCG with managing the registry.
... 5 Registries now, 4 for DIDs, 1 for Verifiable Claims.
... Unlike many other CGs the CCG is active, 30+ people per week participating.
... Lots of work items and stuff to do, we don't have to close but we can't do standards.

jeff: Can you say what's in it?

dsinger: Pointer?

<dsinger> see https://github.com/w3c-ccg/

<fantasai> https://w3c-ccg.github.io/did-method-registry/

<dbaron> https://github.com/w3c-ccg/did-method-registry, etc.

ChristopherA: DID registry, CryptoSuite, Verifiable Claims Status, ...
... Begun to create rules for putting them in. In the CCG Charter there is a statement
... that all documents relating to credentialing are welcome, the group allows new
... work even if it conflicts with previous work as long as there's an editor for it.

dsinger: What is the status of each entry in the registry?

ChristopherA: We list them as preliminary, don't expect all implementations to recognise

dsinger: No presumption of implementation of everything in the registry?

ChristopherA: No, no expectation

dsinger: That affects the IPR.

ChristopherA: CCG has IPR.

edent: What is the user need for it?

dsinger: That's the purpose of this session

jeff: At a metalevel, there must be a need, because all development is continuous, so
... we feel that there's the need somewhere. When it comes down to a particular spec,
... we want to know what kind of document is appropriate.

janina: People think the APA mappings are a good candidate.
... Need to think about how to tell horizontal review when and how.
... W3 cares about more than just good tech, needs to serve community needs.

dsinger: Production side, accessibility mappings, what is the structure?
... And how can we make [miss]

michael: Accessibility API mappings are maintained by platforms not by web devs.
... Largely a mapping exercise. ARIA is a technology. The accessibility API say how it is
... expressed on various platforms. Need to ensure interop.
... There can be discoherence even on the same platform.
... Sometimes a platform has to create a new API feature to map to an ARIA feature, so
... getting 100% coverage is difficult. Lots of people think use evergreen approach for that.

Janina: In terms of useful guidance, we try to look at features [scribe didn't get it]

<azaroth> link: https://www.w3.org/TR/annotation-model/#motivation-and-purpose

<Zakim> azaroth, you wanted to suggest use case for Annotation WG

azaroth: Use case for registry - in a previous WG I Chaired, the Annotation WG, we have a
... list of motivations for annotations. One of the most common questions is "how do I create
... new motivations". There's an index that says technically how to do it.
... The follow-up question is "how do I tell people I've created it"?
... There would be a large number of tables in W3C specs that we could instantiate as
... registries rather than hard coding in specifications.

<azaroth> (Rob Sanderson, J. Paul Getty Trust)

dsinger: One of the challenges in review is the tension between spec reading effort and
... if you leave a feature in a spec for 6 months and then comment then it's too hard to
... change later. Tension between periodic look and staying on the ball.
... If we can manage it with the right tooling and track what's happening as it happens
... then it would make it better for producers and reviewers.
... For the lawyers, I did draft a continuous IPR process, they said we want snapshots.
... Exclusions are rare enough that we don't need to optimise for it.

Michael: I like the tooling idea, but too many updates are hard to stay on top of.

dsinger: I asked Vivien yesterday about HTML GitHub digests - they're basic at the moment,
... if they were tagged with significance that would help me to review quicker.
... Then an a11y group could review major stuff from the HTML group more easily.

fantasai: HR groups don't want to review individual commits.
... The granularity doesn't match what they need to review.
... When we compile the changes list, that the TAG said was useful, it's very explicitly
... not a list of commits, it is a high level "this is what we did".
... Different for diffferent specs.
... For some, every impactful thing has a list of diffs.
... For early stage WD, tightening the definitions doesn't go on the changes list because
... it's too minor, but new features would go on that list, or behaviour changes.
... A reviewer doesn't want a pile of commit messages. Understanding the implications
... and the interconnected change sets is difficult, you cannot automate that with labelling.
... It has to be done by a human doing a real summary.
... In terms of getting in review at the correct time, I'm not in an HR group but when I
... review specs I will not continuously review, I can't context switch that often.
... I will review all the way through and can't keep coming back every 3 weeks to track small changes.
... Need to set things up for chunked review not every month.
... Reviewers need to generate a list of comments and then review.
... I understand the need for registries, but those use cases have scoped additions.
... In some cases you don't need to look because the architecture of the registry has been
... reviewed and its good. It doesn't need continuous HR once the initial structure has been set up.

Ralph: Preface: confession that a significant part of my genetic materials is that tooling can help.
... Connecting what Rob Sanderson said, a living spec is distinguished from a registry
... because the modularisation is different.
... If we can annotate the changing parts of the spec, as per I think dbaron's comment this morning,
... if our specs can record the design rationale then those who follow us are less likely to
... undo what we did and more likely to improve it.

<azaroth> +1 to Ralph

Ralph: Combine those thoughts maybe?
... couple the capture and annotation of design rationale at a granularity that facilitates continuous review as well

+1 to rationale

dbaron: I did say something along those lines this morning. One thought on what Ralph

<burn> +1 Ralph - this is similar to IANA expectations in master docs that define registries

dbaron: said is some of what distinguishes registries from living standards, is that they are
... something that has the expectatino of frequent updates (registries).
... Historically W3C has one process per registry, not the right number!
... I queued myself because you had asked why living standards.
... One of the reasons is so that we have more freedom to choose our priorities rather
... than have the Process dictate our priorities to us.

<Ralph> Ralph: registries tend to favor those things that are highly modular and change monotonically

dbaron: A bunch of things happen during standardisation.
... Iteration, getting more concrete, with testing, implementer feedback etc.
... Depending on the situation and your priorities, speed to market, if you see your
... problems being interop or insufficient features,
... I think I want those needs to drive what we emphasise in terms of where we put effort.
... Waterfall spec dev process has the process driving the effort, e.g. to add one feature,
... you have to start playing the games of "if add this, slow that other thing down".

dsinger: We're not going to CR often so we'll batch it up and that's costly so bundle too much in.

dbaron: Yes you let your needs drive your priorities.
... And it lets you avoid extra work to work around that.

<burn> @edent when you talk can you compare to IANA registries? Arguably fairly successful structure

dbaron: Today instead of integrating text in to the relevant specs we will split things up
... to satisfy the process and pay extra cost

<jeff> https://www.w3.org/wiki/Evergreen_Standards#Use_Cases

jeff: Both Michael and Elika talked about the challenges of HR.
... Draw attention to the use cases doc in the AB wiki.
... Of those, the first 3 I believe will have the property that they're likely to be light from
... the perspective of IPR issues, and they're also likely to be relatively light in terms of
... how much HR is required. If those opinions are correct, then I would suggest that
... those are a low impact way to experiment with evergreen standards process. We don't
... have to face the IPR issues immediately.

Janina: Yes they are low impact.

tink: Some suggestions on HR.
... 1. Don't fall into trap that every spec will jump to evergreen.
... Forgive me for mentioning HTML but we did put a human readable change log in.
... We found it a reasonably low overhead.

<dbaron> (HR == horizontal review)

tink: Only have to review 5-6 issues each time.
... Not an enormous amount of work for every spec.

Janina: It's important we talked about APIs. They're often something we hate to review

<Ralph> [and kudos to the CSS WG for their readable changelogs with pointers to GitHub issues for those who want more detila]

Janina: horizontally. Very little explanation about what they are trying to achieve.
... Have to understand whole architecture before reviewing. If that keeps changing, it
... is worse than impossible.

janina: Even on a 6 month basis it's a pitch for explanations.

<azaroth> <nigel> +1

<david_clarke> +1 to depth needed for APIs

janina: English language can be used, a place to focus and to try to understand
... If you can't understand your API in plain English how do you know you've achieved your task.

ChristopherA: Still confused - theoretically a WG has a period, then it's done, it can be
... extended or reChartered. How can a WG end and still support?

dsinger: Charter periods end, my theory is that specifications are:
... 1. In active use, and actively maintained by a WG
... 2. Irrelevant and dead

ChristopherA: Fundamental shift in the Charter.

jeff: Not really, in the waterfall process today, specific versions of the spec are fixed and
... done but the group doesn't go away.
... You can move up, but each spec completes.

ChristopherA: Maybe working in the more controversial groups, which have to narrow
... down their Charter to begin working that they can't do anything.
... Becoming the standard for new groups is No, do all this work first, then they say you
... must restrict to a limited scope.

dsinger: Question to think about if fixed period WGs matches evergreen process

<Zakim> edent, you wanted to mention "GOV.UK is very happy to share our process for generating and maintaining registries."

<dbaron> There are also precedents for maintenance being handed off to another working group.

edent: On the subject of registries, dbaron made a good point, consistency is needed.
... UK govt maintains a list of 44 registries, with open source tooling we can share.
... One thing to point out, how we compare with some of the other registries.
... One good example is a list of recognised countries.
... On any govt form, the drop down for countries only has the recognised list, and others are not there.
... That's useful, because there are start and end dates.
... IANA registries don't have those dates, and I think they're really useful.
... And to offer in different formats, JSON etc.
... Standardised registers should be accessible in a similar way.
... Consistency of model.

dsinger: Heretical question: I maintain MP4RA including codecs. There's an IPR process

<Ralph> [+1 to Terrence's observation that temporal (as well as provenence) data is useful to include with entries in a {register,registry} ]

dsinger: in specifications, but not in registries. If someone comes to MP4RA I don't care
... about the licensing terms, it is simply a point of information.
... Do we actually have registries where we care about the IPR associated with individual entries?

edent: If you mention it, like IPR good/bad, and the users care, then that's ok

dbaron: Depends which specification establishes that registry.

<edent> For references, UK Government Registers - https://www.registers.service.gov.uk/

dsinger: That was my hypothesis, spec should say what the criteria are for managing the registry, adding entries etc

<Zakim> nigel, you wanted to ask about interface with developers

<inserted> scribe: fantasai

nigel: A lot of spec development seems to happen with main constituent being developers
... Developers like to write some code, try it, if it works fine, if not try again
... They also like specs because they tell the algorithms to implement
... But that's only one constituent, and there are other constituents
... They can't work in a similar timescale

<Ralph> [the IPR policy can be a property of the registry or of each entry in the registry. In the latter case the IPR might be recorded in the registry or might (better IMHO) be recorded separately as a property of the "owner" of the registered entity]

nigel: We need to make sure that we meet the needs of all constituents
... Janina mentioned descriing purposes of the algorithm described
... Another issue is generating tests for the changes to the spec
... Ways to measure the outcomes
... Might be in middle of that onion, rapid development can happen, but on the outside it's slow
... Gathering requirements, and evaluating outcomes can be done less frequently than the iterative process of development in the middle

<inserted> scribe: nigel

fantasai: I wanted to point out I agreed with dsinger's categorisation of specs. Specs

<azaroth> +1

fantasai: are almost never completely done. If it is actively being used, users will always find errors.
... It does not make sense to close out a WG and not recharter some kind of maintenance.
... Need to recharter a maintenance group or the spec is out of date or dead.
... Agree with saying what you're trying to do on a spec.

ChristopherA: Sharing a negative pattern. SSL TLS: I'm co-author.
... The point of review was to have a nice limited list. In the end you needed 110 to get 90% of the internet.
... Things like heartbleed, and an SSL2 attack against SSL3 that are both 20 years deprecated
... hit a whole bunch of servers last year.
... There has to be some way of deprecating. I pitched for every cypher suite to have an
... expiration date and require an explicit recharter to bring it back. Everyone fought that
... and I wish I'd won - it would have been more secure.

<Ralph> [overheard comment: "a use-by date"]

Closing thoughts

<edent> Here is the Open Source code for the UK Gov registers https://github.com/openregister

dsinger: I share Ralph's view that tools are good.
... E.g. in process doc editing, I do a diff between Process versions and florian looks at
... the pull list and we make sure we agree. It's a good habit.
... We write a narrative description of what's changed.
... This is really useful to me. If you want to see this happen please join the Process CG.

<edent> And the specification https://openregister.github.io/specification/

<dsinger> https://www.w3.org/wiki/Continuous_Publications

dsinger: Get on the wiki and say where it is missing points etc.

<Ralph> [and in the course of collecting the pull request list the issue ids can be captured for inclusion in the narrative change log]

dsinger: We need to turn the wiki thoughts into the process.

<edent> Join the Process CG at https://www.w3.org/community/w3process/

<azaroth> Thanks edent!

dsinger: Please join the Process CG.

jeff: The AB has it's next f2f in a month's time and David is on the agenda to share where
... we stand so any input before then is extremely timely.

dsinger: This number of people in the room is really helpful.

<natasha> https://github.com/w3c/w3process

dsinger: The AB's wiki is deliberately public to allow comments.

tantek: Is there still a window for the next process iteration?

<fantasai> https://www.w3.org/community/w3process/

dsinger: Not clear to me, I would like to go to ballot soon for an annual cadence.
... I thought there should be an approved process for registries used by a WG.
... Not sure if that is functionally different from adding a process point.

jeff: I think we're talking about a year from now realistically.
... We have zero time left because of other requirements.

mchampion: the CG process worked quite well.
... This could be handled like errata.

dsinger: Getting published and approved is minor compared to what it says.

<fantasai> nigel: In a WG, I've always assumed that a change to a registry is just subject to consensus of the WG

<fantasai> dsinger: Linkrel registry, they write in the document what is the admission criteria

dsinger: Thank you everyone.

- DRAFT -

Evergreen standards and Registries at the W3C

24 Oct 2018

Attendees

Contents

Evergreen Standards

Closing thoughts

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output