Tracking spec maintenance

13 September 2023


hober, plh
Jeffrey Yasskin
fantasai, jyasskin

Meeting minutes

Slideset: https://jyasskin.github.io/spec-maintenance/meetings/tpac-2023-slides.html

jyasskin: Preface: I haven't done all the homework to get this right

jyasskin: Basic problem I'm hoping to adress
… is we work on specs, and then editors move on to other things
… but then issues come up, or osmeone starts implementing
… and editor doesn't notice
… even if we notice, don't always have allocation to respond

[Slide 3]

jyasskin: [presents numbers]
… e.g. mean time to update is 2 years
… mean issue age is 3 years
… what do we need in order to address this?

[Slide 4]

[Slide 5]

jyasskin: We need to triage, distinguish by priority
… unblock implementers

Slideset: https://jyasskin.github.io/spec-maintenance/meetings/tpac-2023-slides.html

[Slide 6]

jyasskin: need to report problems
… chrome team has internal tools
… chrome bugs, pretty graphs of which teams are doing well or poorly about managing bugs
… so I'd like to integrate specs with that system
… I'm guessing other orgs have similar systems
… also if we have a public description of "this spec is having trouble"
… maybe someone will volunteer from community rather than just browser vendors

[Slide 7]

jyasskin: People have built some tools
… foolip built a tool called "spec Reactions"
… shows how many ppl have reacted across browser specs, baed on recent or total
… to help prioritize action
… W3C also has a list of recommended labels
… mostly about horizontal review, but also bug vs enhancement, help wanted, etc.

[Slide 8]

jyasskin: What did I miss? I want this to work for whole community, not just my company

<jyasskin> https://docs.google.com/document/d/1sVuuxIawZdPaT5Fbc_9nM6k7rC0nnf0N_khdM7tyqNs/edit#heading=h.3t624fypqsd3

jyasskin: Open floor for discussion

plh: Thanks for organizing this session
… spec maintenance has been discussed for many years
… tried to solve, but mostly failed, so glad you're picking up
… big +1, we have many repos, so if we want to help triage the first thing is
… we need to tell repos what to do to help triage
… you mentioned list of labels on github.io, glad you're aware of it
… big +1 to sending PR to update
… but there also needs outreach effort

plh: What I do find is that training is a constant effort
… haven't talked about horizontal review in awhile, and new people don't know what they mean and misuse them

plh: For the problem itself, if an editor is not responsive, talk to Team
… then, idk who is Team
… consider SVG. Who is Team? Not only is editor unresponsive, we don't have an editor, don't have a WG
… Maybe "good first spec to maintain"? But SVG isnot a good example for that
… but how can we attract editors to maintain abandoned specs?

<Zakim> jyasskin, you wanted to answer about teams

jyasskin: Wrt Team, Chrome as a team that's tasked with maintaining implementation of any particular feaure
… not sure who that is for SVG
… we think that the Team should be maintaining spec also
… I was thinking is that, we could link from repo to Google team responsible for that feautre
… maybe put in W3C metadata

hober: Chrome metadata in W3C metadata?

jyasskin: Each company to put implementer metadata
… e.g. for Chrome, here's the ID for the responsible team. For Safari, same thing

plh: W3C data doesn't work for WHATWG

jyasskin: We should have one schema for this

plh: maybe not call it w3c.json in the first place
… but I welcome more data into those files

nicolas: when you have a spec, whose responsibility is it ultimately to triage the GH issues?
… is it all the participants in teh group, or the editor?

hober: depends on the group. E.g. in the WHATWG editors are expected to triage their issues, but other groups it's different

nicolas: If no-one thinks they're responsible, then won't get done

<plh> W3C.json documentation

jyasskin: I think saying editor is good default. If editor is not responsive, then can go to implementer teams and ask for new editor

hober: Chairs have responsibility of appointing editors at W3C
… implies chairs should notice if editors are unresponsive

<plh> Report on W3C Working Group repositories

hober: I think presumption of current process is chairs wil notice
… in practice they don't
… in particular, for CGs etc., there isn't necessarily that expectation at all
… I trust Alan in CSS, to notice if a spec is abandond

astearns: I was nodding when you said this actually

jyasskin: chairs need help finding editors

astearns: yes we do!

hober: plh, you asked earlier, how can we attract editors for abandoned specs
… and as we just learned, everyone agrees it's hard

astearns: I like idea of implementing team registries
… because that's a narrower group of people I can go to and say "something needs to be done, let's figure out who's the victim"
… much harder than going through WG

florian: I'm wondering how this might tie into fact that at certain publication points
… people are expected to produce a disposition of comments
… to say which comments received, which addressed
… that is infrequent, so not sufficient
… but sometimes this will require editor to notice where they didn't do the job
… but many years in between DoCs, so not sufficient
… idk if we have a strong expectation, practice varies by WG and editor
… but maybe we can try to tie two together
… the more we track what's happening, easier to report when you have gone through the issues

hober: wrt finding editors, there are some things we can do
… I really like direction you're going, jyasskin
… but no matter what we do, it is going to always be challenging to find editors
… general economic incentives of the world are that engineers don't have much time
… if have to choose between fix bugs (and get metrics improved) or to fix some spec (and not have bosses notice and give raise)
… I don't know if, as an industry, we have a good story for rewarding this work, recognizing and elevating it
… I would love ideas how to change that, but outside of that
… I don't know what to do to attract people to it

<Zakim> jyasskin, you wanted to talk about spec lifecycles

jyasskin: On florian's comment, transitions on REC track happens much later than this problem
… a lot of this happens in CGs
… a lot of that is Chrome's fault
… but within WG, it's too late for this problem
… but reporting bit, scanning and saying which issues are out of SLO
… my attempt to fix economics
… to show managers if people doing a good job
… if we can show the metrics, our management systems can reward people for making the number go down
… we need management to buy into idea that these bugs are as valuable as implementation bugs
… but if they could see it, then they could reward ppl for improving it

<florian> AB issue on appreciating standards work: w3c/AB-memberonly#53

florian: I want to mention that AB has longstanding issue, without an answer, about appreciating/recognizing/elevating standards work
… if we can find good ways to make that happen, that's a known gap

plh: I'm very supportive of this work, expanding our tooling
… doesn't just have to be in W3C
… e.g. our horizontal tooling works with WHATWG as well
… our horizontal groups work with WHATWG specs as well
… so supportive of adding info
… E.g. in specs, you find link between the spec and WPT!
… maybe the link should not point to team, but within the spec
… my principle is put the info close as possible to the people who can move it
… one of problems with W3C is only Team can edit website
… so that's why w3c.json were created in GH, so that ppl in GH can edit those files

caribou: wondering about tooling etc.
… if nobody actively maintaining something because editor went away
… adding more tooling is not going to solve the problem
… ok to add tooling to help continue work
… but problem to tackle here is to find a way to avoid editors dropping work
… if they are already absent tooling won't solve
… maybe we need some kind of rescue task force to help those cases
… OK as well to drop some specs, if they are not progressing
… currently we have no way to signal specs that are no longer very active
… in some cases not limited to one WG, specs can have interest in several WGs, so maybe find ppl from outside join WG and fix it
… or taking over

[audio cut]

<Zakim> plh, you wanted to mention service workers

plh: One thing I wanted to mention, another spec which may become unfortunate case
… is service worker core spec, nobody is maintaining it
… or pushing it to a more stable state
… actually a breakout on that, we don't even have a WG chair
… but there is interst in new stuff! But nobody interested in core spec

jyasskin: On finding new editors, Google has an internal policy

<caribou> I was suggesting something like a "spec rescue" guideline or procedure to decide how to fix a stalled spec (find new people, stop work, or punt to other group)

jyasskin: that implementation teams are responsible for their specs also
… for service workers, we'll have to discuss with director if he's planning to follow that policy
… but W3C, and ppl looking for maintainer, that's a route they can take that hasn't been visible before
… Tess, would that work for Safari?
… if there was a Safari editor that left Safari and not active

<astearns> agrees with caribou that "spec rescue" guidelines for chairs (and the team) would be good to have written down

jyasskin: is it appropriate to come to you or some team to ask for an editor?

hober: yes

jyasskin: that's the next thing to try then
… we've been assuming editors to volunteer from the community
… but we can ask the rich corporations to provide an editor

<Zakim> jyasskin, you wanted to talk about finding new editors

npm: Is it fair to say if there's an implementer, if Google or Apple is an implementer, it's OK to ask for an editor/maintainer of the spec?
… that seems reasonable if you're implementing something, you should do some basic maintenance
… but don't know if that's often the case

myles: usually not the case

npm: Something to look at, implementers why don't you edit
… giving specific people ... once they have formal responsibility, easier for that person to say "manager, I am formally responsible for that, give me credit for it"
… at least at Google we give credit for spec work
… though more for getting things to ship
… maintenance is much less rewarded
… we need to improve on that
… that can be a route towards having better incentives

florian: if noticing and assigning editors is solution, it works where there's interest and resource
… but also we have interest but no resource
… in some WGs we have invited experts who have ability to do work
… they just might not have time
… so maybe we can make a match
… While pinging to a well-resourced corp to provide editor
… might not need to be your own personnel

jyasskin: contractors exist

jyasskin: OK, talked about finding new editors
… I sketched the sorts of labels i think we need in the Google doc
… how do pppl feel about these labels?

<Zakim> jyasskin, you wanted to https://docs.google.com/document/d/1sVuuxIawZdPaT5Fbc_9nM6k7rC0nnf0N_khdM7tyqNs/edit#heading=h.k9bqpmce10qt

fantasai: blocks-implementation is useful, but I haven't seen it before.
… CSSWG has several types of needs-feedback depending on what's needed. commenter-response. needs-data. needs-proposed-text
… Don't know how many we need to make standardized.

astearns: implementer priority would be helpful, but issues being implementer or not don't want to prioritize like that. All issues are valid

fantasai: most issues in the CSSWG aren't immediately blocking an implementer. Lots are clear problems that will prevent people form getting it right. But if someone's actively implementing and runs into something that needs fixing, it makes sense to prioritize those.

astearns: [didn't hear]
… would be helpful to get implementer feedback on what they prioritize
… but want other than implementers to be able to communicate priorities, too

hober: One of the most interesting sets of labels are blocking labels
… so I can tell, this issue is stuck until something happens
… I woudl like to suggest that should be clear from the label
… who are we blocked on? what are we blocked on?

CSSWG labels
… blocking CR or blocking implementations from advancing or...

florian: Issues from anybody can be important
… but implementer issues have more time-sensitivity
… e.g. if we miss window of opportunity, they will ship something else, and it will be harder to do something about it
… also if team is focused on this area, it's an opportunity for rapid progress if we schedule it right
… not at the expense of recognizing that anyone's issue can be imprtant

<astearns> implementer actions create time-sensitivity, but that can make non-implementer issues more important too

jyasskin: My thought was that the priorities would not care about if implementations
… but impelmenters can boost SLO by marking

fantasai: General prioritization I'd leave to implementation teams. If they need issues addressed, talk to the chairs.
… chairs in CSSWG are very responsive. Just need to say what the implementers are.
… Might have different opinions on the generic pile of issues. Different WGs might want to develop their own priorities.

fantasai: Can't necessarily standardize, except for "this is blocking an implementer"

jyasskin: CSSWG is unusually functional at W3C, doesn't mean other groups don't need
… maybe the CSSWG wouldn't adopt some pieces of system, or wouldn't feed into tooling
… but because CSSWG gets things done, they don't need this as much
… we could still propose this for CGs and less functional WGs

plh: My experience with new things and propagation across groups
… we need to have this successful in one group first
… e.g. horizontal tooling was based on experience of i18nWG
… I took their tools and applied it to the all the HRGs ... TAG is still a bit fuzzy
… groups each have their own culture
… if something works well in CSSWG, and want other groups to adopt
… if not adopting, maybe didn't know it was possible

florian: We've been speaking about triage in terms of priorities
… but e.g. in CSSWG, we have a lot of specs
… part of triage is assigning issue to spec
… that probably doesn't occur in WG with 2-3 specs

hober: CSS uses a mono-repo which causes this problem

plh: very few groups do this
… model is usually one spec per repo

<Zakim> jyasskin, you wanted to offer pilot groups

florian: there are upsides and downsides, that's one downside

jyasskin: You mentioned groups to try out
… initial plan for doing this was that I would figure something out and try it in a Google CG
… realized I shoudl bring to community so that we can adopt as community, but we can pilot on a Google editing group

plh: grassroots efforts work better
… e.g. wpt

plh: Another group to reach out to is spec-prod


fantasai: or chairs@


jyasskin: WICG, PATC, etc.

astearns: agree working out in a group
… I suspect WICG needs more than others
… I'd like to be informed on progress
… e.g. I didn't know spec-reactions tool existed

jyasskin: hasn't been publicized yet

jyasskin: I think we're done! Thanks everyone for contributing
… I'll announce and discuss on spec-prod

Minutes manually created (not a transcript), formatted by scribe.perl version 221 (Fri Jul 21 14:01:30 2023 UTC).


Succeeded: s/maye/maybe/

Succeeded: s/??/nicolas/

Succeeded: s/???/the WHATWG/

Succeeded: s/so not idea/so not sufficient/

Succeeded: s/???/npm/

Succeeded: s/If/But if/

Succeeded: s/etter/better/

Maybe present: astearns, caribou, fantasai, florian, jyasskin, myles, nicolas, npm

All speakers: astearns, caribou, fantasai, florian, hober, jyasskin, myles, nicolas, npm, plh

Active on IRC: astearns, caribou, fantasai, florian, hober, Ian, jyasskin, npm, plh