– DRAFT –
The Future of TAG Reviews & Other Output

scribe dmurph

Jeffrey: This is how the TAg participates in the community
… : We have ideas & listen to everyone's thoughts

Dan: About 10 years ago, we rebooted the tag
… : It helps to set the scene about where we are. A lot of people wanted the tag to be more impactful & response to the community
… : We wanted to have more people to act as a design authority in the W3C, not just answering big hairy questions about future of web arch, but also help people do the work they are doing to design new APIs
… : We had a reputation of coming around and helping people design & issues. But unfortunately it was mostly negative - people felt like you don't really want to ask the tag for help, as that will definitely prolong the work. Raise issues.
… : I was part of that TAG for one term.
… : In 2013, we put in place a more systemized approach on GitHub. Right now - we have an iteration of what we put in place in 2013. It's much more pull-based. We encourage people to request reviews at different stages. There's a template, it has gotten bigger and bigger.
… : Key element: Explainer. Alex Russell promoted that idea, that we should start that explainer. In early days of reboot. We made it clear what that should be, document which slots along side of your spec, and can explain spec to anybody who is not the same mindset as you.
… : There are other things we started to work on such as design principles, security, privacy, ... a lot of that stuff iterated out of the design review process.
… : Then we would take common things that happened, and write a design principle out of that.
… : The current way it works, and we think it provides value, is by that approach.
… : One of the questions I'm interested in - hearing from people - we have some ideas, some feedback is that reviews take a long time...
… : I also want to hear from people that have been interacting with the design group process to interate and make it more useful for people

Martin: This is not just about getting design reviews faster, also about improving the TAG

Dan: I'm only talking abouat the design review thing - we have other options as well. Part of the feedback is that - what is the correct settings on the 'mixer' on the level of power in design reviews vs other things we could do / or are doing.

Dan: I am not asking "what is the role of the TAG" - I don't think it's useful

<Zakim> Jem, you wanted to ask the level and granularity of TAG's design system guidance

Jem: I came to TAG because I saw pictures on the website, and I want to learn more. Dan answered one of my questions. I'm working for ARIA, APA, and AG groups
… : I'm sharing... I heard that TAG is trying to help & guide direction of ARIA w.r.t. HTML
… : Dan said TAG is trying to help people guide systems, and that's great. We have a rough idea of how ARIA and HTML should go together. What is the granularity of TAG reviews for something like this?

Dan: You mentioned that Lia was going to intro the other design of things.... (to jeff) ... That will be useful to do.

Jeff: I heard a disagreement between Dan & Mark - we should explain the disagreement, and then .... Lea & Mark, and then we should listen to people who haven't been involved in tha argument

Lea: I've been in the TAG for 4 years, I do see a lot of room for improvement in process.
… : The design reviews we are currently doing are useful. The question is not whether they are useful, question is how we can optimize time so we can add the most value.
… : The design reviews we are getting, we struggle prioritizing (ad-hoc). We spend time reviewing very low level features (one value, one css property) that are probably more suitable to be reviewed by the WG that is shipping impl
… : Or we find inconsistencies between browsers, and when point out,-- sorry too late
… : Unique position of TAG - broad deep technical expertise, birds-eye view. Time is better spent in the combination of these.
… : Low hanging fruit - prioirtize design review, ones with most impact have design first. Sometimes reviews are happening 6 months later, they are closed.

<Jem> can the TAG raise and frame the micro level issues to macro level of direction? We as the WG don't expect that the TAG dictates to CSS feature and ARIA. We want the clear direction if that is the part of TAG role.

Lea: I suggest a scoring system so... a whole new language is prioritized over CSS properties. Or FIFO. Big impactful changes. This seems like we can have more impact this way.
… : When suggesting a system.... it seems like devil was in the details, hard to agree
… : We should be identifying gaps in platform, where nobody is working.

<tantek> +1 lea "should be identifying gaps in the overall web platform"

Lea: : People are seeing part of that platform just for that WG. CSS, etc.
… : We should be identifying gaps & pain points at a high level
… : Ex: This thing is not possible today on the platform, someone needs to solve it.

<DKA> burgeoning work on gaps w3ctag/gaps

Lea: : Right now, no one is even identifying the pain points, they don't live anywhere
… : burgeoning work on gaps w3ctag/gaps
… : I'm hoping this gets broader engagement. This step has been made to find gaps

Lea: I think it would be useful to provide high level arch guidance. 5 minutes discussion early on can be way more valuable that an hour of telcon early on
… : Right now explainers are written without understanding how TAG reviews happen. There is wasted time making super long explainer, in practice reviewed during call time. There is no summary, tldr, skimmed &processed in the TAG call meeting time.
… : We should change guidance there.
… : I have written a lot more details in the issue linked that has been talked bout back in July

Jeff: Mark, can you describe disagreement

Mark: It's almost as if you were in those discussions 10 years ago. Reviews are really imporatn function, but the way TAG goes about it is inefficient. TAG should be looking more to do things like delegating more & oversee more, intead of using telcom time & flying across the world to do spec reviews

<Jem> I wonder what authority the TAG has over WGs.

Mark: : Not that reviews aren't important & dont' have a roll, more about the time it consumes

<hadleybeeman> Jem: we have no authority. :) We can only help.

<lea> Jem: "leading without authority" as they say in product :)

Mark: : There needs to be more of a leadership roll - guiding groups, guiding teams, thinking about charters. So much time is down in the weeds.

Jeff: Open up to general queue-ing

Dan: Response briefly. There is no argument here. I agree with Lea & Mark.

<Zakim> nigel, you wanted to ask about the approach to "triage" and how quickly "we're not interested, go ahead" responses can be generated

nigel: I want to ask something - about triage. I have submitted specs to review, and really TAG isn't interested. They are fine, they go through easily.
… : Someone could probably look & work that out in a very short period of time. But I don't think that happens. I would be interested to know if that does happen - because it can still take a while to get that kind of response.
… : Identifying gaps in web platforms but not solutions.... there is probably a role that doesn't exist. A deliberate engineer split in some organizations with rrequirements analysis and solution design.
… : What you're suggesting is identifying requirements gaps that are not meant. I think - let's look at technical design & requirements (TAG), and something else that looks at the overal requirements on the web platform
… : The separation is quite helpful, I find. Even if same people. Separation helps set the right mindset for people.
… : I present that as an option.

DKA: One thing that stops fast feedback - when we are closing a TAG review, we want to provide a resolution on that review.
… : We don't want to say resolution satisfied unless there is consensus. It can very often be difficult to get consensus. Time zones, someone is away that week, etc. That is a good goal to have, but it would change the way we need to think about TAG feedback maybe - that people don't view that it's "The TAG opinion".
… : "The TAG opinion" - we need to be careful that it's a real consensus

Hadley: Specifically on TTML - one of the reasons of this frustration
… : Most of TTML is self-contained. A few years ago, something seemed like it overlaped with CSS, so we made sure connections happened with CSS and that was resolved
… : So one of the things here was to make sure there isn't overlap with web plietform. if not, then great.
… : There is a danger in all of them that we end up being a rubber-stamper approval body
… : That is not a good use of TAG time, not something we add a lot of value. We don't ahve time to say 'we checked your work and we said you did a good job' - as we're not experts in the area

nigel: The way it's set up in process IS kind of like it's rubber stamp. The process task of a group to move forwards involves a TAG stamp.

tantek: I'm thankful of Lea's description of gettin glost in too many low-level reviews
… : I'm glad that TAG published ... principles and privacy principles were created

<Jem> all the WG is supposed to be part of horizontal review.

tantek: : Kudos for finding the time to work on high level stuff while having this incredible low level queue. Keep figuring out how to find time there
… : There needs to .... those high level docs are an essential role at the W3C. There are other levels, like TAG findings, that used to be published more. "we need to rule on this in general for people to stop making the same mistake" - that is good

<Zakim> hadleybeeman, you wanted to respond to Nigel

<Jem> +1Tantek

<Zakim> lea, you wanted to react to tantek to reply on TAG findings

Lea: To reply on tag findings, they are very heavyweight. Part of vision behind the [?] repo, it would be a lightweight way to find a finding. You could find similar findings.

tantek: I think we need both, complementary

Lea: We could .... there are old findings in the previous term (ex: findings with double key caching, urging browsers to find ways to solve.... but there was not consensus, so we coudn't progress) that aren't in the repo

<Zakim> ChrisL, you wanted to encourage bringing in content experts where the subject area is outside the expertise of the tag membership

<Jem> +1 chrisL

ChrisL: I wanted to point out that there are two things I see - one is horizontal stuff - this is about API design, you've made the same mistake others have made. This is core competence of TAG
… : The other is detailed expert-level stuff, where the TAG does not and cannot have expertise. They aren't in the position to tell. Maybe there should be more emphasis on bringing in experts
… : I've been called in to various color things. It's unusual knowledge. compression, network... you can't have in a small group expertise in all things.

<Jem> Can the TAG have the taskforce like ARIA WG?

<jyasskin> Jem: Yes, the TAG occasionally establishes task forces.

Lea: That would be wonderful. Challenge is that it's more work for people - already overbooked.

<DKA> +1 to having a pool of people...

ChrisL: I volunteer as tribute!

Yves: To reply about issue of recent reviews - TAG welcomes early reviews as well - it's a better time to do reviews and avoid mistakes, if you get review after 2 years, it can be a waste of time
… : To put that as a change to the way reviews are done in general, that could be a change
… : It would be betetr for everyone to get early feedback

Sam: +1 a bunch of what has been said

<Jem> +1 for early review request. Can TAG should communicate that broadly

<Jem> ?

Sam: High level vs lower level stuff, delegating. Design principles and other principles have been incredibly valuable

<ChrisL> Since people were asking for a TAG finding that didn't get finished - how about Draft TAG finding, 30 June 2003

<ChrisL> Separation of semantic and presentational markup, to the extent possible, is architecturally sound

goto: You would be surprised how many times these have diffused tension

<Jem> +1 delegating

<ChrisL> https://www.w3.org/2001/tag/doc/contentPresentation-26.html

goto: We send that link to other people.

goto: Early design reviews have been incredibly helpful. Much has been done that has been very helpful. I'm personally getting a lot of value. We don't get as much value with low level reviews. Part of that is the construction of the process.
… : Intersecting rubber stamping.... TAG reviews are part of Chrome process. This is unfortunate, as we're flooding you with stuff.
… : I get most value with Intent-to-prototype and early design revies
… : I would make that part mandatory, and the later part optional. Spec reviews for people that want spec feedback optional.
… : The situation we're in is due to the weight of the process. So maybe what we want is to change the process, chromium launch process, we can have influence that way.

<Zakim> lea, you wanted to react to goto

Lea: I want to +1 that - I've tried to make that point in the past. Design principles... design reviews feed into design principles.
… : Tag input in intent-to-prototype stage more valuable.
… : Sometimes we get reviews that are just a checkbox - and they say they can't change spec as they're not editors

mnot: What purpose a TAG review is?
… : If you conceptualize it as authoritative, and a barrier to progression, then it's ahigh expectation & TAG puts a high level of resources into it.
… : Chrome making it part of it's process... then external pressures.
… : I'd like to shift to true community consensus. If you think of TAG as just one part of that process, then.... it lowers the temperature of that
… : As long as you think of TAG as a golden step to go through, then that expectation is too high

<hadleybeeman> +1 to mnot on lowering expectations on the TAG review

plh: The goal - give opportunity for TAG to weigh in on spec. I didn't expect tag to do detailed review of spec. To go in with an early review - "sounds like a good/bad idea" early on.
… : That is really useful information to know early on in process
… : I'd rather have it so that if TAG doesn't respond, that's ok.
… : Tag doesn't have to give a full rubber stamp on details of specification
… : If you want a detailed review, good for you. But that's not as useful in my opinion

<Jem> +1 to ask about what communication channels the TAG is using to talk to WGs to build the community consensus

plh: : I encourage TAG to engage community in process, and engage in triaging.
… : We have to engage the community, or else we will never scale.
… : One person from the ATF said yesterday - what would be the est way to cycle specs from w3c to ATF so we can take a look?
… : The answer is TAG. The person who is working on it has said it's good enough for review - so it's a good time to review

<Zakim> mt, you wanted to point out that this idea that "we're drowning in it" is a meme, not a truth

plh: : Figuring out right time to review is hard. So wait until people ask for review from the TAG. AND encourage people to ask for review early, and not late.

mt: When you say something is a bad idea, it requires extraordinary evidence, when you say something is a good idea, people just accept that.

mt: If you say something is a bad idea and stop, then they won't take you seriously the next time

<Jem> why don't the TAG set up the process for declined or rejected proposal?

mt: : I want to push back on the idea that the TAG is drowning in design reviews. We looked at stats - some reviews have been taking quite a long time. But at the same time, those were really difficult reviews for the reasons I just mentioned.
… : (reasons restated - because 'this is bad' feedback really requires a lot of evidence and thought)
… : We have a responsibility to ensure that, if we think it's a bad idea, that message is heard. We can't force it to be listened to, which is fine.

<Zakim> mnot, you wanted to react to mt to ask MT one thing

mnot: You said you were able to meet service levels... what is the opportunity cost.

<tantek> +1 mnot: what is the opportunity cost?

<goto> yeah, that's the right framing

noamr: My feeling as someone who produces reviews for TAG is that I am drowning people in it. A definitino of a feature - one explainer per feature - is not the right way to discuss things with the TAG
… : I also feel like this async process is not the best way to have design reviews.

<goto> Yeah, the definition of "feature" is a good point. Right now, it is being defined at "what goes into an I2S".

noamr: : I feel like.... for things like core web vitals or view transitions... to have a general discussion about where we're going with these features. Some can be async or explainers, but the ticket system of features produces very .... not useful. Signal to noise ratio that isn't enough. Versus a working group.
… : Working groups could have a mandate to say what should go do TAG. Not just a new value - a new feature. So putting TAG in the process of working groups

<Zakim> DKA, you wanted to explain how we managed to do the privacy principles and ask if we should try to replicate that?

noamr: : Not just TAG cares about design of the web as a whole, we all do. But TAG has the mandate & the time to look at it from that perspective. But the other working groups can be part of this process by deciding what fits going into the tag

DKA: Tantek you mentioned privacy principles, thank you for that feedback. One of the ways we delivered that was putting together a task force, including ppl outside of tag, we picked to work on that. Then TAG came together & reach consensus.
… : Another idea is involving the community more. We have a number of ways we could involve the community in this. Maybe creating CG and delegating authority of triaging to that.

<Jem> +1 to Dan's suggestion to have more resources like community group.

DKA: : Are there other things we can be doing like privacy principles, ways to leverage community more?

<goto> On principles

<goto> A couple of documents that I think are missing are principles on Deprecations and Incubation

DKA: : Given we're an elected group too - there is specific working in the W3C process, TAG members are not supposed to delegate authority / work to other members of community. But we have a history of running task forces.
… : Another point: The TAG does not have formal authority to block things. We don't have that already. But maybe we've set ourselves up to seem like that.

Mark: Or the Chromium process has.

<Zakim> Jem, you wanted to ask about what communication channels the TAG is using to talk to WGs to build the community consensus

DKA: The chromium process can be a separate convo. but good feedback

Jem: I'm here as a customer of TAG's deliverables. I've never hard about the TAG's work until yesterday, about how TAG is giving direction.
… : How will TAG reach out to working groups to convey process & need for help.
… : I really like the comment by Mark.

<Zakim> nigel, you wanted to mention framing of elapsed time vs value provided for time spent

Jem: : I hear that maybe TAG needs a process for ... the decision tree. Rather than arguing with people that were rejected.

nigel: First - I think I've heard that there's concern about elapsed time for design reviews. A lot of response I've heard is that value is being provided by taking longer. Not just spending time because you're queued up, but doing stuff that is giving back to community

<jyasskin> ack

nigel: : Second - TAG's contract is with the wider community, but transactions are with sub-parts of the community. It's a weird tension, and not obvious that is going on. If there is feedback of frustration, it might be worth surfacing that some more.

dandclark: I'll echo that prompt review is more valuable than the same review delivered later.

<Jem> so what would be the future of TAG? ;-)

dandclark: : More likely to be taken.

<jyasskin> Jem: I think we'll have to keep discussing it.

dandclark: : Other end - on the Edge team I encourage people to submit reviews as early as possible. We need to ensure that smaller things are not blocked by large things that take forever
… : Higher level arch review - I agree that is where the TAG provides value.
… : things the WG might miss, tunnel visioned

<hadleybeeman> dmurph: these reviews I'm helping with, or I'm putting... it's a struggle to know what is the right amount of context to put. It would be interesting to have more guidance around ways to provide good context re edge cases or what needs to be described re the future. Sometimes you can't tl;dr stuff

<tantek> great discussion. Thanks TAG members and especially chair jyasskin and scribe dmurph!