See also: IRC log
<timeless> scribe: timeless
<Ralph> Session proposal
<Ralph> +RalphS
SteveZ: this session is because
there were a lot of concerns about what wide-review means in
the Process 2014 document
... so I wanted to describe the rationale between Process 2005
and Process 2014
... and b) suggest things that could be done to achieve Wide
Review
... and c) answer questions / issues
... no slides for this
... in decisions that went into Process 2014 (revision of
Chapter 7)
... a key thing was to enable groups to begin work earlier in
their process than implied in Process 2005
... over the years, we've discovered that groups begin getting
reviews/testcases/implementations earlier in their
process
... are more effective at getting out of the process
... one of the stages was Last Call
... which unfortunately had two things attached
... when it was created it was "The WG thinks it has completed
its work, Object now or forever hold your peace"
... when the Patent Policy was created, it needed an anchor
point
... they says "since, LC is when the group finished its work",
"we'll attach the Patent process to that"
... but, it turns out the group wasn't finished there
... and secondly, WGs weren't serious about LC = Done
... we'd get a series of LCs
... there's an objection that the series of LCs
... you could do LC->CR->LC->CR
... because patent consideration was tied to LC
... we've transferred the Patent Commitment to CR
... it's less likely to cycle
... and if you cycle, you cycle in CR
... this had a bad side-effect
... it removed an opportunity to say
... "speak now or forever hold your peace"
... before i go down that route
... the other thing we did in terms of wide-review
... reviews at LC was way too late in many circumstances
... another thing, was to drop LC as the review step
... saying "review when things are stable"
... review incrementally as things go along
... review continuously
... changing from "met requirement of making an
announcement"
... to "met requirement if people have reviewed it"
[ 7.2.3.1 Wide Review ]
SteveZ: so instead of a check for
publishing a request for review
... you have to show reviews
... one thing was Disposition of Comments (DoC)
... if all comments came from Implementers, it probably wasn't
Wide Review
... there's an emphasis to reach out early
... some groups have an IG (Interest Group) list
... and send things out there
... show by getting comments from outside, that they've met the
requirement
... so, change from checkbox, to actual substantive
response
nigel: going through this, the
W3C has a Liaison list
... in addition to the Charter dependencies, I also went to the
Liaison list
... perhaps it would be beneficial to update that list to
include Liaisons in the process
SteveZ: you're doing Timed
Text
... clearly an area where groups outside W3C have a documented
interest
... other things where it would be less useful
... because the technology is more internal
nigel: but then there wouldn't be anything in the liaisons page, so it wouldn't occur
[ nigel will create an action in #w3process to open an issue to add liaisons to 7.2.3.1 ]
barry: Pete and I are IETF application areas
<Ralph> [Ralph departs]
<nigel> trackbot: Created ACTION-37 - Open an issue to add liaisons to what's considered in the wide review paragraph 7.2.3.1 [on Steve Zilles - due 2014-11-05].
<trackbot> Sorry, nigel, I don't understand 'trackbot: Created ACTION-37 - Open an issue to add liaisons to what's considered in the wide review paragraph 7.2.3.1 [on Steve Zilles - due 2014-11-05].'. Please refer to <http://www.w3.org/2005/06/tracker/irc> for help.
barry: i was attracted by
this
... we too tried to solicit it
... people are often trying to do a semi-final review
... but resist multiple reviews
SteveZ: the process we did before was largely based on IETF process at the time this was written
barry: we thought as well
SteveZ: what are we trying to
do?
... one is dependencies
... adding liaisons is relevant there
... we're trying to advise people to, if you have a dependency,
go talk to those people and work out a schedule for
review
... some are dependent to horizontal WGs
... who are way overloaded w/ work
... so you want to give them time when it's stable enough to
review
... but not before it's concrete -- that it can't be
changed
... LC isn't that, because that's you think it's complete, what
did we miss
... this is a goal, i don't how it will work out
... it's an extra burden on the Chairs
barry: on the Chairs, and the reviewers
SteveZ: presumably the reviewers want to review, because they don't want their ox gored
barry: we're trying on a
case-by-case basis, you as a chair can ask for a review from a
particular group for review
... it doesn't scale
... it's only seldom requested, there's a hope
... we're only getting 15% success
... it's hard to get people to review early/intermediate
versions
... we're stumped
SteveZ: i'm not offering a
panacea
... mentioning a discussion, the other group may come back and
say "here's what's critical to us"
... so you know what might trigger them, to be more successful
in getting reviews
... In our WG, we'll get a Review request, and the answer is
no
... unless you can get something likely to be in the document
that's likely to be a problem, no one wants to look at it
... hopefully conversations may lead to better results, than
just throwing it over the wall
barry: agreed
SteveZ: WG members may be active
and doing this 24 hours a day
... other reviewers aren't
... you're lucky to get one review, you need to make it
count
... when you clearly have an identified dependency, you can
reach out
... the other thing suggested
... fantasai had an action item to propose
... something to help w/ the review process
... we've observed that the development of a standard doesn't
progress at a uniform rate
... some pieces (often core) progress faster than others
... it would be useful to mark sections of the document w/
stability
... CSS developed a system called Shepherd
... that scans through our documents, we have markup, and
ToC
... and our test-database is driven off those IDs
... we use IDs, section numbers may change
... IDs don't change as much
... we can automatically generate a version of the document
indicating which tests have passed in which browsers
... it wouldn't take too much effort to do this, if we could
indicate stability
... to automatically generate section warnings
... and if changes are tied to ids, we could include links to
change fields
... for people reviewing it more than once, they can see which
sections have changed recently
... since they last saw it
... and therefore see what they need to re-review
... trying to make the documents reviewer friendly is another
piece
... we don't have that technology in place
... there's no funding
... it's a couple of CSS WG members working on it
<nigel> +1 to making it easier for people to answer the question "how stable is this [section of this] document?"
SteveZ: we have it for testing
level
... we can in principle extend it
barry: are all W3C drafts developed with the same markup
timeless: there are 2 that are used
barry: that helps
... we have many different
SteveZ: i understand you have an effort to try to standardize
barry: yes
SteveZ: there's a wiki, has most
of the information that I said
... except why the process changed
... in the previous session, nigel pointed out it'd be useful
to say what the goals of wide review are
barry: in IETF, groups will write
a document, and not know who their dependencies are
... they'll stick something in, and not realize that they've
created a dependency
... getting it wrong, they'll break things
... getting them to know that when they say URI/UTF,
... that they need to consult
SteveZ: we have more process in
forming a Group
... we say a Charter needs to list dependencies
... Horizontal WGs need to review charters
... doesn't really address your problem
... because Charters are approved by AC members as a
whole
... there's a Vote
... there's a better chance that the AC member looks at a
Charter to see if a reviewer thinks there's a relation
barry: we have a similar
process
... but someone says "this string is a character encoded in
UTF-8"
... all of a sudden it has comparison
... but it wasn't in the Charter
pete: often in a routing protocol
barry: we have this problem, you have this problem, we're both trying to solve it
SteveZ: no, we're both doing experiments to try to improve this
nigel: you talked about typical duration for review
[ Working Groups should announce to other W3C Working Groups as well as the general public, ]
[ especially those affected by this specification, a proposal to enter Candidate Recommendation (for example in approximately four weeks). ]
nigel: when you talk to industry
bodies
... they might not meet very often
... and their secretarial skills aren't great
... they might sit down in six months
... it probably conflicts with a good practice
... to state in the document when the end of review period
is
SteveZ: i think that review end is required
nigel: reviewers will see the
document after the review ends
... and not know what to do
SteveZ: for a Liaison, you should have set this up
timeless: is there a stopword lbarry, ist that you could write a bot for?
barry: for some, yes
<SteveZ> consider: https://www.w3.org/wiki/DocumentReview
timeless: types of reviews
barry: if we flag "this mentions
UTF8", that's easier to get someone to review
... since they only have to look for UTF8 in the document
pete: security has been good to
have a directorate
... they farm out documents to people
barry: that works at LC
... the area has asked about early reviews
... and Directorate has said NO
pete: because they'd get double their work
SteveZ: i discussed that w/
Accessibility
... and they said it creates as many problems as it
solves
... this document (DocumentReview)
barry: we could have written
that
... we can get more than one shot for a particular
document
... but if we try for lots of documents, it would collapse
SteveZ: if the reviewing group
indicates the kinds of concerns the chair should be aware
of
... he may go ask for advice earlier if he sees those red flags
being raised
... i think opening dialogue will make this work better
nigel: back to time-frames
... how do you prevent that from happening
... the other question for the WG
... what is the status for comments after Review closes
... you got a bunch of comments that came during and you got
out of LC
... then you get more comments
SteveZ: there's an official
answer and an unofficial answer
... there's a risk of a DoS
... practically, if the problems are real, the WG should want
to try to fix them
... if someone feels they're being unfairly treated, there's an
appeals process
nigel: worst-case scenario
... comments come @ CR
... you decide to go to PR
... you have comments w/o DoC
... it'd be clear if you should have a DoC on all, or all
SteveZ: typically the DoC would be postponed to a future version
nigel: so there's a requirement on the WG to make a decision (postponed) on the comment
SteveZ: there's a
default-decision "came after comment deadline"
... you could record that w/o the WG making a decision
... good practice would be that you should note the
comment
... and the person noting should respond saying "i think the WG
should postpone, do you agree?"
... we've had cases where a group was totally overloaded
barry: we have who are people who aren't totally abusive, but they keep drawing the discussion out beyond where it makes sense
SteveZ: these are all judgement calls, you don't want to make rules that don't make them judgement
mdjp: github comments
... we have lots of issues, they're commented on outside
W3C
... that's continuous
SteveZ: if you have a way of
quickly summarizing that, i think that would be perfectly
adequate
... there's no requirement for an official wide review
... if you can provide evidence
... i believe you've established there's been wide review
... it's important to show people beyond implementers
... and the key groups are in that set
mdjp: you can identify logs where
you have gaps
... and you can target those groups
barry: now one who's expert in that has reviewed it
SteveZ: automating is great
... we're not trying to make work
... we're trying to make sure the right eyes have seen it
barry: a lot of this is having
Chairs do the right thing
... and training the chairs
... what do you do in the line of Chair Training?
SteveZ: not much/enough
... this is the only SDO i know of where an appointment doesn't
involve Chair training
barry: there's Chair training
they sometimes do the Sunday before the meeting
... and there's a Wednesday lunch for Chair training
SteveZ: w3c has recently
established a number of Chair Training Phone Calls
... for topics relevant to Chairs
... Tools
... Human Element
... How to be Effective
<nigel> http://www.w3.org/2014/Talks/chairs-part4/#/
mdjp: As a new chair, these sessions would have been really useful before the F2F
<nigel> http://www.w3.org/2014/06/17-chairing-minutes.html
mdjp: without the crossover to the outgoing chair, it would have been really difficult to run the meetings
<nigel> http://www.w3.org/2014/04/24-chairing-minutes
<nigel> nigel: There are others too - I'm not sure if they're collated onto a single page for ease of reference
SteveZ: no rules, and talk to the people you need to talk to
nigel: you mentioned if all
reviews are from implementers, that's not enough, it's not
wide
... but evidence of implementation can be evidence of
review
... since you can't implement without reading
SteveZ: the way our process
is
... you weren't going to get anywhere unless there were
implementations anyway
... but it isn't the reviewers we're concerned about
... it's dealing with the communities that could be disrupted
or disenfranchized if this becomes a standard
<nigel> timeless: I've dealt with implementers who see problems and ignore them
<nigel> fantasai: +1
<nigel> timeless: They pick an answer silently, without notifying anyone. They do /something/ but nobody realises it's not what was intended in the spec until much later.
<nigel> ... Most implementers aren't noisy like that.
<inserted> scribe: nigel
fantasai: An implementer who says something would be counted for wide review evidence
timeless: there are implementers
that will just jump over problems without reporting them so
implementations are not total proof of review
... When browser devs implement something what actually matters
is usage in websites.
... Website devs are even worse than browser devs!
... You have to go and look at website code to see if they've
put nasty comments in about your spec!
... It's a nasty thing to go and hunt those comments
down.
... We've begged for years for this kind of feedback, and we're
lucky to get feedback from 5 sites, out of billions!
... A way to harvest those comments would be beneficial.
... A way to demonstrate searches on stackoverflow, etc would
be good.
... Conversely if people don't comment on the likely sites e.g.
stackoverflow, then that may be evidence that there isn't a
problem
<SteveZ> timeless: reviewing the comments in code on key websites and/or on stack overflow will show evidence of wide review
<timeless> scribe: timeless
nigel: W3C generally wants to
make
... everyone wants to include "the general public" in "wide
review"
... the best way to make things visible
... is "the w3c home page"
... if you put things in /TR/, you can ask staff to put things
on the w3 home page
... it's been proposed to have a Blog or something for
announcing reviews
SteveZ: the DocumentReview wiki
documents making reviews happen
... it's asking the W3C to make a mailing list
... there's an issue of what the name should be
... i think that will happen
... it's not last-call
... the intent is that you identify what you want reviewed in
the Status section
... because it may not be the whole document, perhaps just a
recent change
nigel: some organizations have an
"Official Journal"
... it's widely known to the world
... a clear statement from W3 that this is the place
... your XXXs should be looking there
... boring, but effective
SteveZ: the issue of publicizing this list is known to Team
<Zakim> timeless, you wanted to talk about flagging content
<Zakim> joanie, you wanted to suggest Twitter and hashtag that stateholders are likely to see. (Not a joke)
joanie: i did this by ranting,
and i got a lot of replies
... if your stakeholders are web developers, and spit out a
single tweet with a link to the document and a few appropriate
hashtags
... a couple of stakeholders will retweet
SteveZ: we should consider using all channels that are available
joanie: i've used Twitter to
complain about web developers doing things
... and then developers reply explaining why they did it
SteveZ: two other things in
consideration
... besides the ML
... one is a dashboard page listing which active reviews
... and secondly a Calendar
... listing reviews closing
... I think, if we could automate it
... different views of the same information
... you have an announcement, it shows up in the right
places
fantasai: CSS WG
... W3 will announce it
... we'll announce it on our blog
... we'll send it to places
... we'll tweet a link
... we make our editors write the announcement, because we want
them to explain the key points
... otherwise, W3 staff does boilerplate
... there's people out there, you want their feedback, try to
get it
SteveZ: developing an Interest
community is a valuable way to get comments from others than
implementers
... not the public exactly, but people who are following
... for CSS, it's www-style
... but we can get answers in the 50s
... when we ask
fantasai: i think we'd benefit
from a less technical channel
... lower traffic
... people have wanted to participate, but don't want to
subscribe to the ML
... it's high traffic
... high signal to noise
SteveZ: we tag messages
... you can sort, i consciously sort
... i don't do animation
fantasai: we ask people to tag w/ the spec shortname
SteveZ: i wish to thank our scribe, immensely
barry: indeed
SteveZ: and i'd like to thank
everyone for participating
... i'm trying to do the process, not as written, but how it
is/should be in practice...
... a very different question
[ Adjourned ]
[ Lunch ]
This is scribe.perl Revision: 1.138 of Date: 2013-04-25 13:59:11 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: RRSAgent_Text_Format (score 1.00) Succeeded: s/why the process changed/what the goals of wide review are/ Succeeded: s/is/barry, is/ Succeeded: s/after/during and you got out of LC/ Succeeded: s/it/them/ Succeeded: s/before/the Sunday before/ Succeeded: s/fantasi/fantasai/ Succeeded: s/SteveZ2/SteveZ/G Succeeded: i/fantasai: An implementer/scribe: nigel Succeeded: s/scribeNick: nigel// Succeeded: s/<SteveZ> Timeless:/<nigel> timeless:/ Found Scribe: timeless Inferring ScribeNick: timeless Found Scribe: nigel Inferring ScribeNick: nigel Found Scribe: timeless Inferring ScribeNick: timeless Scribes: timeless, nigel ScribeNicks: timeless, nigel WARNING: No "Topic:" lines found. Present: nigel mdjp Joanmarie_Diggs Pete_Resnick Barry_Leiba Got date from IRC log name: 29 Oct 2014 Guessing minutes URL: http://www.w3.org/2014/10/29-widereview-minutes.html People with action items: WARNING: No "Topic: ..." lines found! Resulting HTML may have an empty (invalid) <ol>...</ol>. Explanation: "Topic: ..." lines are used to indicate the start of new discussion topics or agenda items, such as: <dbooth> Topic: Review of Amy's report[End of scribe.perl diagnostic output]