This is a draft, detailed outline for a process document. It presents ideas rather than complete, well-constructed sentences and paragraphs. It captures the ideas presented in the first OpsGL draft , Karl’s earlier draft (http://lists.w3.org/Archives/Public/www-qa-wg/2002Jun/0078.html), as well as things that have been mentioned in email and at meetings.
Broken into 3 major sections.
Note: There is no implied order to the rambling text below
- Any WG member can request an expert be invited
- Non-W3C members can petition for membership by contacting the WG chair
- All invited experts must provide their credentials and state their ability to meet the WG commitment (attend meetings, 4-8 Hours/ week)
- Who approves acceptance of the expert – the WG votes or the chair decides?
- An expert can be invited to participate in discussions and meetings on a specific topic.
- There are no membership commitment requirements.
- Any WG member can request the invitation and must present the credentials of the expert.
- The chair does the inviting, after the WG approves.
- Topic experts do not need to follow WG participation rules. @@@ should they be limited in their participation – e.g., not invited to attend all meetings.
- A quorum consists of having at least 1/3 the members present.
- Decisions can only be made if a quorum is present; else the decision is considered a preferred action or direction and not closed. This applies to at least the following:
--- all votes, (note that the voting process is described in the Charter)
--- resolution of issues,
--- changes in the WG’s operation or process
- When an action item or assignment results in an unexpected change or modification to fulfill the assignment, it must be called to the attention of the WG.
- Announce and if necessary, present an overview of the change at a telecon. Not only does this ensure that the WG is aware, but it is also documented in the minutes.
- Examples: Reformatting the OpsGL to look like the SpecGL. Writing an issue resolution from a general agreement of what needed to be done.
- When does something get publicized to the IG
- All deliverables get announced to the IG. When?
Upon completion, email to AI editor with AI #
- Issues pertain to the Framework Guidelines (intro, ops, spec, test) and their companion ExTech documents, process, and any other WG deliverables
- @@@ Do all comments on a document become issues? - Issues are derived from comments received in email and/or from meeting discussions.
- All comments on documents must be acknowledged, reviewed and if warranted, made into an issue, resolved, and commenter informed. This includes comments from within the WG, IG, and public.
-Originator
– either WG member or external party Sends in a comment or inquiry that results in an issue
-Chair (document editor?)
– receiver of the issue. Starts the process, puts issue discussion on agenda, and assigns ownership
- Owner
– member of the WG who is responsible for the issue (may not be the originator) - Acts as liaison, if submitted by external party. If needed, formats the issue, presents it for discussion, contacts the originator regarding disposition
- Issues Editor
– owns the Issues List
- The telecon following the receipt of a comment or inquiry is the start of the issue tracking. The chair (or document editor) initiates the issue processing.
- The WG determines if this really is an issue, and if so, the Issues Editor enters it into the Issues List. If the issue isn’t in the proper format, the task of formatting the issue can be delegated.
- Each issue is assigned an owner, who is responsible for ensuring the issue is processed. If the comment is made from a member of the WG, then that member becomes the owner, else the chair or editor assigns an owner. For issues received from non-WG members, the owner acts as liaison between the external party and the WG. The owner sends an acknowledgement to the external party regarding receipt of the comment and provides the issue number and url of the issues list if appropriate, and upon resolution, informs the external party.
- Issues are categorized and prioritized. The chair or document editor decides the priority and requests time on a meeting agenda for discussion. Issues can be discussed via email but resolved at meetings (telcon or f2f). It may be necessary to continue discussions beyond a single meeting. It may be necessary to assign an action item to a WG member to get more information and/or discuss the issue with the Team contact.
- When there is consensus on the outcome of the issue, a resolution is written and recorded in the Issues List. The Issues Editor can either write the resolution or delegate it. The issue is marked as closed.
- The issue owner informs the originator that the issue has been resolved. This is an important step, especially when the originator is not a WG member.
- The WG is informed of updates to the Issues List by either announcing this at a meeting so it appears in the minutes or sending email to the QAWG mailing list. Periodically, email is sent to the IG list to inform them of updates to the Issues List
-Once an issue is closed, the chair or editor rules on requests to reopen or continue arguments on closed issues (remember issues can only be closed if there was a quorum present when closed). There must be new evidence uncovered or altered situation in order to reopen
Potential relationships with other W3C WGs include: (taken from the OpsGL)
--- Liaison and consultation,
--- reviews,
--- adjudicate entry and exit criteria,
--- QA resources supplement,
--- Resolution of external QA requests
-There are different types of communiqués - some more formal than others. Communication with QAWG may be received as email to the chair, email to the WG mail list, IG mail list, or from a verbal request.
- All questions or inquiries must be answered.
- Any WG member can answer, however chair or chair’s designee is responsible for making sure the WG responds with some type of acknowledgement or answer.
An important function of QAWG is to review W3C work with QA issues in mind. This is done as early as possible, at the charter stage, during the document process, and during the development of conformance materials. The QAWG when resources are available will review and provide guidance to WGs.
- Requests can be handled on a case by case basis – with the chair bringing the request to the attention of the QAWG. Alternatively, the QAWG can appoint a team to handle all incoming requests.
- All requests are put on the agenda for the next meeting, where they are introduced and preliminary action is taken.
- Chair solicits interest. A subgroup is formed and a leader is appointed. A subgroup can consist of 1 person. If there are no volunteers, the chair can appoint someone.
- If there is no one available that can do the review in the allotted time frame and/or this is outside the expertise of those who are available, the chair responds that the QAWG doesn’t have the resources or expertise to perform the review.
- If a subgroup is formed, the group makes comments. These comments are submitted to the QAWG for review and approval. When approved, the comments are sent as QAWG comments. The QAWG can waive review and allow the subgroup to send the comments directly as QAWG comments. If time doesn’t permit a QAWG review, then the comments can be sent as an individual comment.
- Review of technical documents and test materials includes verifying that the checkpoints of the appropriate QA Framework documents are met and providing guidance to the requestors on how to meet the checkpoints.
- QAWG is a source of information – people come to for guidance and tools.
- May get requests to help build conformance materials, including drafting documents or conformance sections and developing test suites.
- Upon receipt of request, chair evaluates the request, estimating the type of resources and expertise needed, and duration of effort. Chair can appoint someone to do this. Present to QAWG, who then decides if we can do it. If yes, a person or subgroup is formed to assist. If no, the chair informs the requester.
- QAWG acts as liaison and facilitator between external organization and appropriate W3C WG. Puts the right people in contact with each other.
- May need to ‘translate’ conformance terminology and technology for the 2 groups.
- Provides lessons learned and examples from other groups that have already been through this.
- Asked to participate in or present QA to other groups and conferences. Goal = education and outreach by
-Collaborate with other groups
-present work at conferences -participate in other groups (WG, IG, etc) meetings, including workshops to establish a group.
[ed note: The following applies to the development, review, etc. of test materials corresponding to the QA Framework documents and any other deliverables with conformance implications.]
- appointed and voted on by the WG
- not have a separate mail list, use QA WG mailing list, www-qa-wg@w3.org.
- once WG consensus reached, publish to QA IG mail list, www-qa@w3.org
- have separate web page for test materials to post announcements and latest news, deliverables, releases, FAQs, how to participate, etc.
- the following applies to contributions made by QAWG members, other W3C members and WGs, as well as organizations and individuals external to W3C
- test materials, either whole or in part (e.g., test suites or individual tests) submitted via the mailing lists
- acknowledgement by the moderator, that has been received and in ‘received’ status
- IPR or license applied to submissions?
- cursory review to see that all required info and format properly; if no, send back
- at least 2 reviewers?
- review for relevance, accuracy, scope, clarity, aspect of spec begin tested, quality of code and documentation
- run each test on different implementations – shows device independence
- group reaches consensus on accept, reject, or defer decision while issue forwarded to WG for clarification
- a test can be rejected even if all the review criteria met and reviewers think it’s a good test, but there is a discrepancy with the interpretation of the Spec.
- if test accepted and decided to be stable, which means that it is relevant, that it indeed tests a particular identifiable part of the spec and that it is not wrongly written, it becomes ‘reviewed and stable’
- if test is accepted but needs clarification, mark status as ‘reviewed and pending’
- tests that at any point in the review are judged not appropriate, receive ‘inappropriate’ status. And are archived
- when ‘reviewed and stable’ tests found to be erroneous (i.e., not stable), then go back to Review step (13.2)
-each test material comes fully documented with regard to the following:
-- name of submitter, date of creations, and if appropriate company
-- suggested ID for the test
-- Part of spec being tested
-- (if appropriate) scenario which has preceded the writing of the test
- completed test materials will reside in xxx space - developmental work and published versions via CVS?
-plausible to assume that changes and revisions will need to be made – which will be noted by different version of the test suite
- Moderator responsible for versioning and proper inclusion of tests in the various versions
- any comments received on published test materials follows Issues Processing, (step 5)
- use the Document License
- no branding
- Test materials aim to help implementers write specifications and conform to the guidelines. The test materials do not provide certification. The only claim that can be made is that a particular implementation is conformant to a particular version of the test material.
Created Date: 2002-11-20 by Lynne Rosenthal