See also: IRC log
<MikeSmith> http://www.w3.org/2011/05/testing-ig-charter.html
MS: did several changes since we last talked
... talked with Philippe about Task Forces
... worked well in several settings
<MikeSmith> example: http://www.w3.org/2004/04/wcag-charter
shadi: where would you envision Task Forces? the core scope seems pretty broad anyway
<Zakim> MikeSmith, you wanted to say that there are many parts of the shared testing infrastructure that are common to testing of different classes of apps
mikeS: did not want to limit the authority of
creating TFs
... to create reusable infrastructure
... for instance like a commenting funcationality that is not limited to a
particular class of applications
... distguishing that from the test runner for the browser, which is specific
to particular types of applications
shadi: think that makes sense but may need to have better understanding of the individual components, interfaces, and terminology for these
<francois> +1 to "test runner"
philippe: next would be to send the IG Charter to
the AC, so that I can talk with the AC folks next week
... try to get review period as soon as possible
philippe: previously looked at existing work
... have a "test runner" for HTML5
... also one from Mozilla that seems to be more complete
... for instance, it can display two documents for comparison, which the HTML5
one cannot do
... also know there are other "runners" elsewhere
... Dom developed one for mobileWeb and there is another one for CSS
somewhere
... so question is do we start by building on top of an existing runner, and
if so which
... have a call on that today at noon ET
... trade-off between metadata and how they will be added
... would be good to learn how the CSS folks managed 9k+ tests
... to have ready documentation next time someone wants to contribute tests
francois: already reviewed some of the CSS
testing work
... some documents we should be familiar with before the call
philippe: purely informational call
<plh-home> http://wiki.csswg.org/test
wilhelm: should also look into how the different
browser vendors use the tests
... the importing is really painful
... partially manual and very messy
... would like to know how the other browser vendors handle this
... and the IG could facilitate this process
... also good to look at automation
philippe: automation is an important aspect
... not easy to automate browsers across platforms
... but could provide the necessary data
... maybe could automate such tests in the future but not in the near
future
... also need to educate people on how to write automatable testsd
s/testd/tests
scribe: as to facilitating the use of test suites
for browser vendors
... also a goal but not yet clear how
... if we do our job well, we can harmonize the techniques
shadi: all sounds like metadata is necessary to provide the necessary interface
philippe: metadata is bigger than conventions
... different groups are more specific about their assertions
... even create their specs accordingly
... others are much less involved
... we have w3c-test.org but difficult to get an overview
philippe: now called "test runner"
... need first version by summer
... question is which "test runner" is to be first
shadi: thought this group needs to develop more
than the "test runner" alone but also the infrastructure around it
... like test repository, test metadata, and other parts of what we talked
about
philippe: yes, the group needs to develop the
other pieces
... but need to have the runner ready by mid-summer
... need to develop the pieces along the way
shadi: need a description of the different components and interfaces
philippe: yes, groups should be able to create
their own test runners
... would like to avoid forking as much as possible
... but may be unavoidable in specific situations
francois: we sometimes have to bundle test cases
different in some cases
... does not mean we need cannot have a common repository
... just a matter of repackaging
philippe: need to develop those diagrams