See also: IRC log
<dom> Carmelo's test suite framework
Carmelo: conceived a few years ago, modified along time
... used with XSLT, XML Query
... modified a bit more to use in our effort
... not complete, looking for feedback
... Basically, it defines an XML file that describe a collection of test suites
... each "test-collections" element describes a test suite
... in our case, a test-collection element could point to the CSS MP test suite
... each test-collection can have one or many "test-group"(s), with a URL, a title and a description
... within each test group, one or more test-case that uniquely identifies a test case, with a name, a scenario (e.g. negative test, error triggering test, comparison test, visual test)
... a description, a spec-citation refering back to the spec
... possibly input/output files depending on the test
... (cf the example at the end of the document)
... Looking for feedback
... This would allow us to track a comprehensive list of tests
... There is a schema associated with that format that allows validation
... and there is an XSLT to render it in HTML
<dom> ACTION: Carmelo to send his multi-test suite cataloguing system to the mailing list [recorded in http://www.w3.org/2007/02/06-mwts-minutes.html#action06] [DONE]
<dom> ACTION: Carmelo to send a pointer to how his framework is used in XML Query test suite [recorded in http://www.w3.org/2007/03/06-mwts-minutes.html#action02]
Dmitri: is this XML format extensible to contain link to results coming out of test execution?
Carmelo: the XML file would link to test output files
Dmitri: we may need more extensibility to record more data (e.g. on which devices it applies)
... but looks good for now
<dom> ACTION: Allen to explore the possibility to have a wrapper for the test suites [recorded in http://www.w3.org/2007/02/27-mwts-minutes.html#action09] [ONGOING]
Allen proposed and described a possible schema to navigate the test suite
based on Tests ID's
<dom> ACTION: Dom to work on prototype of test navigation/results recording based on Allen's proposal [recorded in http://www.w3.org/2007/03/06-mwts-minutes.html#action04]
<dom> ACTION: Dom to try and get in touch with user agent developers (OpenWave and Nokia) [recorded in http://www.w3.org/2007/03/06-mwts-minutes.html#action05]
<dom> ACTION: Dom to run DOM test suites on his phones [recorded in http://www.w3.org/2007/02/06-mwts-minutes.html#action02] [DONE]
Dom had very little success with DOM
<dom> Getting the DOM Test suite to work on mobile devices
<dom> ACTION: Carmelo to look into transforming the XML into javascript files for making them easier to run on mobile devices [recorded in http://www.w3.org/2007/03/06-mwts-minutes.html#action07]
Carmelo explained a bit how the DOM test suite is organized (interms of XML instances)
<dom> ACTION: Till to get feedback from his Opera colleagues on making the Javascript more usable on mobile devices [recorded in http://www.w3.org/2007/03/06-mwts-minutes.html#action08]
<dom> ACTION: till to review the WCSS test suite [recorded in http://www.w3.org/2007/02/27-mwts-minutes.html#action01] [DONE]
<dom> WCSS test suite survey from Till
<Zakim> dom, you wanted to ask for a comparison with the CSS Mobile Profile test suite
Till - Explained analysis of CSS Test Suite. He ran on his phone
and found it very well structeure and well put together.
High quality test suite.
Till: WCSS test suite quite better than CSS Mobile Profile test suite; coverage is 98% (vs ~80% for CSS MP) and of high quality
... relies less on font tricks or colors
... I think we should focus on the WCSS test suite - political matters aside
<dom> ACTION: Carmelo to take a closer look at CSS for coverage [recorded in http://www.w3.org/2007/02/06-mwts-minutes.html#action07] [ONGOING]
Carmelo: will send notes I took on this
<dom> ACTION: Dom to get in touch with the CSS WG on cooperating on the test suite [recorded in http://www.w3.org/2007/02/27-mwts-minutes.html#action07] [DONE]
<dom> Cooperating on improving the CSS MP Test Suite?
dom: how should we proceed, given Till's analysis?
Till: I think we're interested in both
... I don't think we should forget CSS MP, but maybe not put too much effort in it
Dom: more to the point, would you be willing to put some of your time in improving this test suite?
Till: would be willing to, but it's not clear this is the way to go
... maybe we should survey the group on what is the proper way to go
Dom: maybe bind that to the feedback we get from other ua dev?
<dom> ACTION: Dom to ask for feedback on the mailing list about WCSS vs CSS MP Test Suite? [recorded in http://www.w3.org/2007/03/06-mwts-minutes.html#action12]
<dom> ACTION: Dom to get feedback on the open test suite on authoring practices from content developers [recorded in http://www.w3.org/2007/02/06-mwts-minutes.html#action08] [DONE]
<dom> Announcement of survey
<dom> On-going results of the survey
Dom: Please help advertize the survey as widely as possible; it will stay open until March 16
Dom: beware of change of time for Europeans (one hour earlier)
... Also, I'll likely be away on March 20.