- From: Simon Raboczi <raboczi@itee.uq.edu.au>
- Date: Tue, 2 Jan 2007 11:42:06 +1000
- To: public-rdf-dawg-comments@w3.org
I took an action [1] at the last teleconference to investigate whether the xUnit Test Driven Development (TDD) community had a standard test reporting format that might be an alternative to EARL [2] for the SPARQL tests [3]. (Posted to the comments list for the administrative reason that I'm not formally on the WG.) Although originally developed for Smalltalk, the Java version JUnit was the first version of the testing framework to break into the mainstream. In JUnit the results of running a test suite can be reported in one of two formats: raw text suitable for human inspection, or an XML format suitable for build automation tools (e.g. the continuous integration framework CruiseControl [4]). It's this XML report format which serves a similar role to EARL. JUnit's popularity saw ports to just about any language in popular use [6], giving rise to the "xUnit" family. As far as I can tell, however, JUnit's XML reporting format does not appear to have become a de-facto standard across the family. There does however appear to a modicum of interest in standardizing, particularly among smaller language communities who stand to gain access to mainstream build automation tools (e.g. Python and Lua, see below) and mixed-language projects needing to integrate test reports from different languages. David Collados Polidura has apparently created a JUnit-compatible XML reporter for CPPUnit (the C++ version of xUnit) and an XML Schema for the format [8], and Marc-Elian Bégin has done so [7] for PyUnit (the Python version). I also ran across Scott Jacobs expressing the desirability of standardizing the XML reporting for Lua's version [5]. The impression I get is that although there's a modest desire out there for a standard test reporting format, and JUnit's XML reports are the closest thing to a standard at the moment, there's neither a formal nor a strong de-facto standard established just yet. Marc-Elian Bégin (the Python guy) [7]: > > Testing of complex software systems is a tough business, especially when > you start mixing languages and test frameworks. The xUnit framework, > from the original JUnit test framework developed for Java, is the > de-facto unit testing framework in open source development. However, > the output format of the tests is not standard across the different > implementations of xUnit, which is now available for nearly all popular > languages (and also exotic ones). This makes it difficult for a build and > test system to generate an overall report of the tests. Meanwhile, Ant’s > dedicated “junit” tasks outputs a test report in a simple XML format. The > reports being XML, makes further processing (e.g. search and > transformation) simple and convenient. If we adopt this format, it would > be interesting to have other xUnit implementations to generate their > report in the same format (i.e. generate XML Schema compliant reports). Scott Jacobs (the Lua guy) [5]: > > Using the common xml format makes the results of your unit testing > available to tools like CruiseControl. For precisely that reason I did > add xunit-style xml output to a unit-testing framework that I wrote for > another language. At the time, I just used examples of output from > unit-test runs that I stumbled across via google. Since then I have come > across this link [8] purporting to be a schema for JUnit's xml output. [1] http://www.w3.org/2006/12/19-dawg-minutes.html#action10 [2] http://www.w3.org/TR/EARL10/ [3] http://www.w3.org/2001/sw/DataAccess/tests/ [4] http://cruisecontrol.sourceforge.net/ [5] http://lua-users.org/lists/lua-l/2005-07/msg00367.html [6] http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks [7] http://meb.home.cern.ch/meb/xPyUnit.htm [8] http://jra1mw.cvs.cern.ch:8180/cgi-bin/jra1mw.cgi/ org.glite.testing.unit/config/JUnitXSchema.xsd?rev=HEAD&content- type=text/vnd.viewcvs-markup
Received on Tuesday, 2 January 2007 01:42:42 UTC