Automatic test framework for media fragment servers

Dear all,

 

in the context of Erik's action-164, we are currently working on a
validation tool for Media Fragment servers, based on input from corrib.
Currently, our validation tool performs the following steps:

1. syntax validation of the Media Fragment URI (e.g., #t=banaan is a syntax
error)

2. semantic validation of the Media Fragment URI (e.g., #t=20,10 is a
semantic error)

3. if the media fragment is syntactically and semantically valid, construct
a valid Range header and perform the request

                3.1 check if the HTTP response code is right

                3.2 check the HTTP response headers (i.e.,
Content-Range-Mapping) syntactically and semantically

4. if the media fragment is syntactically or semantically invalid, construct
an invalid Range header and perform the request. Check if the server reacts
in the right way.

5. functional testing (in case of a valid media fragment)

                5.1 download the full resource

                5.2 download the resource in pieces using byte range
requests and reconstruct the full resource

                5.3 download the resource in pieces using time range
requests and reconstruct the full resource

                5.4 download the resource in pieces using track range
requests and reconstruct the full resource

                5.5 compare the retrieved resources bit-wise

6. visual testing (in case of a valid media fragment): play the media
fragment in the browser (probably using a valid MF player :-))

 

It is important to notice that we do not test Media Fragment players (i.e.,
the translation from Media Fragment URI to HTTP Range header), as mentioned
in the minutes of the last phone conf [1]; our validation tool only
validates servers.

 

Now, in order to get this working with corrib RDF output, we need the
following information out of corrib:

- input media fragment (already available)

- http response code (not explicitly available)

- visual result description (+/- available) 

- media resource (not available) -> a test case is dependent on the
underlying media resource

 

Therefore, we extended [2] with the necessary properties and created [3] to
describe the media resources used in the test cases. The description of the
first test case should then look as follows:

 

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"

  xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"

  xmlns:dcterms="http://purl.org/dc/terms/"

  xmlns:t="http://www.w3.org/2006/03/test-description#"

  xmlns:ctag="http://commontag.org/ns#"

  xmlns:mftc="http://www.w3.org/2008/WebVideo/Fragments/TC/mftc#">

 

<rdf:Description
rdf:about="http://www.w3.org/2008/WebVideo/Fragments/TestCases/v1-mediafrag/
Testd050755546ab4712eab90bb768e7ed6a">

    <rdf:type
rdf:resource="http://www.w3.org/2006/03/test-description#TestCase"/>

    <rdfs:seeAlso
rdf:resource="http://www.w3.org/2009/05/13-mediafrag-minutes.html#item05"/>

    <dcterms:title>plain fragment</dcterms:title>

    <dcterms:contributor rdf:resource="http://sw-app.org/mic.xhtml#i"/>

    <t:reviewStatus
rdf:resource="http://www.w3.org/2006/03/test-description#approved"/>

    <t:input>#</t:input>

    <t:purpose></t:purpose>

    <ctag:tagged
rdf:resource="http://www.w3.org/2008/WebVideo/Fragments/TC/mftc#UndefinedTag
"/>

    <mftc:expectedHTTPStatusCode
rdf:resource="http://www.w3.org/2008/http-statusCodes#statusCode200"/>

    <mftc:expectedVisualResults>entire
representation</mftc:expectedVisualResults>

    <mftc:mediaResource
rdf:resource="http://www.w3.org/2008/WebVideo/Fragments/TC/media#fragf2f.mp4
"/>

    <mftc:mediaResource
rdf:resource="http://www.w3.org/2008/WebVideo/Fragments/TC/media#fragf2f.ogv
"/>

</rdf:Description>

</rdf:RDF>

 

@All: what do you think of this approach?

@Michael: do you think you could extend corrib to cover this information?

 

Best regards,

 

Davy

 

[1] http://www.w3.org/2010/05/19-mediafrag-minutes.html#item03 

[2] http://www.w3.org/2008/WebVideo/Fragments/TC/mftc# 

[3] http://www.w3.org/2008/WebVideo/Fragments/TC/media# 

 

-- 

Davy Van Deursen

 

Ghent University - IBBT

Faculty of Engineering

Department of Electronics and Information Systems 

Multimedia Lab

 

Gaston Crommenlaan 8 bus 201

B-9050 Ledeberg-Ghent

Belgium

 

t: +32 9 33 14893

f: +32 9 33 14896

t secr: +32 9 33 14911

e:  <mailto:davy.vandeursen@ugent.be> davy.vandeursen@ugent.be 

URL:  <http://multimedialab.elis.ugent.be/dvdeurse>
http://multimedialab.elis.ugent.be/dvdeurse