Difference between revisions of "Test Suite Development"

From MultilingualWeb-LT EC Project Wiki
Jump to: navigation, search
(Implementations)
Line 1,174: Line 1,174:
  
 
=Implementations=
 
=Implementations=
 +
 +
'''See the separate [[Use_cases_-_high_level_summary| use cases high level summary]] for more details.'''
  
 
==Implementation 1==
 
==Implementation 1==

Revision as of 17:39, 5 February 2013

1 Test Suite Overview

The ITS 2.0 Test Suite is designed to support the comprehensive and consistent testing of independent implementation of the ITS 2.0 specification.

ITS conformance is specified in the ITS 2.0 specification. ITS has been designed so that conformance can be claimed by showing conformance to any one of the specified data categories, though much of the value of using ITS will accrue from using data categories in combination.

The test suite therefore provides a set of tests for each data category, which constitute a conformance test for that category. For each data category, a different feature conformance test is provided for the different features that represent the different ways in which the data category can be used. Each feature conformance test provides input file containing ITS mark-up and where appropriate a separate file with its rules, together with an output file that contains the association of ITS attributes with nodes from the input file that result from conformant ITS processing. The output files are presented in a common file format in order to facilitate the automation of checking against the output generated in tests and to ease the verification of the test suite itself and the verification of test results.

To qualify for inclusion in the final ITS 2.0 recommendation, each feature must have the results from a successful feature conformance test from two independent implementations published on the Test Suite page.

The test suite files are available here: https://github.com/finnle/ITS-2.0-Testsuite

2 Test Suite Tracking

The test suite is currently under development, tracking the development of the specification itself. The current status is temporarily hosted by TCD, but will be transferred to a more permanent hosting site once conformance testing is to be undertaken.

To develop the test suite the following steps need to be completed by the WG:

  1. A full set of feature conformance tests to be completed to reflect all functionality in the ITS specification (currently being done at TCD)
  2. Each feature conformance test to be checked for correctness by a separate member of the WG.
  3. All parties planning to attempt implementation conformance tests should provide: the name of their implementation, the name of their organization, the name of a contact person. They may also provide some descriptive text detailing the general function of the implementation and how it aims to make use of ITS.
  4. All parties planning implementation conformance test should check they understand the format of the input and output files and then populate the table below with the anticipated date to start testing against each of the feature conformance tests.

If you wish to undertake some implementation conformance testing, but are not currently a member of the MLW-LT working group then please contact the WG co-chair directly to register our interest.

The following table allow this progress to be tracked by feature conformance test.


Features

File Type

Implementor 1

Implementor 2

Implementor 3

Implementor 4

Implementor 5

Translate

           

Global - Embedded rules

XML yes

Global - Linked rules

XML yes

Global - Linked rules

HTML yes

Global - Embedded and linked rules

XML yes

Local - In host element

XML yes

Local - In host element

HTML yes

Local - In <its:span>

XML yes

Global (with //*) and local rules

XML yes

Global (with //*) and local rules

HTML yes

Global (with /doc) and local rules

XML yes

Global (with /doc) and local rules

HTML yes

Global and local rules (inheritance)

XML yes

Global and local rules (inheritance)

HTML yes

Localization Note

           

Global - Embedded rules with locNote

XML yes

Global - Linked rules with locNote

HTML yes

Global - Embedded rules with locNotePointer

XML yes

Global - Linked rules with locNotePointer

HTML yes

Global - Embedded rules with locNoteRef

XML yes

Global - Linked rules with locNoteRef

HTML yes

Global - Embedded rules with locNoteRefPointer

XML yes

Global - Linked rules with locNoteRefPointer

HTML yes

Global - Linked rules with locNote

XML yes

Global - Linked rules with locNote

HTML yes

Global - Embedded rules with mixed notes

XML yes

Global - Linked rules with mixed notes

HTML yes

Local - In host element with locNote

XML yes

Local - In host element with locNote

HTML yes

Local - In host element with mixed notes

XML yes

Local - In <its:span> with mixed notes

XML yes

Terminology

           

Global - Embedded rules with termInfoPointer

XML yes

Global - Embedded rules with termInfoRef

XML yes

Global - Linked rules with termInfoRef

HTML yes

Global - Embedded rules with termInfoRefPointer

XML yes

Global - Linked rules with termInfoPointer

XML yes

Global - Linked rules with termInfoPointer

HTML yes

Local - In host element

XML yes

Local - In host element

HTML yes

Local - In <its:span>

XML yes

Directionality

           

Global - Embedded rules

XML yes

Global - Linked rules

XML yes

Local - In host element

XML yes

Local - In <its:span>

XML yes

Ruby

           

Global - Embedded rules

XML no

Global - Linked rules

XML no

Global - Pointer rules

XML no

Local - ITS markup

XML no

Language Information

           

Global - Embedded rules

XML yes

Global - Embedded and linked rules

XML yes

Global - Linked rules

HTML yes

Elements Within Text

           

Global - Embedded rules

XML yes

Global - Linked rules

XML yes

Global - Linked rules

HTML yes

Local - In host element

XML yes

Local - In <its:span>

XML yes

Local - In host element

HTML yes

Domain (New to ITS 2.0)

           

Global - Embedded rules with domainPointer

XML yes

Global - Linked rules with domainPointer

HTML yes

Global - Embedded rules with domainMapping

XML yes

Global - Linked rules with domainMapping

HTML yes

Global - Linked rules

XML yes

Global - Embedded and linked rules

XML yes

Locale Filter (New to ITS 2.0)

           

Global - Embedded rules

XML yes

Local - In host element

XML yes

Local - In <its:span>

XML yes

Global - Linked rules

XML yes

Global - Embedded and linked rules

XML yes

Global - Linked rules

HTML yes

Local - In host element

HTML yes

External Resource (New to ITS 2.0)

           

Global - Embedded rules

XML yes

Global - Linked rules

XML yes

Global - Embedded and linked rules

XML yes

Global - Linked rules

HTML yes

Disambiguation (New to ITS 2.0)

           

Global - Embedded rules with entityTypeSourceRef

XML

Global - Embedded rules with entityTypeSourcePointer

XML

Global - Embedded rules with entityTypeSourceRefPointer

XML

Global - Embedded rules with entityTypeRef

XML

Global - Embedded rules with entityTypePointer

XML

Global - Embedded rules with entityTypeRefPointer

XML

Global - Embedded rules with disambigType

XML

Global - Embedded rules with disambigSourceRef

XML

Global - Embedded rules with disambigSourcePointer

XML

Global - Embedded rules with disambigSourceRefPointer

XML

Global - Embedded rules with disambigIdentRef

XML

Global - Embedded rules with disambigIdentPointer

XML

Global - Embedded rules with disambigIdentRefPointer

XML

Local - In host element

XML

Global - Linked rules

XML

Local - In <its:span>

XML

Local - In host

HTML

Global - Linked rules

HTML

Global - Embedded rules with entityTypesourceRef and disambigType

HTML

Global - Embedded rules with entityTypeRef

HTML

Global - Embedded rules

HTML

Global - Embedded rules with disambigSourceRef and disambigIdentRef

HTML

Global - Embedded rules with disambigType

HTML

Target Pointer (New to ITS 2.0)

           

Global - Embedded rules

XML yes

Global - Linked rules

XML yes

Global - Linked rules

HTML yes

Id Value (New to ITS 2.0)

           

Global - Embedded rules

XML yes

Global - Linked rules

XML yes

Global - Linked rules

HTML yes

Preserve Space (New to ITS 2.0)

           

Global - Embedded rules

XML yes

Global - Linked rules

XML yes

Local - In host element

XML yes

Local - In <its:span>

XML yes

Localization Quality Issue (New to ITS 2.0)

           

Localization Quality Précis (New to ITS 2.0)

           

3 Implementations

See the separate use cases high level summary for more details.

3.1 Implementation 1

  • Name of Implementation: Okapi Framework
  • Organization: Okapi Development Team
  • Contact Person: Yves Savourel (ENLASO)
  • Details: The framework is a Java open-source project available here: http://code.google.com/p/okapi/ (see also the wiki documentation).
    The detailed of the implementations per component is listed in the Okapi Implementation page.
    The ITS 2.0 implementation provides support at different levels:
    • a library that can be use to decorate an XML+ITS document and access the document tree along with the ITS information.
    • a filter for XML that use the library above to extract localizable content from XML documents.
    • a filter for HTML5 that use the library above to extract localizable content from HTML5 documents.
    • a XLIFF writer that output several ITS data categories in XLIFF.
    • a filter for XLIFF that understand the ITS data categories supported with the XLIFF writer.
    • various step that make use the some ITS data categories. For example the Quality Checker Step can flag entries that do respect the length constraints set with the Storage Size data category.

3.2 Implementation 2

  • Name of Implementation: VistaTEC Linguistic Review Tools
  • Organization: VistaTEC
  • Contact Person: Phil Ritchie
  • Details:

Through our workflow our tools aim to:

    • automatically check and annotate xliff and add localisation quality metadata,
    • use a browser client to manually (human assessment) add additional quality metadata into native HTML5,
    • automatically harvest metadata from above files and add to internal data store to form the basis of metrics and decision support.

3.3 Implementation 3

  • Name of Implementation:
  • Organization:
  • Contact Person:
  • Details:

3.4 Implementation 4

  • Name of Implementation:
  • Organization:
  • Contact Person:
  • Details: