Test Suite

From WebID Wiki

WebID Tests

One you have created a WebID EARL testing service (see below), you can test it for issues by sending a number of broken and working requests.

One implementation of this can be found in the WebID mercurial repository. The README.txt file contains more information.

Different classes of implementation will require different testing. One test concept may apply to multiple implementation classes, but results may differ due to different perspective (e.g., a Generator should only emit valid certificates, no matter the inputs it receives; a Consumer should only accept valid certificates, and should give appropriate errors for each kind of invalidity).

The current list of test cases is very short, and is primarily focused on testing Generator output. If you have some ideas for further test cases add them to this list:

  • update WebID information to see how long agents rely on caches
  • create ambiguous profiles (multiple modulus' in the same key)
  • missing in WebID URI hash
  • leading zeros in modulus
  • content negotiation
  • white space in modulus
  • ...


The following segmentation should help in classifying and specifying appropriate tests.

  • Testing a Generator/Provider
    • Cert Generating Software
    • Cert Generating Services (also known as "Identity Providers")
  • Testing a Consumer
    • Cert Verifiers
      • Cert Verifying Software
      • Cert Verifying Services
    • WebID-compliant Applications (including web browsers, web servers, extensions to either, and other libraries and executables)
    • WebID-compliant Sites and Services (also known as "Relying Parties")

WebID Test Reports

Human Readable WebID Test Reports

Every WebID Server should have an endpoint that can produce human readable feedback on the success or failure of a WebID Authentication. This is very useful in helping explain to users why they failed to gain access. It should explain errors such as --

  • no WebID was found in the X.509 Certificate.
  • no useable WebID was found in the X.509 Certificate. Perhaps the server only understands http/https but not the acct scheme
  • no WebID Profile could not be found by server (network problem, missing file, etc.)
  • the public key in the profile does not match the key in the certificate
  • the WebID Profile information is wrong (e.g., two modulus for one key could lead to inconsistent behavior)

Machine Readable WebID Test Reports

The idea here is to simply make the human-readable page also machine-readable. This will then allow testing agents to be built, that can test a number of web sites and build a detailed interoperability report. It should be very useful for library writers to help them see if there are bugs in their code. And it will also be very useful to writing the spec, as it will help find cases where the test writer and library writer disagree over the interpretation of the spec.

To do this the previous machine readable test reporting page need only generate and RDF version of the report. This can be done using the EARL ontology, with a specific adaption for the WebID test suites.

EARL Test Ontology

The EARL vocabulary for publishing reports on WebID authentication attempts is available at its namespace url http://www.w3.org/2005/Incubator/webid/earl/RelyingParty#. It allows WebId endpoints to report on 20 or so tests as shown in the diagram below.

The vocabulary contains detailed descriptions of what needs to be described for each test. An example report available at RelyingPartyExample.n3 should help along.

WebID Auth Test Reporting

The above ontology can be used by RESTful Web servers to produce reports an authentication attempts by clients.

The following endpoints currently produce these EARL WebID reports:

To access the n3 version use curl on the command line, as shown below, using a PEM file containing an X509 certificate.

Note the foafssl.org server no longer produces the RDF below, as it has been rewritten and the EARL output has not yet been implemented.

$ curl -H "Accept: text/rdf+n3" -E ~/bblfish.pem https://foafssl.org/test/WebId

@prefix cert: <http://www.w3.org/ns/auth/cert#> .
@prefix log: <http://www.w3.org/2000/10/swap/log#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix test: <http://www.w3.org/2005/Incubator/webid/earl/RelyingParty#> .
@prefix earl: <http://www.w3.org/ns/earl#> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .

[]   a earl:Assertion ;     
     earl:result [ a earl:TestResult ;
                dc:description "Found one Modulus" ;
                earl:outcome earl:passed
              ] ;
      earl:subject _:b4 ;
      earl:test test:pubkeyRSAModulusFunctional .

[]    a earl:Assertion ;
      earl:result [ a earl:TestResult ;
                        dc:description "Profile was fetched. The information about when this was done is not yet very detailed in Clerezza." ;
                        earl:outcome earl:passed
              ] ;
      earl:subject <http://bblfish.net/people/henry/card#me> ;
      earl:test test:profileGet .

// [snip]

_:b1  a   cert:Certificate ;
      log:semantics [ log:includes _:b2 ] ;
      cert:principal_key _:b3 .
     cert:base64der """
MIICzzCCAjigAwIBAgIGASX2Ykm+MA0GCSqGSIb3DQEBBQUAMFQxCzAJBgNVBAYTAlVTMRcwFQYD
VQQKDA5GT0FGIENlcnRzIEluYzEWMBQGA1UECwwNMTI2MjU1OTA1NDIwODEUMBIGA1UEAwwLSGVu
cnkgU3RvcnkwHhcNMTAwMTAzMjI1MDU0WhcNMTAxMjI1MjI1MDU0WjBUMQswCQYDVQQGEwJVUzEX
MBUGA1UECgwORk9BRiBDZXJ0cyBJbmMxFjAUBgNVBAsMDTEyNjI1NTkwNTQyMDgxFDASBgNVBAMM
C0hlbnJ5IFN0b3J5MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC6ERNG91Vaxa1DeMc84Pkh
/E9N1p3OoAMLbSlOb4sTPOKYEuHL/YvM60PH2Hpgg6nx/bZ6Jn/jKsf/RkO3mI0fY77pJGQ/szxe
FoWbm2BrAkK8aekQacbpP0xKLMP7Eoh7cZDGdfzvJPEKBWafDnUNf8mSLpWLedjz4TCCESMlnwID
AQABo4GrMIGoMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgLsMBEGCWCGSAGG+EIBAQQE
AwIFoDAdBgNVHQ4EFgQUaoLsIvS+zlB5UVt8aC9EN4O6mvwwHwYDVR0jBBgwFoAUaoLsIvS+zlB5
UVt8aC9EN4O6mvwwMgYDVR0RBCswKYYnaHR0cDovL2JibGZpc2gubmV0L3Blb3BsZS9oZW5yeS9j
YXJkI21lMA0GCSqGSIb3DQEBBQUAA4GBAHg5aY9KZURf6xJJ+alMawq+FcrghA3LkssdgZEycKBw
gGqukm2C2pmUD2ByE+D33U8VpdtD4a67ZLIsnPFxjFcSLmRe+H7E4Fvo2nJH/xRNLdEDI2uVcn/p
FB6MWYlg4n+s+fNSsSfLOdINsMfzMnw4awLsMtW45Vq8yuYmOZwg
"""^^xsd:string ;
 
_:b2  a   test:WebIDClaim ;
      test:claimedIdentity "http://bblfish.net/people/henry/card#me"^^xsd:anyURI ;
      test:claimedKey _:b3 .

_:b3  a    rsa:RSAPublicKey ;
      cert:modulus  """
ba111346f7555ac5ad4378c73ce0f921fc4f4dd69dcea0030b6d294e6f8b133ce29812e1cbfd8bcceb43c7d87a6083a9f1fdb67a267fe32ac7ff4643b7988d1f63bee924643fb33c5e16859b9b606b0242bc69e91069c6e93f4c4a2cc3fb12887b7190c675fcef24f10a05669f0e750d7fc9922e958b79d8f3e130821123259f
     """^^cert:hex ;
     cert:exponent 65537 .

// [...]

Note: To translate p12 files containing a certificate created in a browser to other formats see this howto .

The code running on foafssl.org is available in the Apache Clerezza bblfish branch. This can be used to see how to build this in a different language. (That is only one, so please add the output to your implementation and especially send feedback about the EARL report.)


Some WebID Verification Sites

Further Reading

Todo

  • write/generate a human readable html version of the EARL vocabulary (using XSPARQL) preferably
  • write a more detailed step by step guide on how to describe an endpoint
  • highlight which of the tests are required and which are optional
  • simplify the tests by simplifying the spec if possible
  • keep the tests in line with the spec
  • link tests in the EARL vocabulary to parts of the spec
  • elaborate more second order tests to test the EARL endpoints.
  • generate reports on compliance of different libraries to the spec.
  • write an XSPARQL transform for EARL results so that any server can produce nice looking html just by adding that XSPARQL to their views