W3C Technology and Society Domain | XKMS home

XKMS Candidate Recommendation Implementation Report

Created: 12 Jan 2005

Last revised by $Author: kahan $ on $Date: 13 April 2005 - 19:02:22$

Table Of Contents

1. Introduction

In order to move forward the XKMS specification from Candidate Recommendation to Proposed Recommendation status, the XKMS Working Group (WG) had to give enough implementation experience to satisfy the PR entrance criteria. The method adopted by the WG was to create an XKMS Assertions and Test Collection. This document ennumerate all the assertions on the XKMS specification, specify tests for these assertions or, otherwise, give rationale as to why an assertion could not be tested. The tests were grouped into four categories: X-KISS, X-KRSS, Compound, and Optional, corresponding to the two basic XKMS services, combining multiple requests, and finally, tests for optional features, respectively.

2. Modifications to Proposed Recommendation Entrance Criteria

The following Entrance Criteria were proposed in the XKMS Candidate Recommendation announcement (you need a W3C member password to browse this link):

The Working Group succesfully tested all of the HTTP Transport and Soap 1.2/1.1 bindings, As a result of the feedback received, the Security Binding sections was made more concise and clear and now lists a number of security considerations and how these considerations could be taken into account by the XKMS payload security features, the TLS ones, or by a combination of both of these. Although there were no specific tests for the payload security bindings, all of the XKMS features on which these bindings depend were included in one or more of the tests. The TLS bindings were not tested as the developers felt it was out of scope for the XKMS interoperability test; TLS security properties have already been tested and discussed in other fora and are independent from XKMS.

During the CR period, the WG further refined the two server implementation exit criteria to request that there are at least two servers that are contacted by two or more clients. Likewise, the WG added an extra criteria requesting that there be at least one client that contacts two different servers. The WG felt these refinements gave a better proof of interoperability.

3. Summary of the Interoperability Tests Results

The Working Group defined a total of 36 test scenarios (18 X-KISS, 14 X-KRSS, 1 Compound, 3 Optional). The results are summarized in Appendix A. Two clients (VM, GA) implemented all the tests. An additional client (TL) implemented reported success on all the tests, except for the Optional ones as our reporting rules didn't allow for a developer to report results against his own server. Two servers (TL, SQL Data) supported all the tests except for the Optional ones. Only one server (TL ) supported the Optional tests. Both servers were tested against two clients at least. These tests satisfy the interoperability entrance criteria to Proposed Recommendation. Appendix A gives a breakdown of the tests.

3.1 Outstanding Interoperability Issues

The Working Group received a total of 43 issues. All but three suggested changes to the XKMS specification which were accepted by the WG (see the Changelog of the XKMS specification and its bindings).

The Working GroupG declined the following three issues, the first two as being deliberately not addressed and the third one as being out of scope of the specification. All the reviewers agreed to the Working Groups responses:

3.2 Success Stories

The Environmental Information Exchange Network (EIEN)of the United States is in a process of deploying a live XKMS 2.0 service. It will go live in a couple of months. The Exchange Network is a web service network that links information systems in the state governments and federal government agencies, and allows automated and secure data exchanges between Network Node (the service endpoint). The project started about 3 years ago, currently there are 32 states participating in live data exchanges, many more are in the development and testing stage. The goal is to have all 50 states to join the Exchange Network. It is perhaps the largest web service network in the US.

The Exchange Network has a centralized security service - Network Authentication and Authorization Services (NAAS), the idea is to have a live XKMS service and move toward public key authentication with signed authentication messages, at least between Network Nodes:

  1. Users generate a pair of keys and register them in the central XKMS service (there is an approval process, off course).
  2. Users construct an Authenticate message and sign the message using a registered key, the KeyInfo element will contain a unique key name.
  3. When an authentication message is received, NAAS will validate the key through the XKMS XKISS, and verify the signature. The user is considered authenticated if both the key and authentication message are valid.

3.3 Advisory Interoperability Data

Two of the interoperability participants prepared and contributed a set of XKMS messages that were exchanged during the XKMS CR interoperability phase. These messages are to be seen as advisory, rather than normative.

Appendix A: Interoperability Results Matrix

How the table was constructed

XKMS client developers were asked to report their success or failure of running the tests described in the the XKMS Assertions and Test Collection document against a number of XKMS servers using an online questionnaire, that was open from 2004-09-14 to 2005-01-28 (you need to have a W3C member or public password in order to browse that link). The following table is a summary of the questionnaire results from the point of view of the entrance criteria. Only the clients and servers that were reported as succesful in the XKMS CR test suite report were taken into account. Some developers built both client and servers. In those cases, only the tests of those clients against other servers were taken into account.

The following information was reported directly by the developers. It does not necessarily represent the latest state of any given implementation over this or later specifications.

X-KISS

The WG defined 18 tests. These tests and their results satisfy the REQUIRED entrance criteria as stated above: at least two client and two server implementations of each feature; for each feature, at least two servers contacted by at least two different clients; there should be at least one client that contacts two different servers.

We had four server implementations, of which two (TL, SQL) implemented all the tests and were contacted by at least two different clients each. We had up to seven client implementations of which three implemented all the tests (TL, GA, VM).

test server

implementations

client

implementations

at least two servers contacted by at least two clients

at least one client contacts at least two servers comments
XKISS-T1: Locate 3

TL Server, SQL Data, ASF-XKMS

5

BL, VM, YZ, TL, GA

yes yes
XKISS-T2: Validate 4

TL Server, Entrust, SQL Data, ASK-XKMS

7 yes yes
XKISS-T3: Locate - not found 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
XKISS-T4: Validate an expired cert 4

TL Server, Entrust, SQL Data, ASF-XKMS

6

RL, BL, YZ, VM, TL, GA

yes yes
XKISS-T5: Validate a revoked cert 4

TL Server, Entrust, SQL Data, ASF-XKMS

6

RL, BL, YZ, VM, TL, GA

yes yes
XKISS-T6: Two Phase

3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
XKISS-T7: Asynchronous 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
XKISS-T8: Two Phase + Asynchronous 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
XKISS-T9: Compound 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
XKISS-T10: Two Phase Compound 3

TL Server, SQL Data, ASF-XKMS

5

BL. YZ, VM, TL, GA

yes yes BL's self-test answer ignored.
XKISS-T11: Asynchronous Compound 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKISS-T12: Compound with inner asynchronous requests 2

TL Server, SQL Data

3

VM, TL, GA

yes yes
XKISS-T13: Soap 1.1 4

TL Server, Entrust, SQL Data, ASF-XKMS

6

BL, RS, YZ, VM, TL, GA

yes yes TL: Used T2 for Entrust,, but not important as we were testing the Soap 1.1 bindings.
XKISS-T14: Soap 1.2 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
XKISS-T15: Opaque Client Data 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
XKISS-T16: Request Signature Value 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes
KISS-T17: Unsuccessful Request Signature Value 3

TL Server, SQL Data, ASF-XKMS

4

BL, VM, TL, GA

yes yes
XKISS-T18: Response Limit 3

TL Server, SQL Data, ASF-XKMS

5

BL, YZ, VM, TL, GA

yes yes

X-KRSS

The WG defined 14 tests. These tests and their results satisfy the REQUIRED entrance criteria as stated above: at least two client and two server implementations of each feature; for each feature, at least two servers contacted by at least two different clients; there should be at least one client that contacts two different servers.

We had two server implementations (TL, SQL); both of them implemented all the tests and were contacted by at least two different clients each. We had up to four client implementations of which three implemented all the tests (TL, GA, VM). The remaining client implemented all the tests except for one (XKRSS-T13).

test server

implementations

client

implementations

at least two servers contacted by at least two clients at least one client contacts at least two servers comments
XKRSS-T1: Register Client Generated Key 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T2: Register Service Generated Key 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T3: Reissue 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T4: Recover 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T5: Revoke 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T6: Revoke with shared secret 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T7: Two Phase 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T8: Asynchronous 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T9: Asynchronous + Two Phase 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T10: Compound 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T11: Two Phase Compound 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T12: Asynchronous Compound 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes yes
XKRSS-T13: Compound with inner asynchronous requests 2

TL Server, SQL Data

3 VM, TL, GA yes yes VM reported a different behavior on SQL Data, hut this is without incidence on th entrance criteria.
XKRSS-T14: Unsuccessful authorization 2

TL Server, SQL Data

4

YZ, VM, TL, GA

yes

yes

COMPOUND

The WG defined one test. This test and its result satisfy the REQUIRED entrance criteria as stated above: at least two client and two server implementations of each feature; for each feature, at least two servers contacted by at least two different clients; there should be at least one client that contacts two different servers.

Two servers (TL, SQL) and three clients (VM, TL, GA) implemented the tests.

test server

implementations

client

implementations

at least two servers contacted by at least two clients at least one client contacts at least two servers comments
Compound-T1: XKISS and XKRSS 2

TL Server, SQL Data

3 VM, TL, GA yes yes

OPTIONAL

The WG defined three tests. These tests and their results satisfy the OPTIONAL entrance criteria as stated above: at least one client and one server implementations of each feature.

One server (TL) and two clients (VM, GA) implemented all the tests.

test server

implementations

client

implementations

two client requests

to each server

a client contacts

at least two servers

comments
Optional-T1: Authentication with Private Key 1

TL Server

2 VM, GA yes no
Optional-T2: Authentication with NotBoundAuthentication 1

TL Server

2 VM, GA yes no
Optional-T3: Validate with RetrievalMethod 1

TL Server

2 VM, GA no no


Jose Kahan