The Case for SOIF-2

Many wide-ranging application level protocols now under development in the networking community have a need for a compact, easy to manage encoding of a set of key/value pairs. This includes groups working in all of these areas:

• Indexing (Harvest's SOIF format, Whois++ templates, IAFA templates)
• Personal Data Interchange (Versit's PDI specification, the new application/directory MIME format)
• public key infrastructures (PGP, SPKI, PKIX and other X.509 profilers)

There is a clear need for progress towards a standard text representation with enough flexibility to handle these applications and more. Interoperability and implementation will be easier if the community can work towards a more standard template format. The first SOIF specification is a good starting point, but there is a good case to be made for SOIF-2.

Based on our experience, first with IAFA templates and more recently with the development of the Whois++ technology, we have distilled a list of issues designers of attribute/value encoding schemes need to consider.

Hard/soft newlines

The template representation must be able to distinguish between newlines existing in the attribute value and those inserted in order to facilitate transport through systems with line length limitations.

Complex structures

Some applications require either the ability to have attribute values that are themselves attribute/value pairs, or have some mechanisms for definition of relationships between attributes. In either implementation, a further consideration to be addressed is how (or whether) to specify individual subcomponents.

Template representation character set

In order to distinguish between attributes and values, a clear definition of the characters used in the template representation must be available (e.g., delimiters, ans the associated quoting rules).

Attribute value character set

The template representation must be capable of handling the character set(s) necessary to store data in a variety of languages.

Attribute metainformation

Some applications require that metainformation regarding the language, format, or other qualities of the value be carried along with each attribute/value pair.

SOIF, as it is currently defined, carries the attribute name, the attribute length (in bytes), and the attribute value. Handling of line breaks and binary attribute values is incompletely specified. Metainformation and character set issues are not addressed at all. Some of the issues outlined above are handled more completely in other template systems, (line breaks in Whois++, metainformation in Versit).

The SOIF specification is halfway to a solution to a problem shared by many applications. The question is, "should we try to join these applications by a common template format"? We believe such a standard would be a very important step forwards towards creating interoperating indexing services. SOIF-2, an extended form of the current SOIF templates, should be specified and deployed as quickly as possible. It should address the issues above in simple ways, making the SOIF-2 specification useful in a broad variety of applications.