W3C

LOTP Architecture

This is the LOTP (Layered Object Transport Protocol) Architecture document. It describes the design goals, structure, and functionality of LOTP. This includes interrelations between the Core, Transfer Adapters, and modules. Please send non-editorial comments to xml-dist-app (archives).

Status of this Document

This document is a work in progress, not endorsed by the W3C membership. It is a living document. While, some effort will be made to maintain the anchors from version to version, it is impossible to guarantee anchor or information persistence over the life of this document.

LOTP is still in experimental stages. It may evolved into an deployed generalized XML protocol or it may be used as a learning experience for designing another protocol. It is drawn largely from SOAP and I'd like to see the LOTP advantages ported back to SOAP. I use a separate name and namespace for the LOTP protocol to prevent collisions with the current SOAP protocol.

This document may evolve into a fully normative document and is therefor intended to be as short as possible (except for examples). To this end, all discussion points are designated with one of supporting arguments, question or deprecating arguments and are expanded in the Discussions document.

Table Of Contents

Goals

Standard message format
messages with routing and other information that may be processed by generic agents and libraries.
Common serialization format
convenient way to store arbitrary data structures in LOTP messages.
Common object vocabulary and locations
common data elements with overloadable content. supporting arguments
Procedure call structure
standard ways to represent procedure calls, parameters, results, and exceptions.

XML Tools

LOTP uses standard XML tools and data formats wherever possible. Use of these tools helps provide interroperability with other XML applications and probably faster code deployment through reuse of standard XML utility libraries.

XML Schema

XML schema provides a standard way to describe data primitives and structures.

Schema Extensions

LOTP defines additional user derived types to provide a common vocabulary for LOTP agents. These types include:

Message
storage unit for the LOTP payload
Exception
any error condition that the underlying protocol should know occured
Deferment
any delayed response that the underlying protocol should know will be finished later
Originator
source of a LOTP message
Recipient
intended target of a LOTP message
Routing
preferred routing and log of actual routing
Security
authentication and encryption information
Transaction
ACID-related message identifiers
Connection question
immediacy of message transfer

Use of these types enhances application convergance and promotes interoperability. supporting arguments

Interface Discovery

Including or publishing a dataset's schema provides strong type checking for application developers. Tools may help developers by reading a schema and generating stubs for reading and writing data in a particular schema. Remote procedure mechanisms may use schemas and a dictionary of conversion mechanisms to automaticly convert data to a form that a backend service expects.

Minimal Wire Traffic

While embedded XML schema markup can make a LOTP message available for interface discovery, it will necessarily bloat the message. In scenarios where two parties are specificly written to communicate with eachother, this bloat has little value. Removing this schema information to a separate document allows compatilibility with LOTP while keeping the wire traffic to the minimum required to demarcate and identify the parameters.

A minimal message consists of a Message demarcation and nested data structures. This example has a single data structure that requests and RPC-like service:

<LOTP:Message xmlns:LOTP="http://.../LOTP/v1/">
   <m:GetLastTradePrice xmlns:m="Some-Namespace-URI">
      <m:symbol>DIS</m:symbol>
   </m:GetLastTradePrice>
</LOTP:Message>

LOTP is designed to be able to use XML schema for type checking and interface discovery. Many LOTP agents will not require a schema, realtime access to the schema, or a DTD for simple message processing. Application designers wishing to make their LOTP messages accessible to other naive applications should include portions of the schema information that will allow applications to interpret the as richly as possible. This will be explained later on in this document.

XML Namespaces

The Namespaces in XML document describes a mechanism to fully qualify tag names and attributes. This prevents name collisions and provides a name to match when looking for a schema for a LOTP message. While there is not yet any sitemap protocol defining what happens when you dereference a namespace, the namespace itself may be used as a sort of decentralized FPI. The namespace for this version of LOTP is @@http://www.w3.org/namespaces/LOTP/v1#@@. All namespaces under @@http://www.w3.org/namespaces/LOTP/v@@ are reserved for future versions so that LOTP agents may know when to respond with a version mismatch exception.

Vocabulary Reuse

New schemas may be based on pieces of other schemas. This promotes the organic evolution of a set of standard ontology. While this ontology will be continuously eveloving and likely never reach anything resembling perfection, it still provides a usefull function in unifying some of the vocabulary used in LOTP and other XML utilities.

Version changes can be treated identically to overloaded schemas; change the namespace for all the vocabulary that you don't want to be available to the earlier parser.

Extensibility

Namespaces, combined with directives stating what to do with unknown namespaces, provide an extensibility mechansim. These directives tell the core what to do if it encounters tags and a namespace that it knows nothing about. At the least, this specifies required or optional. A transparent mode allows a processor to dive into data that was wrapped in an envelope that the processor doesn't need to understand to process the data. An example is an optional signature envelope. The data inside may not be critical, or it may have been transmitted over a secure line (discussed more fully in transfer adapters). Foreign directives are allowed in the required flag to extend the extensibility mechanism. question

LOTP:requiredkill the root node
LOTP:optionalignore the element
LOTP:transparentdive into the element
my-ext:my-directiveapplication should treat as LOTP:required unless it is a known enum

Any tag that does not come with a requirement directive and is not supported by the agent must be interpreted as ext:required. supporting arguments

Structure

The fundament of the LOTP is a one way message. This message is identified and demarcated by a <LOTP:Envelope> or <LOTP:Message> tag. Either of these tags may appear at the root of a document, in a flat-file, message queue, mail message, or embedded in some larger document. While error messages are defined, the backchannel is not defined. supporting arguments The three pre-defined LOTP messages classes are:

Message
fundamental unit of LOTP-transportable information.
Exception subclassof Message
a class of error codes analogous to exceptions in java, C++ etc.
Deferment subclassof Message
any message that give no useful information back to the caller other than the fact that more information will come later.

By deriving error classes from Exception and Deferment, application designers may leverage off a LOTP agent's mechanisms for dealing with deferred or erroneous processing.

Multiple Inheritance

A Message or message class may be of more than one type at once by including all of the elements from the subclassed Messages. supporting arguments For instance, an application may wish to return multiple types of document descriptors in response to a request for a particular document. Given to common classes for document descriptors, DublicCoreDocument and ALADocument:

@@@

The appication designer may elect to return a message that will support both of these classes:

@@@

Note: the common tags were not repeated. In cases where identical tags have conflicting purpose in the base classes, multiple inheritance may not be used. supporting arguments

@@more work is required to figure out the interaction between namespace requirement directives and multiple inheritance@@

Modularity

Because the LOTP core provides only serialization and extensibility, the real application support comes from the extensions. Extension functionality may be identified by finding functionality common to multiple applications. That functionality may then be examined and separated into modules of orthogonal functionality. Proper modularization can break a complex process down into simple, recombinable modules.

Suggested Modules

While module evolution should be organic, standardization can get a head start by defining a set of suggested modules:

An example application doing a secure object-oriented update to a database would need security (Authentication), transactions, and object persistence. The modules would, in turn, require others:
start
These extensions are identified by namespace and may be overloaded or replaced in any LOTP message. supporting arguments

Minimal Implementation

A LOTP agent may implement all of the suggested modules, or it may be only a subset required to implement a particular application. This permits fast rollout while allowing for future expansion and maintaining complatibility with other LOTP agents.

Maximal Code Reuse

Applications my rely on libraries supporting module functionality. The code to implement an application may be very small once the proper components are assembled.

Transfer Adapters

LOTP is designed to opperate over many protocols, including HTTP, SMPT, FTP, and flat-files. The different protocols support different amounts of addressing and routing. The transfer adapter attempts to provide a homogeneous routing/security layer to the LOTP handlers. The Transfer Adapter document has more details on this.

Serialization

Goals

serialize all programming structures
create and object model that is flexible enough to represent and initialize all programming structures.
schema and DTD independence
represent all programming structures without requiring an application to read a schema or DTD to regenerate the structures. supporting arguments

Example 1

The serialization syntax will be explained with references to the following example:

<LOTP:Envelope>
   <LOTP:Header>
      <LOTP:Originator>eric@w3.org</LOTP:Originator>
      <LOTP:Recipient>http://www.danesupplies.com/orders</LOTP:Recipient>
      <LOTP:Security<dsig:stuff LOTP:href="#theBody">...</dsig:stuff>
      </LOTP:Security>
   </LOTP:Header>
   <LOTP:Body dsig:ID="theBody">
      <order:Gimme LOTP:type="RPC">
         <order:item>
            <order:id about="http://www.danesupplies.com/items/213"/>
            <order:description>Gameldansk 750 mL bottle</order:description>
            <order:price>$13.28</order:price>
         </order:item>
         <order:item>
            <order:id about="http://www.danesupplies.com/items/73"/>
            <order:description>Ibuprofin 40 tablet bottle</order:description>
            <order:price>$3.28</order:price>
         </order:item>
      </order:Gimme>
   </LOTP:Body>
</LOTP:Envelope>

Identifiers

There are three types of LOTP URI identifiers:

LOTP:ID
an XML ID (aka XML schema ID). This identifier is provided to permit LOTP agents a schema-independent way to identify the nodes in the graphs of transported object models.
LOTP:href
a pointer to an ID, either in the same document, or in another. If the ID is within the current document, a fragment identifier is sufficient to uniquely identify the object. LOTP:hrefs are used to uniquely identify XML nodes, ie for XSLT, but are not automatically followed as are objectHrefs.
LOTP:objectID
an XML ID (aka XML schema ID). This identifier is distinct from the LOTP:ID in that it is used to identify the object specified within a node rather than the node itself.
LOTP:objectHref
a pointer to an objectID, either in the same document, or in another. If the objectID is within the current document, a fragment identifier is sufficient to uniquely identify the object. objectHrefs are automatically followed by the LOTP agent when reconstructing an object that is either pointed to, or nested within, another object. objectHrefs are not to be used in XSLT transforms as any transform on the XML may detach the instantiation of the object from the node that originally instantiated it.

Example 2

Example 1 may be decked out with inline typing everywhere to make it easier for naive applications, proxies, gateways, and UI generators to process and data they can:

<LOTP:Envelope>
   <LOTP:Header>
      <LOTP:Originator
       xsd:type="http://www.w3.org/namespaces/LOTP/SMTP#Addr"
       >eric@w3.org</LOTP:Originator>
      <LOTP:Recipient
       xsd:type="http://www.w3.org/namespaces/LOTP/HTTP#URI"
       >http://www.danesupplies.com/orders</LOTP:Recipient>
      <LOTP:Security
       xsd:type="http://www.w3.org/namespaces/dsig#@@@">
         <dsig:stuff LOTP:href="#theBody">...</dsig:stuff>
      </LOTP:Security>
   </LOTP:Header>
   <LOTP:Body dsig:ID="theBody">
      <order:Gimme LOTP:type="RPC"
       xsd:type="http://www.ecomstuff.org/namespaces/order#BasicOrder">
         <order:item
          xsd:type="http://www.ecomstuff.org/namespaces/order#OrderItem">
            <order:id
             xsd:type="http://www.ecomstuff.org/namespaces/order#ItemId"
             about="http://www.danesupplies.com/items/213"/>
            <order:description
             xsd:type="http://www.ecomstuff.org/namespaces/order#ItemDescription"
             >Gameldansk 750 mL bottle</order:description>
            <order:price
             xsd:type="http://www.w3.org/namespaces/currency#USDollars"
             >$13.28</order:price>
         </order:item>
         <order:item
          xsd:type="http://www.ecomstuff.org/namespaces/order#OrderItem">
            <order:id
             xsd:type="http://www.ecomstuff.org/namespaces/order#ItemId"
             about="http://www.danesupplies.com/items/73"/>
            <order:description
             xsd:type="http://www.ecomstuff.org/namespaces/order#ItemDescription"
             >Ibuprofin 40 tablet bottle</order:description>
            <order:price
             xsd:type="http://www.w3.org/namespaces/currency#USDollars"
             >$3.28</order:price>
         </order:item>
      </order:Gimme>
   </LOTP:Body>
</LOTP:Envelope>

Example 3

Alternatively, an appliation designer may wish to establish communications with the bare minimum of transmitted information:

<LOTP:Body>
   <order:Gimme LOTP:type="RPC">
      <order:item>
         <order:id about="http://www.danesupplies.com/items/213"/>
         <order:price>$13.28</order:price>
      </order:item>
      <order:item>
         <order:id about="http://www.danesupplies.com/items/73"/>
         <order:price>$3.28</order:price>
      </order:item>
   </order:Gimme>
</LOTP:Body>

Arrays

Arrays are specified as types and their elements are untyped except by their immediate nesting tags. deprecating arguments An array of arrays of strings like:

char * ArrayOfArrayOfString[][3] = {{"r1c1", 
				     "r1c2", 
				     "r1c3"}, 
				    {"r1c1", 
				     "r1c1"}};

will be serialized in LOTP like:

<!-- note LOTP:type substituting for non-existent xsd:type -->
<ArrayOfArrayOfString xsd:type="LOTP:Array" LOTP:ArraySize="2">
   <ArrayOfString xsd:type="LOTP:Array" LOTP:ArraySize="3">
      <string>r1c1</string>
      <string>r1c2</string>
      <string>r1c3</string>
   </ArrayOfString>
   <ArrayOfString xsd:type="LOTP:Array" LOTP:ArraySize="2"/>
      <string>r2c1</string>
      <string>r2c2</string>
   </ArrayOfString>
</ArrayOfArrayOfString>

supporting arguments

Object References

For example, consider moving a doubly linked list from one agent to another. Given a link entry like:

typedef struct QueueEntry_s QueueEntry_t;
struct  QueueEntry_s {
    QueueEntry_t * last;
    QueueEntry_t * next;
    char * stuff;
};

LOTP can serialized the data intact:

<LOTP:Body>
   <data:transfer>
      <data:QueueEntry LOTP:objectID="QE0">
         <data:QueuePointer LOTP:objectID="last" LOTP:value="NULL">
         <data:QueuePointer LOTP:objectID="next" LOTP:pointer="#QE1">
         <data:QueuePointer LOTP:objectID="stuff">first entry</data:QueuePointer>
      </data:QueueEntry>
      <data:QueueEntry LOTP:objectID="QE1">
         <data:QueuePointer LOTP:objectID="last" LOTP:pointer="#QE0">
         <data:QueuePointer LOTP:objectID="next" LOTP:value="NULL">
         <data:QueuePointer LOTP:objectID="stuff">second entry</data:QueuePointer>
      </data:QueueEntry>
   </data:transfer>
</LOTP:Body>

Deserializing

Wait for the close tag before allocating memory for a class. In cases where an object reference turns out to be part of a larger object, a realloc and copy will have to be performed. Pointers to yet undeclared objects will require late binding.

Interoperability

LOTP attempts to have a generic enough object model to encompass other XML protocols. This promotes simple interoperability where messages may be transformed from one protocols to another, going beyond mere opaque wrapping.

Transforming Gateways

Most XML protocols can easily be programmatically tranformed to and from LOTP.

Transforming Stylesheets

In cases where elements of another protocol are isolated in sepparate and identifiable elements, an XSLT transform may be applied to transform back and forth between another protocol and LOTP. supporting arguments

Conformance

There are two arenas for LOTP conformance, the agent and the application message.

Conformant Agent

A LOTP-conformant agent provides a generic transfer adapter to give access to a LOTP core and whatever apps and modules the agent is running. Furthurmore, the conformant agent must handle all namespace directives and any error/status codes must be subclasses of the defined set. A conformant agent must handle foreign types subclassed by xmlschema directives. supporting arguments

Conformant Message

A LOTP-conformant message is one that may be read and processed correctly by a LOTP-conformant agent. This level of conformance is of great value in providing a target for developers looking to solve a simple problem in a standard way. Agents using only message conformance are strongly encouraged to treat discard all messages with unknown namespaces.


Valid HTML 4.0! Eric Prud'hommeaux,
@(#) $Id: 31-LOTP-Architecture.html,v 1.18 2000/04/20 00:49:48 eric Exp $