Abstract

This document is a strawman proposal for an abstract model for XP.

The challenge of crafting a protocol specification is to create a description of behaviour that is not tied to any particular approach to implementation. There is a need to abstract away from some the messy implementation details of buffer management, data representation and specific APIs. However, in order to describe the behaviour of a protocol one has to establish a set of (useful) concepts that can be used in that description. An abstract model is one way to establish a consistent set of concepts. An abstract model is a tool for the description of complex behaviour – it is not a template for an implementation... although it should not stray so far away from reality that it is impossible to recognise how the required behaviours would be implemented.

Status of this Document

This document is a strawman proposal for an abstract model for XP. It is by no means complete or polished and at this stage the authors just like to get some sense as to whether there is an consensus that such a model is a useful contribution to the XP effort.

This document is not a W3C Technical Report. It is to be considered work in progress and does not imply endorsement by the W3C membership nor by the XML Protocol WG. It is currently under review and discussion by the XML Protocol WG.

Discussion of this document will take place on the public <xml-dist-app@w3.org> mailing list (Archives) as indicated by the email communiation rules provided in the WG charter.

Table of Contents

Issues

1. Intermediaries: XP processors above as clients of XP layer or integral to XP layer.
   • If integral to layer, path model and operation semantics have to extend across intermediaries.
   • If clients, then path model and operation semantics across intermediaries can be deferred for further study.
2. Best Effort, Single Hop, Unordered and possibly with Duplicates, one-way and two-way request/response operations... Is that it for the core?
3. Do we need BOTH one-way and request/response as separate operations. I think so, if we want the messaging layer to look after request/response correlation. The use of a separate operation serves as a trigger to the binding to include headers to do the correlation if the underlying protocol has no intrinsic mechanism of its own that enable correlation.

@@@

1.      Abstract Model for XP

An abstract model is a useful means to develop a description of a system. It abstracts away from practical details such as specific API definitions, data representation, and buffer management. It provides a way to give a precise description of the externally visible behaviour without being prescriptive of implementation architecture.

2.      Outline (Abstract) Architecture

The diagram below exposes the various elements in an XP system:

Figure 2.1 XP System Architecture

XP processor: A client/user of the messaging services provided by the XP Layer. XP Processors can act in the role of sending/initiating XP processor, receiving/responding XP processor, or in the role of an Intermediary XP processor.

Intermediary XP processor: An XP processor that terminates one XP messaging operation and initiates one or more subsequent XP messaging operations as a direct consequence of the first operation. The messages forwarded by intermediaries are derived from the messages they receive. The transformations intermediaries perform on the messages that pass through them are defined by the XP modules they support. For example, an explicit 'XP Message Routing Module' may define syntactic structures and processing rules for those structures to facilitate the source routing of messages through a predetermined chain of intermediaries; an 'XP authentication module' might provide syntactic structures and processing rules for the inclusion and checking of digital certificates and signatures - such a module would be expected to leverage the work of other groups (eg XMLDSIG).

This draft positions intermediaries as XP Processors (or clients of the XP Layer) rather than as entities that 'live' within the XP layer. Some are not comfortable with this positioning [see http://lists.w3.org/Archives/Public/xml-dist-app/2001Feb/0007.html]

XP layer entity: XP layer entities are the entities that provide the XP messaging service to XP processors that use the XP layer. XP layer entities are responsible for enforcing the XP rules of procedure that govern the exchange of XP messages and for the binding of the XP layer to underlying communications protocols.

Application Message: An application message is application specific component of an XP message. It consists of application header and body parts that are opaque to the XP messaging layer in the sense that their structure and semantics are unknown to the XP messaging layer.

XP Message: An XP message is the unit of communication between peer XP layer entities. In the abstract it is a structure that represents the contents of the XP envelope and plus any Application Message components (whether carried within the envelope or externally in-line'd within the binding specific 'message wrapper' cf ebXML and  SOAP with attachments). An XP message may differ from an application message through the inclusion, processing and removal, of XP message headers by XP layer entities to facilitate the operation of the XP protocol - for example, operation IDs headers may be added by layer entities to correlate request/response pairs, sequence numbers may be added to support duplicate prior to delivery from a layer entity to an XP processor. 

In degenerate form, it may be the case that an XP Message and an Application message are semantically and even syntactically equivalent, however we will not know this until we have completed the design process.

XP Service Access Point is a conceptual point at which XP services are accessible. The address of an XP service access point is denoted by a URI. Since XP may be used asymmetrically, the address of the sending/initiating XP client  may be unavailable, hence the initiating/sending entity may not be explicitly addressable by a remote or responding XP client.

I have tried, and I think largely succeeded to get a much better alignment between the terms in the glossary of the requirements document and the terms used here. I find the glossary narrative and diagrams for the terms "XP processor", "XP Sender" and "XP Receiver" inconsitent see [http://lists.w3.org/Archives/Public/xml-dist-app/2001Feb/0004.html]. I have  taken and used "XP Processor" to be a user of the XP layer that processes the application semantics of  delivered message, equivalent to an "XP Client". I have taken "XP Sender" and "XP Receiver"  to be providers of message sending and receiving services to "XP Processors" which jointly correspond "XP layer entity" a term that I have retained...

This has caused me to introduce a distinction between what an application (or XP Processor) exchanges with its peer and what the provider of the XP messaging service exchanges with its peer. I believe that this an important distinction not well captured with the simple notion of an "XP message" . I have introduced the term "application message" which is intended to encapsulate application headers and bodies. There is some discussion as to whether header/body distinction is meaningful. I'd certainly be in favour of the notion of multiple application body parts all of which might be thought of as XP blocks. The header/body distinction then becomes a distinction between what gets generated/processed by an XP Layer Entity and what gets generated/processed by the application/XP processor. Such a view returns us to a comfortable and conventional view of header/body distinction.

Alternates for Application Message might be Application Payload, Body(ies), Payload... other suggestions?

3.      Strawman XP Messaging Service Definition

This section focuses on the definition of the abstract messaging services provided at the XP layer interface. Two operations are modelled: a one-way XML datagram operation (XP_UnitData); and a two-way XML request/response operation (XP_Data). Both are assumed to be best effort  and capable of re-ordering and duplicating messages.

We may be able to relax these, particularly message duplication - the mechanisms by which this may occur depend heavily on the underlying protocols and on any mechanisms introduced into the XP layer through modules intended to improved the the reliabliity of delivery.

• XP_Data – A two-way request/response message exchange model by four primitives (or events) corresponding to the transmission and reception of the request and response messages.
  

• XP_UnitData – A one-way  datagram service modelled by three primitives (or events) corresponding to the start and end of transmission (from the senders point-of-view) and the reception of an XP message
  
  A response message may arise, however that is a matter of application semantics, to be dealt with by the XP Processor (and above) through the future invocation of XP_UnitData and/or XP_Data operations in the reverse direction.

Replaced uni-directional and bi-directional with one-way and two-way in response to  http://lists.w3.org/Archives/Public/xml-dist-app/2001Jan/0090.html item #3

The XP Messaging Service is a single hop service, so in-general the delivery semantics of the service apply only to the direct exchanged of messages between peer XP processors. Where an XP processor acts as an XP Intermediary, it may extend and fork the message delivery path over multiple hops. However, delivery semantics in relation to messages delivered through an chain of intermediaries and the semantics in relation to messages that have been partially processed through a chain of intermediaries remain a topic for further study as our thinking on Intermediary XP processors matures. Sec

Figure 3.1 Single Hop XP Messaging Service

3.1.        Operation Primitive Patterns

Operation primitives are abstract events that cross the boundaries between protocol layers. They are a descriptive device that allow us to abstract away from practical implementation details such as buffer management and specific APIs and invocation mechanisms such as upcalls (callbacks), downcalls, queing and polling mechanisms.

It is important to note that operation primitives are an abstraction. From a descriptive point-of-view they become events and actions that drive and are driven by the procedural rules of the protocol.

.request        Used by initiating or sending XP processors to start an operation. Typically used to start the sending of a message.

.indication    Used at the receiving or responding XP Processor to indicate the arrival of a message on its 'outward' journey.

.response    In two-way services, used by the responding XP Processor  to return a response or fault toward the initiating XP Processor.

.confirm        Used at the initiating XP Processor to indicate the completion of an operation started by the corresponding .request primitive.

In the case of a two-way operation (XP_Data) it may also carry a response or fault message on its 'return' journey toward the initiating XP Processor . 

Ray Dennenberg  [seehttp://lists.w3.org/Archives/Member/w3c-xml-protocol-wg/2001Jan/0100.html]  suggested renaming .request and .indication primitives as .send and .receive for use with the UnitData operation. At present I have not done this. I did not want to end up with 6 types of primitives instead of 4. If the group would prefer this change, I'd be inclined to split this subsection into two and embedded each half into the description of the relevant operation. ie. Figure 3.2 and the description of .request, .indication, .response and .confirm would move to be with the rest of the description of XP_DATA, and Figure 3.3 and .send, .receive and .confirm (again) would move to be with the rest of the description of XP_UnitData.

Alternatively, .request and .indication could be relabelled as .send and .receive for both Data and UnitData operations - however, .response and .confirm would probably need to be relabelled to feel comfortable with .send and .receive. Any suggestions?

These four primitives are used in two common patterns to model the behaviour of two-way request/response operations and one way datagram operations as illustrated in the diagrams below.

Figure 3.2  XP_Data Operation: Two-way request/response operation

Figure 3.3  XP_UnitData Operation: One-way datagram.

From the initiators point-of-view both of types of operation start with the invocation of a .request primitive and terminate with the invocation of a .confirm primitive.

A subtle difference is that in the two-way case the .confirm primitive arises only after a .indication has been raised at the responding client, whereas in the one-way case it merely indicates that the operation is complete from the initiating end's point-of-view. In the one-way case a (silent) failure may prevent a message from being delivered as an .indication to the responding client, even though a .confirm is delivered back to the initiating client.

3.2.        XP_Data Operation

XP_Data is an unreliable two-way request/response operation which is modelled by four primitives. Each primitive models a transmission or reception event at interface between an  XP layer entity and an XP Processor:

XP_Data.request( To, [{ModuleList}], Body, [{AttachmentList}],[BindingContext]);

XP_Data.indication( To, From, [{ModuleList}], Body, [{AttachmentList}, [BindingContext]]);

XP_Data.response( [{ModuleList}], [Body|FaultBody], [{AttachmentList}], [BindingContext]]);

XP_Data.confirm( From, [{ModuleList}], [Body|FaultBody], Status, [{AttachmentList}], [BindingContext]);

All parameters are detailed in Section 3.4

I have removed path related parameters from these primitives, consistent with the positioning of intermediaries as XP Processors above the layer.

The operation is unreliable since it can fail silently through the loss of messages in transit. Immediate failures that prevent the invocation of a primitive are reported as the result of the attempted invocation.

An XP processor may engage in multiple concurrent operations with the same or different recipients/paths. These concurrent operations are independent and the order in which they are processed by the recipient may be different from the order in which they are invoked or complete at the initiator. 

Failures that arise during message processing at XP intermediaries or at the intended recipient result in the inclusion of a FaultBody rather than a ResponseBody in the .response and .confirm primitives.

This operation may be implemented over HTTP, HTTPS, SSL/TCP, TCP and SMTP.

3.3.        XP_UnitData Operation

XP_UnitData is an unreliable one-way datagram operation  which is modelled by two service primitives that model the transmission and reception of request message as follows:

XP_UnitData.request(To, [{ModuleList}], Body, [{AttachmentList}], [BindingContext]);

XP_UnitData.indication(To, From, [{ModuleList}], Body, [{AttachmentList]}, [BindingContext]);

XP_UnitData.confirm( [From], Status, [BindingContext]);

All parameters are detailed in Section 3.4

I have removed path related parameters from these primitives, consistent with the positioning of intermediaries as XP Processors above the layer.

The operation is unreliable since it can fail silently through the loss of messages in transit.

In general there are no ordering guarantees with respect to the delivery of messages. Transport intermediaries and recipient processing may re-order messages. Thus, the order in which concurrent XP_UnitData operations complete is independent of the order in which they were initiated.  In applications where sequencing is important an optional XP module may be used to establish the original ordering within XP layer clients.

The one-way and unreliable nature of the interaction mean that failures during recipient XP processing should result in a message sent in a one-way message carrying a FaultBody in the reverse direction. It is also important that a failure to deliver a fault message does not give rise to further fault messages.

This service may be implemented over HTTP, HTTPS, SSL/TCP, TCP, SMTP and UDP.

3.4.        Common Parameters in XP Operation Primitives

This section describes the (abstract) parameters that are used in the XP_Data and XP_Unitdata operation  primitives. These parameters are abstract in the sense that no attempt is made to relate them to any concrete representation of values. They would typically relate to the parameters of an API (if the an API were closely modelled on this service definition), however, they are given only a narrative description here.

[The service primitives and particularly their parameters are likely to evolve. In an abstract sense with the exeception of BindingContext and Status, the remaining parameters are an abstraction of the message exchanged between XP clients. With a suitable definition of an XP message abstraction most of these parameters could be collapsed into fields within that abstraction. Particular bindings would be free to 'lift' fields (like To) from an outbound message and re-express them in lower-level headers or indeed leave them in place and still use them for the formation of lower-layer headers. Likewise on inbound messages, bindings could synthesise message fields from lower-layer headers if they are not present in within the XML payload. An advantage of expressing attachments as a separate parameter is that their inclusion within the XML envelope or another wrapper (eg. MIME multipart) is deferred to the binding...there will be issues related to the formation of a manifest and internal references (URIs) for and to attachments].

There is discussion of how useful the distinction between header and body is [see http://lists.w3.org/Archives/Public/xml-dist-app/2001Jan/0110.html] Application Header and Bodies may become absorbed within a single construct. This document will evolve to accomodate whatever final view is taken.

The has been considerable discussion of the handling of binary attachments. There are multiple views about whether XP should be capable of carrying multiple (application) attachments or payloads. The view taken here is that the XP messaging service will intrinsically be able to handle multiple arbitrary payloads and one such payload may be marked out as special - ie a body to which things are attached - or not, ie. just a collection of related payloads. Again this section will evolve with this debade.

4.      Operation through an Intermediary XP Processor

An intermediary XP Processor is an XP processor that processes XP messages and forwards derived messages on to further XP Processors. The message path through intermediaries may be allowed to fork (cf. SMTP). In the case of two-way operation it is then the responsibility of the intermediary that introduced the fork to coordinate the formation of the response that is returned toward the initiating XP processor.

The processing behaviour of an intermediary XP processor is similar to that of an ordinary XP processor. Inbound messages (.indication or .confirm primitives) are subject to XP Processing and XP Module Processing. Depending on the function of the intermediary suitably derived XP message body and modulelist parameters are formed and a revised message sent on the next leg of its journey. For the two way XP_DATA service, there are two possible interaction patterns plus hybrid derivatives.

·        Fully Interlocked: The .request propagates fully through to the responding client which generates a response that propagates fully back to the initiating client

·        Hop-by-Hop: where each intermediary responds locally and immediately to its neighbour. Note that with this pattern the response received by the initiating client does not originate from the responding client.

Figure 4.1 Interlocked XP_DATA operation through Intermediaries

Figure 4.2 Hop-by-Hop XP_DATA operation through Intermediaries

[Personally, I don't think this hop-by-hop pattern really works. It's here to illustrate the difference and someone else may be able to articulate circimstances where it would be of value... I am at a loss!]

The XP_UNITDATA service operating through intermediaries follows a similar pattern. However, the .confirm primitives arise through the operation of the underlying protocols between neighbouring XP layer entities.

Figure 5.2 XP_UNITDATA operation through Intermediaries

5.      XP Message Processing and Extensibility (XP Modules)

Broadly there are two types of XP Modules. Those that encapsulate some application semantic, which are processed by XP processors above the XP messaging service layer, and those that enhance the delivery properties (QoS?) of the XP messaging service itself, eg. through the addition of mechanisms to improve the reliability of message delivery or to filter out dupicate messages prior to delivery to an XP processor.

5.1    XP Protocol Extensibility

The functionality of the  XP layer may be extended through the introduction of XP Modules that improve the delivery semantics of the XP messaging service without substantially changing the operation of the interface to the XP layer (XP_Data and XP_UnitData operations). For example,  such XP modules might improve the reliability of a message exchange through the introduction of message id's and the use of timeouts and retransmissions to reduce the probability of message loss - and to filter out duplicate messages in the event that multiple copies of the same message are delivered to the receiving XP Layer entity. Each such module can be viewed as introducing a 'micro' protocol into the XP layer. 

 Figure 5.1 XP Layer Entity Processing

The XP modules shown in figure 4.1 are purely illustrative. The intent is to suggest that there may be some precedence order to the module processing and that within a given precedence level ordering may insignificant - ie. modules within a given precedence level are orthogonal.

Probably want to think about this a little more. If we retain bestEffort semantics... ie. some modules put more effort in... but there is still no guarantee of delivery, it is likely that there will be no natural order for processing these modules. eg. should one authenticate a message before processing say its reliable delivery headers. In a Byzantine world I guess you would, because someone may be trying to force some mechanism to misoperate. 

5.2    Application Message Processing

Application specific processing of an XP Message occurs within XP Processors (initiating, responding and intermediary). XP Messages are delivered to XP clients in .indication and .confirm service primitives. Application processing of an XP message may then give rise to the either a response, in the case of an XP_Data.indication, or the generation of further .request primitives as governed by the application protocol operating between peer XP clients. Such application protocols are outside the scope of the XP activity.

I have retained the term 'XP Message' here although I am strongly tempted to change it to Application Message - although I would like a better term than that - XP Payload might be better - other thoughts and comments?

In the case of intermediary XP processors, the response returned toward the initiator may be dependent on the outcome of processing at downstream  XP Processors and at the responding client.

The diagram below illustrates the components of an XP client within this abstract model:

Figure 5.2 Model of XP Message Application Processing

The XP Processor is the principle application component. It is responsible for the application specific processing of XP messages in accordance with the intended application semantics of the message body, attachments and any optionally included module blocks. The processing of XP module blocks in accordance with the rules of procedure associated with a particular XP module is handed off to a module specific XP Module Processors.

The model of the interface between XP Module processors and an XP Processor is deliberately vague. From an XP module specifcation point-of-view, the important thing to specify with respect to an XP module are:

• the syntax and semantics of the associated XP module blocks

• the rules of procedure with respect to the generation of XP module blocks either in response to the receipt of an XP module block in an inbound message or at an initiating XP client.

[We will need to decide whether the order of module processing is semantically significant. We should have an architectural rule about this. It is probably easier and safer to assert that module processing order is semantically significant and that it is determined by the order in which module blocks arise in XP messages. XP clients initiating the transfer of an XP message may exert an influence of the module processing order by: targetting module blocks at specific intermediaries; by controlling the ordering of module blocks within the message]

6.      Layer Entities and Protocol Bindings

Bearing in mind that this is an abstract model, one of the principal roles for an XP layer entity is to provide an abstraction for a binding to an underlying protocol. In this context it is a container for the abstract rules that combine parameters taken from .indication and .response primitives (abstract) to form the outbound message and headers for the underlying protocol and vice-versa in the case of inbound messages and the .indication and .confirm primitives.

[I'd like to do a more detailed treatment of the SOAP HTTP bindings to show how using this form of model it become a non-issue whether the URI of the intended recipient is carried within the SOAP envelope or whether it is carried within the HTTP headers – the binding will dictated the mapping between underlying protocol headers, underlying protocol payload and XP service parameters. Likewise the derivation of the SOAPAction header can be explained. Also, the treatment of attachments would also be useful. I believe that we have to define a default way to carry binary attachments within the XP message envelope – possibly base64 and/or an escape mechanism for XML/text. Thus the support for attachments becomes intrinsic, however the use of more efficient underlying mechanisms for direct handling of binary become part of the binding. I'd also suggest that the construction of manifest, concrete attachment URIs and concrete attachment references be undertaken by the binding (or at least exposed for manipulation by the binding)].

7.      Modelling SOAP

[This section at present is just a place holder – I'd like to do a much more thorough job]

8.      References

[1]        "Simple Object Access Protocol (SOAP) v1.1"

[2]        "Axis Architecture Overview"
http://www.soap-wrc.com/axis/axis_arch_overview_v03.zip