Draft (19th November 2001)
John Ibbotson, IBM Corporation, ( firstname.lastname@example.org )
This document describes the XML Protocol Usage Scenarios and how they may be implemented using the SOAP 1.2 specification
This document is not a W3C Technical Report. It is to be considered a work in progress and does not imply endorsement by the W3C membership nor by the XML Protocol WG.
As a work in progress this document represents the current state of the Usage Scenarios. It is currently under review and discussion by the XML Protocol WG.
1. This is an initial draft document - no issues raised so far.
2. Need to add scenarios S21, DS24, S805, S807, S809 and S810 and more SOAP examples
3. Scenario S810 to be added only. Encrypted header and payload example SOAP to be added.
Introduction to the usage scenarios.
A sender wishes to send an unacknowledged message to a single receiver (e.g. send a stock price update every 15 minutes)
Figure 1 Fire-and-forget to single receiver
A “fire-and-forget” feature in scenario S1 requires a mechanism to send a message to a single SOAP Receiver and is illustrated in Figure 1. The SOAP Sender does not require any status information that the message has been sent to or received by the recipient. The underlying transport protocol may implement a response mechanism, but status on whether the message was successfully sent or otherwise is not returned to the sending SOAP Processor.
A sender wishes to send unacknowledged messages to a set of receivers (e.g. send a stock price update every 15 minutes)
Figure 2 Fire-and-forget to multiple receivers
Scenario S2 extends S1 to implement a “fire-and-forget” feature to multiple SOAP Receivers and is illustrated in Figure 2. This requires a mechanism to deliver the same message to multiple SOAP Receivers. The delivery of the messages could be implemented using multicast distribution technology if the underlying transport layer supports this. An alternative implementation may use repeated applications of scenario S1 with a distribution list of intended recipients.
Two parties wish to conduct electronic business by the exchange of business documents. The sending party packages one or more documents into a request message, which is then sent to the receiving party. The receiving party then processes the message contents and responds to the sending party. Examples of the sending party's documents may be purchase order requests, manufacturing information and patient healthcare information. Examples of the receiving party's responses may include order confirmations, change control information and contractual acknowledgements.
Scenario S3 requires a “request/response” message feature. A request containing some business document is sent by a SOAP Sender to a SOAP Receiver where some business application is invoked. The business application processes the request and generates a response, which is returned to the SOAP Sender that originated the request. Two alternative solutions are described which depend upon the characteristics of the underlying transport layer. In either case, the SOAP Sender is informed of the status (successful or otherwise) of the request message delivery.
Figure 3 Request/Response using underlying transport
If the underlying transport protocol supports the correlation of a request and its matching response directly, then the solution illustrated in Figure 3 may be appropriate. An example of such an underlying transport protocol would be a synchronous HTTP POST. This implementation would make use of the transport binding proposed in other XML Protocol WG documents. The business document sent as a request by the SOAP Sender would be inserted as the payload of the request message. Following the receipt of the request, the processing application would generate the a document which would be returned as the payload of the response message with appropiate status codes. If for whatever reason, the request message was not received or processed by the intended business application, suitable status messages would be generated by the underlying transport layer and reported to the SOAP Sender.
Figure 4 Request/Response using SOAP headers
If the underlying transport protocol does not support a request/response model, then the configuration shown in Figure 4 may be appropriate. Examples of such an underlying protocol may include unidirectional queuing middleware. In this case, message identification and correlation is provided by SOAP Headers. In the request SOAP message, a Message Identifier Handler is responsible for generating a unique message identifier and inserting it into a SOAP Header. This forms part of the SOAP request message and is sent from SOAP Application 1 to the receiving SOAP Application 2. The request message is processed by a business application and a response message is assembled. This includes a SOAP Header built by a Message Correlation Handler which links the response message to its associated request.
The sender invokes the service by passing parameters that are serialized into a message for transmission to the receiving server.
Scenario S4 differs from scenario S3 in that the request message consists of a set of serialized parameters used to invoke some remote procedure which responds with a set of results. This is a different programming model to the document exchange one illustrated by scenario S3. Scenario S4 requires a request/response mechanism as in S3, with the parameter and result serialization needed for the RPC programming model form the SOAP Body element.
Figure 5 RPC using underlying transport
Figure 5 illustrates an RPC invocation over an underlying transport protocol such as HTTP that supports request/response. In this case, no additional headers are needed to correlate the request and response messages. Example request and response SOAP messages are:
Figure 6 RPC using SOAP headers
In Figure 6, the underlying transport protocol does not support request/response directly. The RPC request and response elements again form the Body of the SOAP messages. Correlation of the request and response is provided by the Message Identifier and Message Correlation handlers as described in scenario S3.
A sender wishes to reliably exchange data with a receiver. It wishes to be notified of the status of the data delivery to the receiver. The status may take the form of:
Figure 7 Request with acknowledgement
Figure 7 illustrates a request/response scenario with the SOAP Sender requesting status information from the matching SOAP Receiver. This status may provide delivery information to the sender in addition to other business related responses that the receiving application may generate. Figure 7 assumes that the underlying transport protocol supports the request/response exchange model. A Status Handler on registered with the SOAP Sender is configured to request the status information. A matching handler on the SOAP Receiver generates the requested status information and places it the response message which is then returned to the originating SOAP Sender.
In the example SOAP messages below, a StatusRequest header element includes an identifier for the message being sent. The inclusion of the StatusRequest header results in the receiving SOAP processor including a StatusResponse Header in the response. This includes information about the delivered message including an enumerated status and timestamp.
A sender wishes to exchange data with a receiver and has agreed to encrypt the payload. The sending and receiving applications agree on the encryption methodology. Data is encrypted by the originating application and sent to the receiver via SOAP. The data reaches the receiving application untouched, and may then be decrypted in the agreed-upon manner.
Figure 8 Request with encrypted payload
Scenario S6 describes two applications that wish to share encrypted data as an opaque body in a SOAP message. It places no requirements on the SOAP messaging layer. Figure 8 illustrates this scenario.
The following is the encrypted version of the above plain SOAP message. The body entry <m:GetLastTradePrice> is encrypted using a symmetric key identified by the key name "Symmetric Key" and replaced by the <xenc:EncryptedData> element with an id "encrypted-body-entry". A <SOAP-SEC:Encryption> header entry for this encrypted data is added to the SOAP header. Note that the <SOAP-SEC:EncryptedDataList> element in the header entry has a reference to the <xenc:EncryptedData> element. The symmetric key used for encryption is stored in the <xenc:EncryptedKey> element in the header entry in an encrypted form, that is, it is encrypted by John Smith's RSA public key.
[EDNOTE] Need to establish whether this does have an impact on headers. Is this really an opaque Body or does the application encrypting the Body also insert encryption Headers as well ?
A blind auction marketplace serves as a broker between buyers and suppliers. Buyers submit their requirements to the marketplace hub, which
broadcasts this information to multiple suppliers. Suppliers respond to the marketplace hub where the information is logged and ultimately delivered to the buyer.
Figure 9 Marketplace intermediary
Figure 9 illustrates an infrastructure where SOAP based messaging is used to support a third party marketplace acting as an intermediary between buyers and sellers. The market place business model involves the recruitment of multiple suppliers for goods and services. Buyers may then connect to the marketplace and take advantage of the services they provide. The marketplace acts as a channel for the commercial transactions between a buyer and its chosen seller. A marketplace can exist to serve both B2B and B2C transactions.
In scenario S7, the marketplace acts as a blind intermediary. A buyer connects to the marketplace and places an order for items or services it requires. The buyer may be as simple as a browser or as complex as a procurement application. Once the marketplace has received the buyer’s order, it contacts an appropriate set of sellers who then provide competitive bids aginst the order. The marketplace can then select the most attractive bid and connect the winning seller to the buyer. A purchasing process is then initiated with the marketplace acting as an intermediary in the transaction.
From a SOAP messaging point of view, the scenario illustrated in Figure 9 consists of a set of request/response messages between the buyer and the marketplace resulting in the buyer’s order being registered. Once received, the marketplace then contacts its set of selected sellers – again by a set of request/response messages. Design decisions made during the implementation of the marketplace software will determine whether supplier messages are sent from a single SOAP Sender to multiple SOAP Receivers, one at each of the seller’s sites. Alternatively, a SOAP Sender could be instantiated for each supplier and a physical 1:1 relationship established. Prior agreements on message qualities such as reliability, security and structure would be put in place between the marketplace and its sellers. These qualities would define what additional SOAP Handlers were needed for the message exchange patterns between the marketplace and sellers.
Two partners are engaged in a long-running process, which involves multiple message exchanges. Examples of such processes may be complex supply chain management, dynamic manufacturing scheduling or information retrieval. There may be multiple instances of the same process in progress between the same two partners.
Figure 10 Conversational message exchange
Interactions between business partners are usually more complex than a single request/response message exchange. A long running set of message exchanges may, for example be used to implement a business interaction such as procurement of goods or services. In this case there are advantages in grouping individual messages into a longer running set of exchanges. Such an exchange of messages is known as a conversation. Conversations may continue between a pair of trading partners for a long time. Completion of a conversation instance may take days, weeks or months.
A conversation between two trading partners may be defined by shared configuration information such as an ebXML Trading Partner Agreement (TPA). A TPA includes information such as expected response times, business process actions that each party undertakes to complete, security information and message content structures. In a procurement process, an example conversation may be:
All of the example message exchanges are related an instance of the TPA between the two partners. For a message to be valid as part of the agreed rules, each partner has to check whether the current message is valid within the scope of the TPS.
Figure 10 illustrates how this scenario could be implemented. Each partner’s SOAP processor has access to a database which is configured by the TPA agreed between the two partners. A Conversation State Handler in the SOAP Sender configures its SOAP Block with information that identifies a message with conversation instance it is part of. A matching handler in the SOAP Receiver uses the sender’s information to test whether the received message is acceptable within the rules of the TPA. It does this by checking with its own rules database where the state information on each of the conversation instances currently active is stored. If a message violates the rules of the TPA, then the application can raise a fault condition.
Note that Figure 10 does not include handlers for other message headers to support reliability or security which may be required under the agreed TPA.
In the following request and response examples, a ConversationState Header is used to identify which agreement governs the exchange between the two trading partners (AgreementId). To support multiple concurrent conversations under the same agreement, a ConversationId element is included. The values of AgreementId and ConversationId will remain constant for the lifetime of a particular conversational exchange and will appear in both request and response messages.
Two trading partners engaged in a message exchange may agree to cryptographically sign and verify either the message header, the routing header(s) and/ or the payload. The sender or originating application may perform the signing of the payload. The sending message handler signs the message header. A routing header may be appended to the message header. The routing header may also be signed by a message service handler.
Figure 11 Header and payload encryption
In scenario S6, two applications communicated using encrypted payloads. These opaque payloads had no impact on the SOAP processing layer. In this scenario, the action of signing and/or encrypting the headers or payload is the responsibility of the SOAP processing layer. Figure 11 illustrates how the encryption agreements are accessible to a Message Signing Handler on the SOAP Sender and a matching Message Verification Handler on the SOAP Receiver. An additional Message Routing Header may also be part of the SOAP message. This header may also be signed and verified if needed by the security requirements of the message exchange.
example encrypted SOAP message
An intermediary forwards a message to the ultimate receiver on behalf of an initial sender. The initial sender wishes to enforce the non-repudiation property of the route. Any intermediate message service handler that appends a routing message must log the routing header information. Signed routing headers and the message readers must be logged at the message handler which passes the message to the ultimate receiver to provide the evidence of non-repudiation.
Figure 12 Routing and logging through intermediaries
Scenario S11 requires an audit chain to be created between a SOAP Sender that originates the message and the ultimate SOAP Receiver including any SOAP Intermediaries that the message passes through. Figure 12 illustrates a possible implementation of this scenario. Each SOAP Node on the message path has access to a persistent store (typically a database) that can be used to store an audit record for each message. A Routing Logging Handler on each SOAP Node has the responsibility of logging each message in the persistent store. A further responsibility of the handler is to sign the message routing header before passing the message on to the next SOAP Node in the path. Support for certificates and other artifacts required for signing a message are not shown.
Figure 13 Asynchronous messaging
Scenario DS17 is the same as the basic request/response pattern described in scenario S3. The difference is that the request and response messages are separated in time and implemented as two unidirectional messages. The sending SOAP Application does not block and wait for the response to return. The sending SOAP Application is notified when a response is received by its SOAP Receiver. It then uses the correlation information within the received message to match the response to a message it sent some time earlier.
Figure 11 illustrates a possible implementation. In the request SOAP message, a Message Identifier Handler is responsible for generating a unique message identifier and inserting it into a SOAP Header. This forms part of the SOAP request message and is sent from SOAP Application 1 to the receiving SOAP Application 2. The request message is processed by a business application and a response message is assembled. This includes a SOAP Header built by a Message Correlation Handler which links the response message to its associated request.
A digital camera wishes to transmit image data over a wireless link using SOAP to a remote server. The binary image data (non-XML) accompanies the message. The digital camera represents a situation in which connections from the receiver to the sender may not be permitted due to device limitations or firewalls.
Figure 14 Sending non-XML data
Support for non-XML data has been described elsewhere. The SOAP with Attachments [SOAPAttach] note to the W3C has been adopted by the ebXML Message Services specification [EBXML] as the basis for defining a message structure which can support non-XML data. Supporting non-XML data requires additional packaging of the message which can be provided by a MIME multipart structure and impacts the binding of a message to its underlying transport protocol. Figure 14 illustrates a unidirectional SOAP message path. A Message Manifest Handler is implemented which creates a set of references to the different parts of a multipart MIME package. Each part is referenced by its content identifier.
Figure 15 Using MIME packaging for non-XML data
Figure 15 illustrates how different parts of a message are packaged using MIME multipart. The outermost MIME envelope packages a set of individual MIME parts. The first MIME part contains a SOAP message which includes the Manifest Header block created by the Message Manifest Handler. The second and subsequent MIME parts contain payload(s) which may be XML documents or any other MIME content type such as image, audio or video data. The SOAP manifest header can contain elements that reference the separate MIME parts using their content identifiers. This may be achieved using XLink references as shown in the following example. The XLink role attribute may be used to further qualify the type of data contained within the payload.
An application requests some information from a server, which is returned at a later time in multiple responses. This can be because the requested information was not available all at once (e.g., distributed web searches).
Figure 16 Multiple asynchronous responses
Scenario S20 is an extension of scenario DS17 – asynchronous messaging. Instead of a single response message, more than one can be sent by the receiving application to the originator. A simple architecture would be the same as DS17 with multiple responses received by the originating application and corelated to the original request by a Message Correlateion Handler. Figure 15 illustrates an extension to this using a Sequence Handler. The Sequence Handler ensures that a unique sequence number is added to each response message. If the responding application knows in advance that there will be a fixed number of multiple responses, then the Sequence Handler may use an N of M format to indicate how many response messages are to be expected.
An SOAP sender generates a lengthy SOAP message that is incrementally transmitted and received by a SOAP receiver. The SOAP receiver employs a SOAP handler that can incrementally process the body as it is received (e.g., employing a SAX-style XML parser on the body as it arrives). Note that the entire message need not be present at one time at any point in its existence.
This would be particularly helpful for memory-limited processors. It is also very efficient for services which are consistent with incremental, real-time transformations of the data, direct archiving of received data, etc. It would also be useful in scenarios in which voluminous body data can be directly transduced into application data structures or events by a SOAP (module) processor. In particular, there is no need for the explicit construction of a DOM model of the data. Support for SOAP data models might still be possible even with incremental processing if the models are incrementally constructible.
Scenario S21 requires the incremental parsing and processing of a SOAP message by a receiver. This is a general scenario with memory-limited processor requirements forming a subset of the scenario. If the SOAP Body contains a large amount of data, then it may be processed incrementally by a SAX parser if the data is chunked as in the following example. The SAX parser will have a handler triggered by the BodyDataChunk element.
If a SOAP request is being streamed and processed incrementally, then the matching response message may be streamed to the original sender. In this case, the design of the receiving application is critical with respect to timing and error handling.
1. If errors are generated by the SOAP request Headers, then a SOAP Fault is inserted in the response and processing of the request message is terminated.
2. The SOAP receiving application may treat each BodyDataChunk element as atomic. A positive or negative acknowledgement is streamed to the SOAP response depending on whether the BodyDataChunk element was successfully processed or not. The SOAP response message is terminated once the end of the SOAP request is reached.
3. Alternatively, the SOAP receiving application may process each BodyDataChunk until either the end of the SOAP request is received or a fault occurs. In the case of a fault, a SOAP Body fault element is streamed to the SOAP response and processing of the SOAP request is terminated.
Figure 17 Publish and subscribe
Scenario S23 describes event notification using a publish subscribe mechanism. An implementation of this scenario uses an example of the request/response scenario S3 to register a subscription and fire-and-forget to multiple receivers scenario S2 for the notification. Figure XX illustrates how a request/response message pattern can be used with a Subscription Request Handler to register an interest (or subscription) in some set of events. The registration is made with some subscription service. The success or otherwise of the registration is returned to the subscribing application using a Subscription Ack Handler which provides an acknowledgement to the subscribing application.
Delivery of an event noification to a set of subscribers may be implemented using the fire-and-forget to multiple receivers scenario S2. The subscription service provides the list of valid applications that have registered an interested in a particular event. This list may then be converted into a group address or distribution list to support the implementation of the fire-and-forget scenario.
A subscription request may include a list of events within the SOAP Body as in the following example.In this example, a subscription is registered with a stock price notification service. The subscribing application will be informed of company BigCo’s stock price, volume traded and time whenever the price is greater than 100.
An acknowledgement may include an identifier to the subscription as in the following example:
The identification may be used in subsequent notifications to the application as a result of the subscription:
Some applications may wish to make caching possible for latency, bandwidth use or other gains in efficiency. To enable this, it should be possible to assign cacheability in a variety of circumstances. For example, "read" caching might be used to store messages at intermediaries for reuse in the response phase of the request/response message exchange pattern. Such caching might be on the scope of an entire message, a SOAP module, or scoped to individual SOAP module elements.
Similarly, "write" caching may be useful in situations when a request message in a request/response message exchange pattern (as well as similar messages in other message exchange patterns) does not need to be immediately forwarded or responded to. Such cachability might be scoped by different methods, as outlined above.
Cacheability scoped by different elements might be associated by an attribute to the target element, through use of XML Query or XPath to describe the target elements in a header, or implied by the document schema, for example.
Cacheability mechanisms applied to messages, bodies or elements might include time-to-live (delta time), expiry (absolute time), entity validation, temporal validation, subscription to invalidation services, and object update/purge.
Finally, some applications may be capable of describing the dependencies and relationships between message elements. For example, a response element may be applicable to a wide range of requests; it would be beneficial to describe this element's relationship with request elements, so that it may satisfy a wide range of requests in an economical fashion. Similarly, the presence of a particular element may be a trigger for a cacheability mechanism to be applied to another element, such as validation or invalidation.
Caching is frequently used as an optimization in distributed systems. It can be used to avoid re-doing computations or complex database access when the results remain valid for an extended period of time. In this case, subsequent requests for the same information can be served with the cached version rather than repeat the processing with the associated overheads. Another use of caching is in the transmission of data where copies may be held at leaf servers for local service provision rather than repeatedly access a central information repository. This has the combined effect of providing faster access to the information, reducing network bandwidth requirements and reducing the workload on a central server. Caching may be provided as part of an underlying transport infrastructure but in the case of this scenario, it is assumed that the caching is independent of any underlying transport.
An example of this kind of scenario is the caching of the response to a request in situations where a subsequent request can be safely answered with the same result. This example coincides with scenario S809 (Caching with expiry) where a catalog is updated at 8am each morning. Once the catalog has been updated, all price queries against it are valid until 8am the following day. If a price query request is repeated against the same item, then a cached response can be returned to the SOAP Sender otherwise the request is forwarded to the catalog server and its response is cached. All entries in the cache are purged at the time of the updated catalog being available. Figure 18 illustrates a possible architecture.
Figure 18 Result Caching
SOAP Application 1 initiates a request for catalog price information illustrated in the following example.
The caching intermediary SOAP Application 2 is unable to fulfil the request from its local store so it forward the request which ultimately arrives at the catalog server SOAP Application 3. The catalog server process the request and assembles a response message containing the requested price information. An additional SOAP Header is placed in the response to control any caches that may exist in the return path. The CacheControl Header contains a CacheKey which allows matching of future requests to the cached response together with an Expires element that sets the time the local copy must be purged. This response is returned via the caching intermediary.
At the caching intermediary, the CacheControl header information is used to make a local copy of the response message, keyed by the CacheKey. The copy will be purged at the time specified by the Expires element. The CacheControl header element is removed by the intermediary and the catalog price information is returned to the original sender. The request/response path for this message is the complete roundtrip between the original SOAP Sender and SOAP Receiver and is shown by Message Path 1 in Figure 18.
Since there is now a local copy of the price information for item ABC-1234 in the intermediary cache, subsequent requests for price information can be fulfilled by the intermediary. This in the shorter request/response Message Path 2.
This scenario has been addressed in detail by the WS-Routing [WSRP] (formerly SOAP-RP) specification.
A service provider wishes to track incoming messages to see exactly which processing intermediaries have touched it by the time it arrives at its destination. It therefore requires a tracking extension to be included by all clients, and by any processing intermediaries along the message paths from the clients to the server.
Figure 19 Message Tracking
Scenario S805 describes a routing requirement whch is addressed in detail by the WS-Routing [WSRP] (formerly SOAP-RP) specification. This describes how a message may be reouted through some messaging infrastructure. Once the message has arrived at its ultimate receiver, the route the message has taken may be required for auditing purposes. A track of the message path may be created by adding a tracking header to the message in addition to any routing information.
This is illustrated in the following example. A routing header has been added to the message in accordance with WS-Routing [WSRP]. A TrackingHeader is used to mainitain a list of Intermediary names and associated Timestamp elements. As the message passes through each intermediary, a Tracking Handler appends a Via element to the TrackingHeader. The Via element contains the name of the intermediary together with the date/time the message arrived or was forwarded by the intermediary. The list of Via elements therefore forms the audit trail for the message.
BizCo updates their online price catalog every morning at 8AM. Therefore, when remote clients access their SOAP inventory service, clients and intermediaries may cache the results of any price queries until 8AM the next day.
See description for DS24.
An SOAP sender (not necessarily the initial SOAP sender) wants the SOAP message to be handled with specific quality of service as it traverses the SOAP message path to include multiple SOAP Processing intermediaries. Information in the SOAP message is used to select appropriate QoS mechanisms (e.g., RSVP, Diffserv, MPLS, etc.). Selection of QoS may be constrained by QoS policies, Service Level Agreements (SLAs), Service Level Specifications (SLS).
[SOAPReqs] "XML Protocol (SOAP) Requirements" http://www.w3.org/TR/2001/WD-SOAP-reqs-20010319/#N2082
[WSRP] “Web Services Routing Protocol (WS-Routing)” http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsrvspec/html/ws-routing.asp