XML Processing Model Workshop

Sun Position Paper 22 Jun 2001

This version:
Workshop Submission: 22 Jun 2001
Editors:
Norman Walsh <Norman.Walsh@Sun.COM>, Eve Maler <Eve.Maler@Sun.COM>, Christopher Ferris <Chris.Ferris@Sun.COM>

Abstract

We observe that the lack of an underlying processing model allowing applications to describe the semantics of a mixed environment of XML processes poses a serious threat to the development of interoperable web applications.

Motivated by a selection of use cases, we describe our notion of an XML process and of the consequences of processing order on a successful application.

We conclude with our thoughts on how the W3C should proceed.

Table of Contents

1 Introduction
2 What Is a Process?
2.1 Infoset Composition
3 Use Cases
3.1 Simple Business Transactions
3.2 Transforming to a New Schema
3.3 Document Publishing
3.4 Business Transaction Hub
3.5 Web Service Implementation
4 Processing Order
5 Conclusion

Appendixes

References

1. Introduction

There is a large, and growing, set of specifications that describe processes operating on XML documents. Considering how these specifications interact raises a large number of issues. Our principal concern in this position paper is interoperability.

There are several broad classes of processes that need to be considered:

  1. Constructive processes. These are processes that build new XML documents. An [XML 1.0 (Second Edition)] parser (with required, or perhaps optional, support for [Namespaces] and [XML Base]) is one example of this class. [XSL Transformations] and [XQuery] are others. Basically, processes which produce a new or substantially different XML document from some input source can be described as constructive.

  2. Augmenting processes. Processes, like [W3C XML Schema] validation or [CSS] styling, that add new information to an existing document are performing some sort of augmentation. Verifying a digital signature might also leave the documentation unchanged, but indicate somehow that it passes or fails a signature check. [XInclude] might be viewed as an augmentation process.

  3. "Peephole" processes. Processes may sometimes operate on only a small fraction of an existing document; perhaps to perform validation of a particular namespace, for example, or to decrypt a region of encrypted content.

    It seems possible that these processes will have different characteristics than more global operations, especially with respect to implementation efficiency.

  4. Extraction processes. Some processes reach into an existing document and copy (or link or remove) parts of it. Processes such as [XPointer] and [XLink] (and applications which are built on top of them) fall into this category. So do [XInclude] and [XQuery].

  5. Packaging processes. Distributed or federated web applications will need to package a collection of resources to transmit to another location or service. This packaging may be performed using using XML Protocol, for example, or [SOAP]. We note that the whole issue of packaging resources and providing a useful manifest is not adequately addressed.

  6. Packaging processes. A meta-issue that has not been clearly addressed yet (although processes XML Protocol and [SOAP] may take steps in this direction) is packaging. If you need to send a set of documents and processing model information to another system, how do you do it?

This is by no means an exhastive list of the processes, nor do processes always fit cleanly into the categories, as you can see. What appears to be augmentation in one application might also be a peephole process in another or even a constructive process. All of which is compounded by the fact that processes can be hierarchical, with a transformational process performing a bit of peephole validation or extraction, for example.

The fact that documents may be augmented or transformed raises another set of issues with respect to addressing into augmented or transformed results. The [XML Linking and Style] discusses this at some length, but there are still many unresolved issues.

The use cases described in Section 3 demonstrate that no fixed order of processing can be imposed, but but the order of processing actually used is an important aspect of the semantics of the application. It follows that the ability to identify an underlying processing model and describe how processing is to proceed is an imperative precondition to developing successful, interoperable web applications.

2. What Is a Process?

Before we can investigate what it means to have a processing model, we need some common understanding of what constitutes a process.

For the purposes of this document, we consider any application which constructs, transforms, or augments an [Infoset] to be a processor. A process begins with zero or more Infosets and produces zero or more Infosets (it may also produce ancillary information, such as whether it succeeded or failed). Some processes (such as [Schematron] and [RELAX NG] validation) don't strictly speaking construct, transform, or augment an infoset, but we can finesse that point for the moment by imagining that they perform the identity transformation.

Actually, processes that augment an existing infoset but don't clearly produce a new one represent a special class of process because they may change the fundamental model on which some other process expects to operate.

Although some, perhaps many, applications work with concrete object models that are not identical to the Infoset, it's still useful to describe the processing model in terms of the standard Infoset.

2.1. Infoset Composition

On the surface, the model we describe here, where processes consume and produce infosets, seems to imply that infoset composition is well-defined. In fact, that is not the case. Some infoset properties introduce dependencies between information items:

  • Schema validity properties introduce dependencies on descendants.

  • In-scope namespaces may introduce dependencies (at least implicitly) on ancestors.

  • Markers, as have previously existed, introduce dependencies among siblings.

These dependencies make it difficult to perform arbitrary composition. We believe that this issue deserves immediate consideration.

3. Use Cases

We motivate the rest of our discussion by considering a few simple use cases.

3.1. Simple Business Transactions

Two parties agree to conduct business electronically. They will exchange purchase orders, invoices, and other business documents using some appropriate transport protocol. Before responding to a request, each party wishes to validate the request against a known schema so that errors do not result in mismanaged funds.

This use case depends on approximately the following processing order:

  1. Documents must be parsed and verified as correct with respect to [XML 1.0 (Second Edition)], [Namespaces], and [XML Base].

  2. Any XInclude elements must be expanded.

  3. Validation must be performed against a specific schema, ignoring any schema location information in the document.

3.2. Transforming to a New Schema

Suppose I have a collection of documents in "XYZ Schema V1.0". A new release, "XYZ Schema V1.1" is released which contains a small number of backwards incompatible (but programmatically correctable) changes and a few new features. I want to update my existing documents to the new schema in order to exploit the new features.

This use case depends on approximately the following processing order:

  1. Documents must be parsed and verified as correct with respect to XML 1.0, XML Namespaces, and XML Base.

  2. Validation must be performed against a specific "XYZ Schema V1.0" schema, ignoring any schema location information in the document (in order to be sure that local extensions won't interfere with the automated conversion process).

  3. Documents must be transformed with a specific stylesheet, ignoring any style information in the document.

  4. The transformation must preserve all existing markup (it must be an identity transformation) except for the specific changes required to convert to V1.1. (In particular, it must preserve existing XInclude elements.)

  5. Schema location information in the document must be updated.

  6. The converted document must be validated against the V1.1 schema, making use of any local schema information that is present, in order to assure that the transformation did not introduce errors.

3.3. Document Publishing

Imagine that I have a collection of documents in XML. I wish to publish them on a periodic basis.

This use case depends on approximately the following processing order:

  1. Documents must be parsed and verified as correct with respect to XML 1.0, XML Namespaces, and XML Base.

  2. The document must be schema validated using local schema information that is present.

  3. The document must be transformed and published either with the stylesheet information in the document or with a specific set of stylesheets.

3.4. Business Transaction Hub

Imagine that I've built some sort of a hub for doing business transaction processing. I accept documents in a variety of schemas, transform them to some internal schema, operate on those documents, and return results in the schema appropriate for the requestor.

This use case depends on approximately the following processing order:

  1. Documents must be parsed and verified as correct with respect to XML 1.0, XML Namespaces, and XML Base.

  2. XInclude elements must be expanded.

  3. The document must be schema validated using either the schema identified by the document or an out of band schema.

  4. The document must be transformed to the "hub schema". This new document may use XInclude elements to refer to standard boilerplate or other constant information.

  5. XInclude elements must be expanded again.

  6. Validate again, to make sure the transformation didn't introduce errors.

  7. Perform whatever processing is required.

  8. Transform the result into an appopriate outbound schema.

  9. Perhaps expand XIncludes again.

3.5. Web Service Implementation

Consider a web service that is part of some larger service chain. It may need to operate only on portions of a document (because other portions are encrypted, for example, or simply because it only deals with a certain namespace). It may perform validation on only some elements, for example, or expand only certain XIncludes.

This use case depends on approximately the following processing order:

  1. Documents must be parsed and verified as correct with respect to XML 1.0, XML Namespaces, and XML Base.

  2. Selected portions of the document must be schema validated.

  3. XInclude processing must be performed selectively.

  4. The infoset may be augmented or transformed as a result of the web services operation.

4. Processing Order

It seems clear from the use cases that no single, fixed order of processing will satisfy all possible applications. Although it would in many ways be easier to build interoperable systems if a fixed order could be defined, any such order would impede some classes of applications.

However, the fact that no fixed order can be defined, does not imply that the order of operations is entirely independent. For any given application, it may be imperative for some operations to proceed others.

This observation offers a key insight into a possible successful processing model: for any given application, the processes that must be performed are partially ordered, but not totally ordered. In other words:

  1. There are dependency relationships between the processes and any processing order that violates these dependencies is erroneous. It does not produce reliable, accurate, or interoperable results. It does not even produce correct results.

  2. But any order that does not violate the dependency relationships is equally correct.

The satisfaction of dependencies between processes is already a well understood problem in software development, it forms the heart of systems like make and even more appropriately ant.

For the purposes of illustration, we consider a simple application of XInclude, XML Schema validation, and XSL Transformation.

The specific case we illustrate is as follows:

  1. Begin with a source document, mydoc.xml.

  2. Expand any XIncludes that it contains.

  3. Make sure that it is schema-valid with respect to someschema.xsd.

  4. If it is, transform it with mystyle.xsl, and return the result. Otherwise return an error document.

When presented with a process control document that describes this process, an "XML Process Manager" builds a dependency graph and selects a valid order in which to execute the steps. And it executes them (in series or parallel). It returns the resulting document(s).

Note that the order of the processes in the control document is insignificant. There's no reason why users should have to figure out the right order for the whole pipeline; they need only declare the dependencies.

We will shortly be making a proposal for an XML vocabulary that will allow these dependencies to be expressed so that an "XML Process Manager" can carry out the required processing.

5. Conclusion

We are encouraged to see increased awareness within W3C of the fundamental interoperability issues surrounding the use of XML. We believe that this is a fruitful area of pursuit and we would like to see it reflected in the XML Activity's next charter.

We hope, in particular, that W3C resources on the XML Activity can be augmented appropriately to address the pressing problems of infoset composition and interdependency of specifications.

References

John Barton, Satish Thatte, Henrik Frystyk Nielsen, editors. SOAP Messages with Attachments. World Wide Web Consortium, 2000.

Tim Bray, Jean Paoli, and C. M. Sperberg-McQueen, Eve Maler, editors. Extensible Markup Language (XML) 1.0 Second Edition. World Wide Web Consortium, 2000.

Tim Bray, Dave Hollander, and Andrew Layman, editors. Namespaces in XML. World Wide Web Consortium, 1999.

Bert Bos, Håkon Wium Lie, Chris Lilley, et. al., editors. Cascading Style Sheets, level 2. World Wide Web Consortium, 1998.

Don Chamberlin, James Clark, Daniela Florescu, et. al., editors. XQuery 1.0: An XML Query Language. World Wide Web Consortium, 2001.

James Clark, editor. XML Transformations (XSLT) Version 1.0. World Wide Web Consortium, 1999.

James Clark, editor. OASIS RELAX NG Technical Committee. OASIS. 2001.

Steve DeRose, Eve Maler, Ron Daniel Jr., editors. XML Pointer Language (XPointer) Version 1.0. World Wide Web Consortium, 2001.

Rick Jelliffe. The Schematron. Academia Sinica Computing Centre. 2001.

Jonathan Marsh, editor. XML Base. World Wide Web Consortium, 2000.

Jonathan Marsh and David Orchard, editors. XML Inclusions (XInclude) Version 1.0. World Wide Web Consortium, 2001.

Henry S. Thompson, David Beech, Murray Maloney, et al. editors. XML Schema Part 1: Structures. World Wide Web Consortium, 2000.

Richard Tobin, John Cowan, editors. XML Information Set. World Wide Web Consortium, 2001.

Norman Walsh, editor. XML Linking and Style. World Wide Web Consortium, 2001.