W3C Workshop on Binary Interchange of XML Information Item Sets Position Paper

Position Paper for "The W3C Workshop on Binary Interchange of XML Information Item Sets", September 24-26th, 2003, Santa Clara, California, USA

Stephen D. Williams Aug. 8th, 2003 sdw@lig.net 703-724-0118

The topic of this workshop appears to be entirely anticipated by my work and recent presentation on what is currently called Efficiency Structured XML, or esXML, and its library esDOM. This position paper will briefly answer the requested items in the meeting announcement and incorporates by reference the full presentation and discussion of esXML at: http://www.esxml.org

Characteristics suggested and comparison to esXML/esDOM:

It would not be restricted to a single schema or vocabulary, and hence could be interoperable between vocabularies;

esXML achieves this by remaining self describing with XML 1.0 semantics. Greater efficiency can be obtained, similar to the use of a prearranged schema or IDL, by the use of a delta/copy on write reference semantic that esXML supports. In this case, a template or ealier message could provide a tokenization table or element/attribute/data content that was unchanged.

It would not be restricted to a single application or hardware device, and hence could be interoperable between implementations;

esXML is fully network/architecture/cpu/os portable and self-contained. It also requires far less total memory than other methods.

Improved network efficiency and reduced storage needs: compression techniques that make use of domain-specific knowledge often do better than more generic compression;

esXML is not a 'compression' step, but rather somewhat similar to a persistant DOM or a 'preserialization' method. The key idea is that memory format and wire format are the same and are both compact, efficient, and modifiable in-place.

Sending pre-parsed data could reduce the complexity of applications, and may facilitate creation of simpler internal data structures.

This is exactly what I propose. In fact, with methods, an application can represent all application/business data directly in esXML objects/documents with no 3GL/native data structures and be more efficient and require less coding overall.

Web Services may need more efficiency, and a pre-parsed binary transmission format may help people to continue to work with Web Services rather than to explore proprietary interfaces.
- Definitely an area that esXML is targetted at.

What work has your organization done in this area? (We are particularly interested in measurements!)
- An earlier prototype of esXML was created for internal work in an MPEG4 scene graph manager. The new, extended, and refined design is being implemented and benchmarks will be forthcoming soon. It is quite apparent that for some common application models, that the theoretical efficiency and like actual performance will have a significant improvement. The esDOM API has been implemented on Xerces/Java and found to be useful and straightforward.
What goals do you believe are most important in this area? (e.g. reducing bandwidth usage; reducing parse time; simplifying APIs or data structures, or other goals)
- The most important goal is removing, not just reducing, parsing and serialization and associated object creation/destruction in nearly all cases where you are not crossing an esXML/XML1.0 boundary. Simplifying APIs, especially by making the API more like accessing native variables and collection libraries, is extremely important. Bandwidth usage is also important, but less so.
What sort of documents have you studied the most? (e.g. gigabyte-long relational database table dumps; 20-MByte telephone exchange repair manuals; 2 KByte web service requests)
- 1-300K application messages, some containing thosands of fields, meta-data, etc., and others with significant text. Also, encrypted and signed XML has been considered. The structure is not limited to certain sizes, but is optimized to be efficient at various sizes through the use of variable sized length indicators and optional token tables and element indexes.
What sorts of applications did you have in mind?
- The area that is most important to solve is online application
If you implemented something, how did you ensure that internationalization and accessibility were not compromised?
- Character data is stored in UTF-8, but any type of data can be stored.
How does your proposal differ from using gzip on raw XML?
- esXML is not a compression step. Adding compression like gzip adds processing and does nothing for existing overhead. esXML avoids parsing, serialization, and associated processing. Data is stored and manipulated in a compact format.
Does your solution work with any XML? How is it affected by choice of Schema language? (e.g. W3C XML Schema, DTD, Relax NG)
- esXML is intended to support any XML data and does not use or need to be aware of schemas or DTDs. It is completely self-describing in the XML sense.
How important to you are random access within a document, dynamic update and streaming, and how do you see a binary format as impacting these issues?
- Random access and in-place dynamic update are very important and are supported by esXML. I don't believe that streaming is nearly as important or as widely needed, especially for web, online, and business applications. Streaming however can make sense and can be supported in a reasonable way by esXML. Streaming can be done with esXML by the use of esXML deltas which reference a one or more base, i.e. 'before', versions of the object/document and efficiently records changes to anything. An application would have to make some decision to 'flush' a stream chunk, a page for instance.

One question you didn't ask that seems obvious is the handling of binary data. Binary data, such as an image, included in esXML is stored unencoded as binary data. Of course, conversion to XML 1.0 text encoding would require b64 or similar, but in many cases this would allow great efficiency.

It is a design requirement of esXML/esDOM libraries to support both XML 1.0 and esXML via a run-time configuration step so that debugging or other inspection can occur when needed. Of course utilities will be availble for such conversion also. The loss of source viewability is an issue, but it just has to be traded for efficiency. I can't see any details about an image until I view or edit it, and this is a similar tradeoff.

I believe I am proposing a number of new paradigms which appear to be viable and useful. I would very much like to be invited to give a presentation on the first day of this working group.

Thank you for your consideration.

Stephen D. Williams

Senior Technical Director at High Performance Technologies, Inc.