Copyright © 2007 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
The purpose of this document is to provide guidelines for the interoperable deployment of the Efficient XML Interchange (EXI) format. It provides explanations of format features and techniques to support interoperable information exchanges using EXI. While intended primarily as a practical guide for systems architects and programmers, it also presents information suitable for the general reader interested in EXI's intended role in the expanding Web.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This First Public Working Draft was developed by the Efficient XML Interchange (EXI) Working Group. A complete list of changes to this document is available.
Comments on this document are invited and are to be sent to the public public-exi@w3.org mailing list (public archive). If substantive comments are received, the Working Group may revise this Working Draft or publish it as a final Working Group Note.
Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. The group does not expect this document to become a W3C Recommendation. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
1. Introduction
2. Deciding Whether or Not to Use EXI
3. XML/EXI Interoperability
3.1 EXI's Place in the XML Family of Technologies
3.2 Preserving XML Features in EXI
3.3 Security
3.3.1 XML Signature
3.3.2 XML Encryption
3.3.3 Alternatives to XML Security
4. Mixed XML/EXI Environments
4.1 Identifying EXI Content
4.1.1 Distinguish EXI from XML
4.1.2 Filename Extensions and MIME Types
4.2 Negotiating Content in HTTP
5. Shared Encoding Knowledge
5.1 Schemas
5.2 Pluggable Codecs
5.3 Out-of-Band Encoding Knowledge from Prior Agreements
6. Maintain a "View Source" Capability for Debugging
7. References
7.1 Normative References
7.2 Other References
A Acknowledgements (Non-Normative)
The EXI Best Practices are guidelines for effectively employing the EXI format. The guidelines pertain not only to EXI's effective use in constrained environments, but also to addressing interoperability in environments where both XML and EXI are options for serializing the [XML Information Set]. The EXI format is interoperable with XML technologies. However, since it is an alternative serialization of the XML Infoset, consideration is warranted when employing EXI in environments in which textual XML is also used. While EXI features support interoperability with XML, specific techniques for addressing mixed XML/EXI environments is beyond the scope of the format specification. These techniques are included in this document.
Use EXI only in cases where a primary objective is to leverage XML technologies, but use of the XML 1.x format cannot support the required efficiency. Optimized implementations and parsing technologies should be considered in this determination.
| Editorial note | |
| This section will recommended a process (e.g., a decision tree) as guidance for determining the applicability of EXI in a given scenario/environment. | |
"XML is a family of technologies" [XML In 10 Points]. These technologies have been developed to accomplish frequent, important tasks and their number continues to grow. The majority of XML technologies interact with XML1.x instance documents via an abstract model defined in [XML Information Set]. In other words, an XML instance document can be viewed as a serialization of an XML Infoset. In this manner, XML 1.x serves as an interchange format between XML technologies.
EXI is also intended as an interchange format within the XML Family of Technologies. Like XML, it can be used to serialize an XML Infoset, but more efficiently than XML. Thus EXI is an alternative interchange format, designed for resource constrained environments. EXI is interoperable with XML technologies which operate at the level of XML Infosets (as most do). XML technologies which operate at the text (or octet) level can interoperate with EXI, but require special considerations and may affect how EXI is employed. These are discussed below in 3.3 Security
Even though XML has been enormously successful in diverse areas and use cases, different applications often require different XML feature sets. Treatment of those unused features implies unnecessary overhead. The EXI Format addresses this imbalance by offering a simple way to set fidelity options. Via these fidelity options, EXI supports preserving the following lexical characteristics: comments, processing instructions, insignificant whitespace, doctypes (entity references), namespace prefixes, and element/attribute content. As discussed above, these options are set in the "preserve" section of the EXI header.
While the meaning of preserving XML features like comments, processing instructions, and doctypes might be rather evident, additional explanation is required for other features. For instance, the option of preserving or not preserving whitespace does not pertain to all whitespace characters. In fact, if whitespace is not preserved, only insignificant whitespace characters are eliminated. Mixed content elements and XML elements set to preserve spaces (xml:space="preserve") are understood as significant whitespace and therefore retained in any case.
In addition, consider namespace prefixes. XML prefixes are by definition bound to namespaces, but the precise lexical expression of the namespace prefix can be suppressed without loosing information required to resolve the references. All that is really needed is to maintain consistent namespace references. However, the precise lexical representation of a prefix (e.g. "xsi") can aid a human reader in understanding certain namespace references. In cases like these, the actual strings used as namespace references can be preserved in EXI.
EXI also provides the option to preserve the lexical form of element/attribute content. In an XML instance document, a single value may be represented by more than one lexical string. Equivalent lexical representations are mapped to a single value in the EXI format by default. Mapping to a distinct representation enables a more efficient serialization. However, if the lexical representation plays a important role in an application, EXI provides the option to preserve all lexical content, regardless of whether they can be mapped to the same value.
It is important to note that all fidelity options in EXI are by default set such that no lexical features are preserved. This is in order to achieve high compression and processing efficiency. The option to preserve a particular category of lexical values must be explicitly set in the EXI header. The EXI fidelity options are defined in [EXI Fidelity Options]
| Editorial note: MC | 19 November, 2007 |
| The EXI and XML Security working groups met during the 2007 W3C Technical Plenary on 9 November to dicuss this material. No issues were uncovered concerning the proposed approach. | |
Use EXI fidelity options with XML Encryption and Signature.
Any XML canonicalization algorithms employed by encryption or signature require the following options to be enabled: Preserve.pis, Preserve.prefixes, and Preserve.lexicalValues.
If the canonicalization algorithm also preserves comments, then Preserve.comments must also be enabled.
The phrase XML Security generally refers to a set of technologies that enable integrity, confidentiality and authentication of XML messages. [XML Encryption] and [XML Signature] are two W3C recommendations that address these requirements. XML Encryption defines a mechanism by which messages and message parts can be encrypted with a key to provide confidentiality. XML Signature defines a mechanism by which messages and message parts can be digitally signed to provide integrity, to ensure that the data is not tampered with, and authentication, to verify the identity of the message producer (although, in practice, other methods like User-Token are preferred means of establishing a producer's identity). If more than one property is required, such as integrity and confidentiality, these technologies can be combined to produce messages with encrypted parts and signed parts, or even parts that have been both encrypted and signed.
In general, XML technologies that operate at the level of the [XML Information Set] require no modifications to use the EXI format. Digital signature and digital encryption technologies operate at the level of octets. Thus, an XML Infoset needs to be serialized into an octet stream before it can be signed or encrypted. Furthermore, in the case of XML signature, this serialization must be clearly defined so that the processor that verifies the signature can do so unambiguously, thus enabling interoperability. For this reason, the XML Signature recommendation mandates the inclusion of a canonicalization method element as part of the meta-data describing a signature. Several XML canonicalization methods exist, typically varying on which XML Infoset items they ignore and on how they handle namespaces. Regardless of these differences, they all define a proper function (i.e., not just a relation) between XML Infosets and octet streams.
Conceptually, the operation of securing a message can be regarded as an XML Infoset transformation [1] guided by certain configuration parameters defined by the application's developer. In this sense, securing a message takes place after its Infoset is produced and before it is serialized; conversely, unsecuring a message takes place after a message is parsed and before it is consumed. In this regard, and provided that the appropriate encoding options are chosen, the EXI format can still be used as the on-the-wire encoding without affecting these two pre-serialization and post-parsing steps. Note that, as long as the process of mapping XML Infosets into octet streams is not altered, this approach still requires the use of an XML parser and an XML serializer during the message securing and unsecuring phases. Therefore, a software stack that supports this type of processing will have to include libraries for both XML and EXI. Although we expect this to be the norm in most cases, it nonetheless can present an architectural challenge in small devices where memory capacity is at a premium.
As indicated earlier, XML Signature provides a way to specify the canonicalization method or algorithm as part of its meta-data. Canonicalization algorithms are identified via the use of URIs. Since there are multiple ways of encoding an XML Infoset into EXI, for example by using different fidelity options, it cannot be used as a way of mapping an XML Infoset into an octet stream. However, if a canonical EXI and a new URI to identify it are defined, the extensibility points in XML Signature could be used without requiring any additional changes to the existing recommendation.
XML canonicalization can be inclusive or exclusive, depending on what contextual information is included, as well as with or without comments. Regardless of the XML canonicalization algorithm, the following [EXI Fidelity Options] must be enabled to ensure that the relevant Infoset items are available to the signature verifier: Preserve.pis, Preserve.prefixes, and Preserve.lexicalValues. When EXI is used as the on-the-wire encoding and comments are preserved by the canonicalization algorithm, then the fidelity option, Preserve.comments must also be enabled. There may be cases for which the preservation of lexical values is not required, but this is not a general rule. For example, both "true" and "1" are in the lexical space of an xs:boolean and an EXI processor is free to pick either one at decoding time. Similarly, there are cases in which the lexical representation of an xs:float is not preserved when encoded as EXI float.
XML Encryption defines a mechanism by which elements or their content can be replaced by a new element containing a cipher and meta-data describing how the cipher was generated. The cipher generation process requires the transformation of the XML items into a sequence of octets. The cipher that results from an encryption step is a binary sequence of octets that is encoded in base64. Although XML canonicalization is not a requirement to produce these octets, many XML encryption processors rely on exclusive XML canonicalization in order to reduce the amount of contextual information in the fragment being encrypted. This process simplifies not only the fragment but also the process by which it can be re-inserted (re-enveloped) in a different context.
Transmitting self-contained fragments as ciphers in base64 format (and their related meta-data) using EXI as the on-the-wire encoding is not affected by any of the EXI fidelity options. Unlike the XML signature case, confidentiality requires the data being secured to be removed from the Infoset that is ultimately encoded using EXI.
For certain applications, transport level security is an alternative to XML security. In transport level security, the mechanisms that provide integrity, confidentiality and authentication are not part of the message but are instead built into the transport layer. The most popular solution is HTTP over TLS, also known to as HTTPS. Although HTTPS is commonly used to provide confidentiality (encryption) between two parties, it also includes capabilities for integrity and authentication. Naturally, since the transport layer operates at the level of octets, there is no distinction between XML packets and EXI packets. Thus, the use of transport level security is an alternative for EXI in the same way it is for XML.
Although simpler and more efficient, transport level security is not as versatile as the message level security offered by XML Security. Message level security has a number of advantages over transport level solutions in that security information, as an intrinsic part of a message, can persist over the lifetime of a connection. Thus, for example, secured messages can be copied to secondary storage and their properties (integrity, authentication and confidentiality) verified at later point in time. Moreover, message level security enables system architectures wherein messages flow through multiple nodes before reaching their destination, that is, architectures that are not just point to point.
To summarize, the choice of message security over transport security depends on the application in question. It is often a trade-off between processing efficiency and flexibility. Regardless of the selection, there are ways of integrating EXI to enable message interchange in a secure manner.
Use caution when employing EXI in Mixed XML/EXI Environment (options in order of preference):
Don't send EXI to recipients unless they want it/ask for it/can use it (may be agreed ahead of time).
Negotiate EXI content via protocols like HTTP.
Identify content by examining file/stream before feeding to an XML processor.
Let the XML processor produce an error when fed EXI.
Use the Distinguishing Bits to distinguish between XML and EXI content before deciding on which processor to use. Do not send EXI content to an XML processor.
The seamless introduction of EXI in existing environments is a major goal. To non interrupt existing applications on one hand and to facilitate the use of EXI on the other hand Distinguishing Bits were introduced.
The EXI Distinguishing Bits, a leading bit sequence ("10"), are a non optional part of the EXI Header. They distinguish EXI from any well-formed XML in any conventional character encoding [2]. They may therefore be used as a reliable distinguisher, even in cases where no external type content information is available.
Note that XML processors do not give standardized error messages when parsing non-XML content - such as EXI - and it is therefore recommended to check for EXI's distinguishing bits before feeding data to an XML parser.
| Editorial note | |
| The EXI WG is thinking about introducing a more powerful "header" to distinguish EXI from more than XML. | |
Do not use per-application MIME types or filename extensions for EXI-encoded XML data in a mixed XML/EXI environment, to allow a generic EXI processor to transform EXI into equivalent XML.
| Editorial note | |
| This item is still under discussion within the WG. The current thinking is that, in a mixed XML/EXI environment, an EXI processor should have the means to transform the content back into its XML form. | |
Use HTTP content negotiation to determine whether a recipient can process EXI. Do not send EXI to recipients unless they specifically request it.
EXI can naturally be used to more efficiently transfer XML resources over [HTTP 1.1]. Since EXI - by its very definition - will retain the full identity and structure of any XML document, it is suitable as an HTTP transfer encoding. This enables HTTP's built-in content negotiation features to ensure that clients which understand EXI can transparently take full advantage of it, without forcing any clients or servers without such capability or requirement to change.
| Editorial note | |
| The following represents the current thinking of the EXI WG and has not been blessed by the IETF, which is responsible for maintaining the HTTP protocol and the allowed content encodings. So neither the encoding identifier nor the actual use of EXI for an HTTP content encoding can presently be considered final. | |
The presumed standard pattern for an EXI-enabled client would therefore
be to include exi in the Accept-Encoding:-Header of an HTTP GET, POST or
PUT request. If the server agrees to use EXI, it will put a suitable
Content-Encoding:-Header into the reply.
Mock-up GET request:
GET /XML/Group/EXI/docs/best/exi-best-practices.xml HTTP/1.1 Host: www.w3.org Accept:text/xml,application/xml,application/xhtml+xml,text/html;q=0.9, text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Encoding: x-exi,gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Mock-up reply:
HTTP/1.1 200 OK
Last-Modified: Mon, 01 Oct 2007 20:34:06 GMT
Accept-Ranges: bytes
Content-Length: 7360
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Content-Encoding: x-exi
Content-Type: application/xhtml+xml; charset=utf-8
{EXI data goes here...}
Some situations may require a pre-agreement between client and server, for example if an initial PUT or POST request contains large amounts of data. In this case, the HTTP OPTIONS method can be used by a client to inquire ahead of time about the availability of EXI as a transfer encoding.
If a server receives a request for an EXI-encoded resource that it cannot fulfill, it is expected that the server will return a status code 406 (Not Acceptable).
| Editorial note | |
| The EXI WG intends to discuss the use of EXI within "AJAX" environments with the respective working groups, to ensure that those clients that choose to implement EXI as HTTP content encoding would do so in an API-compatible and backwards-compatible manner. | |
| Editorial note | |
| This section will recommend best practices for sharing schemas used for EXI encoding/decoding. | |
Use EXI pluggable codecs only when there is explicit agreement between the communicating parties and not for encoding documents intended for general recipients. Using pluggable codecs without prior understanding and agreement will adversely affect interoperability.
Pluggable codecs are very useful in closed systems, but their use is strongly discouraged beyond the circle of the pre-supposed parties.
If pluggable codecs must be employed, standardization of codecs within the applicable communities is strongly encouraged.
| Editorial note | |
| This section will recommend best practices for using pluggable codecs for EXI encoding/decoding. | |
| Editorial note | |
| This section will discuss recommended approaches for coordinating and documenting encoding options (including schemas and pluggable codecs) which are pre-agreed, rather than declared or negotiated at the time of the information exchange. | |
Provide EXI to XML translation as a "View Source" capability for debugging purposes.
"XML is text, but isn't meant to be read" [XML In 10 Points]. Taken in context, this statement refers to the advantage offered by a textual format like XML. While XML is text that people shouldn't normally have to read in unprocessed form, it can be viewed with a compatible [3] text editor. This is particularly useful in cases where the text gives a comprehensive view of XML structure and data, without needing to reference this information via an API. In effect, this provides a "view source" capability for debugging purposes.
The XML expression of an EXI document is also useful for debugging purposes. On occasion, application developers may need a human-readable form to get a comprehensive view of structures and/or data which may be causing problems. The EXI format is designed such that EXI documents may be translated into equivalent XML documents. After translation, the resulting XML can be viewed in a text editor, if needed. This translation capability may also used in conjunction with other commonly available tools, including web browsers.
It is recommended that whenever practical, EXI-capable system maintain an ability to process textual XML data, and allow it to be switched back and forth between EXI and XML. This has in the past proven useful is similar systems for debugging purposes, as it allows to easily determine whether any occurring problems are due to a faulty EXI deployment or whether such problems occur similarly in a pure XML setting as well.
This document is the work of the Efficient XML Interchange (EXI) WG.
Members of the Working Group are (at the time of writing, sorted alphabetically by last name):
The EXI Working Group would like to acknowledge the following former members of the group for their leadership, guidance and expertise they provided throughout their individual tenure in the WG. (sorted alphabetically by last name)
Except for detached signatures where the document being signed isn't altered.
These "conventional Character encodings" include all required encodings (UTF-8, UTF-16) and every other commonly used encoding known to the Editors at time of publication, including but not limited to UCS-2, UCS-4, EBCDIC, ISO 8859, Shift-JIS, EUC, and GB18030.
That is, compatible with respect to the character encoding used in the text file to be viewed.