W3C

Efficient XML Interchange (EXI) Profile for limiting usage of dynamic memory

W3C Recommendation 09 September 2014

This version:
http://www.w3.org/TR/2014/REC-exi-profile-20140909/
Latest version:
http://www.w3.org/TR/exi-profile/
Previous version:
http://www.w3.org/TR/2014/PR-exi-profile-20140506/
Editors:
Youenn Fablet, Canon Research Centre France
Daniel Peintner, Siemens AG

Please refer to the errata for this document, which may include normative corrections.

See also translations.


Abstract

This document describes a set of parameters that can be used to define profiles of the EXI 1.0 format suited to applications involving devices with dynamic memory constraints.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This is the W3C Recommendation of the Efficient XML Interchange (EXI) Profile for limiting usage of dynamic memory specification. It has been produced by the Efficient XML Interchange Working Group as part of the W3C XML Activity.

This document has been reviewed by W3C Members, by software developers, and by other W3C groups and interested parties, and is endorsed by the Director as a W3C Recommendation. It is a stable document and may be used as reference material or cited from another document. W3C's role in making the Recommendation is to draw attention to the specification and to promote its widespread deployment. This enhances the functionality and interoperability of the Web.

The list of changes since Candidate Recommendation is exhibited in the Change Log. A diff-marked version against the CR version of this document is also available.

The EXI Working Group has produced a test suite and an implementation report.

Please report errors in this document to the public mailing list public-exi-comments@w3.org mailing list (Archives). When preparing comments to send in, please provide a separate email message for each distinct issue to the extent possible. It is inappropriate to send discussion email to this address.

The goals of the Efficient XML Interchange (EXI) Format are discussed in the Efficient XML Interchange (EXI) Format 1.0 document. The EXI Profile is intended for low-resource or "ultra-constrained" devices. Such devices lack run-time memory allocation capabilities or at best have extremely limited dynamic memory resources. The authors of this document are the members of the Efficient XML Interchange Working Group.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 14 October 2005 W3C Process Document.

Table of Contents

1. Introduction
2. Outline
3. Grammar Capping
    3.1 Grammar Learning Disabling Mechanism
    3.2 Grammar Learning Disabling Parameters
4. Local Value Capping
5. Parameters representation
6. Conformance
    6.1 EXI Profile Stream Conformance
    6.2 EXI Profile Processor Conformance

Appendices

A References
B Guidelines (Non-Normative)
C Header Considerations (Non-Normative)
    C.1 General Considerations
    C.2 Implementation Strategies
    C.3 Element exi:p integration
D Prefix Workarounds (Non-Normative)
E Name Table Workarounds (Non-Normative)
    E.1 Name Table Encoder Workarounds
    E.2 Name Table Decoder Workarounds
F Grammar Restriction Considerations (Non-Normative)
    F.1 Grammar Restriction Encoder Considerations
    F.2 Grammar Restriction Decoder Considerations
G Workaround for Even More Restricted Devices (Non-Normative)
H Specification Changes (Non-Normative)
    H.1 Changes from Candidate Recommendation


1. Introduction

Many device classes and use-cases desire to use EXI as its exchange format. Due to various restrictions, some of those application areas are not capable or allowed to require arbitrary memory growth at runtime. Though EXI provides a number of format options that help to constrain runtime memory usage, there are aspects of memory use that are left open-ended in EXI format, the consequence of which are deemed to be generally of less severity. However, each EXI application needs to confront the need to cope (or purposefully not to cope) with those loose ends in order for low-resource devices to participate in the application natively.

There is a known practice used by some applications to address this issue by requiring to send schema-valid EXI documents consisting only of schema-declared elements to low-resource devices thereby not incurring any dynamic mutation of EXI grammars (see G Workaround for Even More Restricted Devices). It is a valuable practice that is applicable to a wide set of use cases involving low-resource devices.

However, certain evaluations of EXI in the context of such areas exposed use cases with additional requirements that are not practically attainable by such a practice alone while honoring the limited threshold of each device's memory usage.

This EXI profile document specifies rules to ensure that the memory restrictions are respected while keeping compatibility with the EXI 1.0 specification. Section 3. Grammar Capping defines the mechanisms and parameters defined to limit the grammar learning. Section 4. Local Value Capping defines the mechanisms to simplify value indexing. Section 5. Parameters representation defines how the parameters defined in the profile can be represented as part of the EXI header.

To keep EXI 1.0 compatibility, the EXI profile does not provide a specific mechanism to bound the memory used for name tables. The working group discussed strategies and rules that allow EXI processors to overcome this issue. The appendix section (D Prefix Workarounds, E Name Table Workarounds) describes some of these implementation strategies and best practice rules.

2. Outline

The arbitrary memory growth at runtime when processing EXI streams is due to indexing QNames and string values for frequent use (string tables growth) and adapting the processing strategy to unknown input in case of schema-less encoding or deviations from the schema (built-in element grammar generation and evolution). The EXI Profile provides mechanisms to restrict the growth of the string tables and the grammar evolution in order to predictably limit the memory growth at runtime. Limiting the size of the string table and the grammar evolution bounds the memory growth for encoding unknown input such as schema deviations but may lead to less compact EXI streams.

Some mechanisms to lower the memory usage are already available in the EXI 1.0 specification and should be considered in conjunction with the EXI profile (see B Guidelines). For example the EXI specification provides a mechanism to limit the memory growth due to global value string tables by using the valuePartitionCapacity. The local value string table can only be disabled if the global value string table is also disabled by setting the valuePartitionCapacity to 0. The mechanism described in Section 4. Local Value Capping allows for disabling the local value table even if valuePartitionCapacity is greater than 0.

According to the EXI 1.0 specification, an EXI processor creates a new built-in element grammar for each unknown element it encounters. The EXI profile limits the number of built-in element grammars created at runtime by using schema informed complex ur-type grammars instead. Each built-in element grammar that is allowed to be created by the EXI profile processor can further evolve by insertion of new productions and hence leads to memory growth during runtime. For that reason, the EXI profile defines a mechanism to limit the number of dynamically inserted grammar productions in the built-in element grammars if such are allowed during processing.

The disabling of the local value table, the number of built-in element grammars created at runtime and the number of dynamically inserted grammar productions are all controlled by a set of Profile parameters described in Section 5. Parameters representation.

3. Grammar Capping

To disable grammar learning, the xsi:type attribute may be used to switch from an evolving built-in element grammar to a non evolving schema-informed grammar. In particular, the xsd:anyType complex type can be used to represent arbitrary XML elements.

Note that the EXI profile can only limit grammar learning for schema-informed EXI streams but not for schema-less EXI streams. In the case where no schema is available, the "schemaId" element may be set to the empty value, so that all grammars derived from the built-in XML Schema types become available through xsi:type grammar switching, in particular the grammar corresponding to the xsd:anyType complex type.

Several prefixes are used throughout this document to designate certain namespaces. The bindings shown below are assumed, however, any prefixes can be used in practice if they are properly bound to the namespaces.

PrefixNamespace Name
exihttp://www.w3.org/2009/exi
xsd http://www.w3.org/2001/XMLSchema
xsihttp://www.w3.org/2001/XMLSchema-instance

3.1 Grammar Learning Disabling Mechanism

For a given element E, the disabling of grammar learning E is done by inserting an xsi:type attribute event with the xsd:anyType value after the SE event of the element E. Note that NS events may appear between the corresponding SE event and the inserted xsi:type attribute. If an element E already has an xsi:type attribute and grammar learning is disabled for the grammar representing the element E, the xsi:type attribute value MUST refer to a known schema-informed grammar that can represent the given element.

If grammar learning is disabled for an element E in the case of a new built-in element grammar G, the following rule happens:

  • An xsi:type attribute event, if not already present, is inserted with the xsd:anyType value after the SE event representing this element. The xsi:type attribute event MUST always be represented by the AT(*) production whose event code length is 2.

  • For all elements following the element E in the EXI stream, that have the same QName as the element E and are represented by a built-in element grammar, an xsi:type attribute event, if not already present, is inserted with the xsd:anyType value after the corresponding SE event. The xsi:type attribute event MUST always be represented by the AT(*) production whose event code length is 2.

A new grammar G for an element E is expected to be instantiated at the time the production of the element E SE event is evaluated. Even in the case where no production will be inserted in grammar G, implementations MUST behave as if grammar G is instantiated.

In a case where this first mechanism is not active and grammar learning must be disabled, a second mechanism is made possible as described below. As the cost of this second mechanism is generally higher than the first mechanism, this second mechanism SHOULD only be used if the first mechanism cannot be used. That said, implementations are free to use any of the two mechanisms and hence also need to be prepared to deal with the associated effects (e.g., diverse EXI events and event code lengths).

If grammar learning is disabled in the case of a production insertion in a given grammar, named G, the following rule happens:

  • All events to be encoded that can be represented by top-level productions already inserted in the grammar G are represented by the corresponding top-level productions.

  • All events of the given element E that remain to be represented are represented using productions whose event code length is 2. Note that in such a case, the EXI processor must ensure to increment the productions event code according the rules defined in section 8.4.3 of the EXI 1.0 specification but does not need to create and insert the top-level productions. In particular, the EXI processor may need to keep track whether a CH production is already inserted or not in the grammar G to keep the number of top-level productions consistent with the rules defined in section 8.4.3 of the EXI 1.0 specification.

  • For all elements following this element E in the EXI stream and represented by the grammar G, an xsi:type attribute event MAY be inserted with the xsd:anyType value directly after the corresponding SE event. The xsi:type attribute event MUST always be represented by the AT(*) production whose event code length is 2.

Once production insertion is disabled for a given grammar, the grammar use is restricted so that only the productions already inserted and the number of top level productions needs to be stored for that grammar. It should be noted that even if production insertion is disabled for a given grammar, the productions inserted before production insertion is disabled for that grammar may be used throughout the whole document. Additional information, in particular with regards to implementation strategy, technical reasoning and impact is available in the appendix section F Grammar Restriction Considerations.

3.2 Grammar Learning Disabling Parameters

To limit increasing memory consumption due to grammar learning, the EXI profile enables to limit the number of evolving built-in grammars and the number of inserted productions. Two parameters are defined for that purpose:

[Definition: The maximumNumberOfBuiltInElementGrammars parameter is the maximum number of QNames for which built-in element grammars can be dynamically instantiated. ]

[Definition: The maximumNumberOfBuiltInProductions parameter is the maximum number of top-level productions that can be dynamically inserted in built-in element grammars. ]

Grammar learning is disabled for an element E for which a new built-in element grammar G must be created if the following condition is true:

Grammar learning is disabled in the case of a production insertion if the following condition is true:

  • The sum of the number of dynamically inserted top level productions of all built-in element grammars is equal or greater than the maximumNumberOfBuiltInProductions value.

Note that only the productions of the first maximumNumberOfBuiltInElementGrammars instantiated grammars are counted, the sum of which being compared with the maximumNumberOfBuiltInProductions value. In particular, the AT(xsi:type) productions that would be inserted in grammars that would be instantiated after the maximumNumberOfBuiltInElementGrammars threshold are not counted. This allows production insertion to happen even though no new grammar can be instantiated.

Note also that when the maximumNumberOfBuiltInProductions value is reached, it may often be more compact to insert an AT(xsi:type) event with a xsd:anyType value on all elements for which a new built-in element grammar must be created.

Whenever the parameters are set, the rules above MUST be properly applied. The parameters may be left unspecified in the case of an application that wants to define a finer-grained control on the application of the grammar learning disabling mechanism. In such a case, the EXI processor will not set the two parameters defined and should use an out-of-band mechanism to convey the precise grammar disabling strategy in use. It may for instance be beneficial to apply grammar learning for elements that occur frequently and that are regular while disabling grammar learning for elements that occur rarely.

4. Local Value Capping

Some classes of EXI processors may not afford the cost of building local value table representations. This profile defines a parameter that can disable the use of local value references. Global value indexing may be controlled using the options defined in the EXI 1.0 specification.

[Definition: The localValuePartitions parameter is a Boolean used to indicate whether local value partitions are used. ] The value "0" indicates that no local value partition is used while "1" represents the behavior of the EXI 1.0 specification.

When the value localValuePartitions is set to "0", it is an error to represent a string value as a reference to an entry of a local value partition.

Some processors may decide to have fine grained strategy on the local value tables building and usage. For instance, some processors may decide to use local value references based on the QName of the element. In such a case, the localValuePartitions should not be set to any value and the fine-grained strategy should be exchanged by an out-of-band mechanism.

5. Parameters representation

The use of the EXI profile is advertised by encoding an exi:p XML element in the user-defined meta-data section of the EXI options of an EXI stream (see example below). The content of the exi:p element indicates the value of the three parameters of the profile, encoded using a single decimal value. Each profile parameter is represented as follows:

  1. The localValuePartitions parameter is encoded as the sign of the decimal value: the parameter is equal to 0 if the decimal value is positive and 1 if the decimal value is negative.

  2. The maximumNumberOfBuiltInElementGrammars parameter is represented by the first unsigned integer corresponding to integral portion of the decimal value: the maximumNumberOfBuiltInElementGrammars parameter is unbounded if the unsigned integer value is 0; otherwise it is equal to the unsigned integer value - 1.

  3. The maximumNumberOfBuiltInProductions parameter is represented by the second unsigned integer corresponding to the fractional portion in reverse order of the decimal value: the maximumNumberOfBuiltInProductions parameter is unbounded if the unsigned integer value is 0; otherwise it is equal to the unsigned integer value - 1.

An exi:p element is always represented using the following lists of EXI events in a EXI header option element:

Find below an example EXI options document with the most restrictive EXI profile parameters settings (all three parameters, namely localValuePartitions, maximumNumberOfBuiltInElementGrammars, and maximumNumberOfBuiltInProductions are set to 0 (zero)).

Example 5-1. EXI Profile document for most restrictive parameters settings
<exi:header xmlns:exi="http://www.w3.org/2009/exi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <exi:lesscommon>
        <exi:uncommon>
            <exi:p xsi:type="xsd:decimal">
                1.1
            </exi:p>
        </exi:uncommon>
    </exi:lesscommon>
</exi:header>

Inserting an exi:p element within the metadata section is invalid according the EXI Options Document XML Schema defined in Efficient XML Interchange (EXI) Format 1.0. Nonetheless, EXI processors are able to represent such an exi:p element. As described in C.3 Element exi:p integration, losing schema validity was deemed acceptable given the desire to achieve a compact representation of EXI options elements.

6. Conformance

6.1 EXI Profile Stream Conformance

[Definition: A conformant EXI profile stream consists of a sequence of octets that follows the syntax of a conformant EXI stream that is defined in the document Efficient XML Interchange (EXI) Format 1.0 and also follows the additional rules defined in this document according to EXI profile parameters. ] By definition a conformant EXI profile stream constitute also a conformant EXI stream.

6.2 EXI Profile Processor Conformance

The conformance of EXI Processors is defined separately for each of the two processor roles, EXI profile stream encoders and EXI profile stream decoders. The conformance of EXI profile stream encoders is described in terms of the conformance of the EXI profile streams that they produce. The conformance of EXI profile stream decoders is based on the set of supported format features for processing conformant EXI profile streams.

[Definition: The term EXI parameter set consists of specific values or range of values for EXI options and EXI profile parameters.] An EXI parameter set constraints the EXI options and EXI profile parameters used to encode the body of the EXI stream. An EXI parameter set can be defined for both, EXI profile stream encoders and EXI profile stream decoders.

An EXI profile stream encoder is a fully conforming EXI profile stream encoder if and only if it is capable of generating conformant EXI profile streams using any EXI parameter set, given any input structured data it is made to work on. On the other hand, a fully conforming EXI profile stream decoder MUST support any EXI parameter set. A partially conforming EXI profile stream encoder and/or a partially conforming EXI profile stream decoder, in turn, MUST support at least one EXI parameter set. In practice, standard organizations and/or other communities may specify and also label their appropriate EXI parameter set.

A References

Efficient XML Interchange (EXI) Format 1.0 (Second Edition)
Efficient XML Interchange (EXI) Format 1.0 (Second Edition), John Schneider, Takuki Kamiya, Daniel Peintner, Rumen Kyusakov, Editors. World Wide Web Consortium. The latest version is available at http://www.w3.org/TR/exi/. (See http://www.w3.org/TR/2014/REC-exi-20140211/.)

B Guidelines (Non-Normative)

This document specifies means to ensure memory restrictions for various use-cases. Doing so guarantees support for a broad range of application fields. However, it is advised to use the most restrictive parameters settings when the recipients capabilities are not known.

The suggested profile parameter settings for unknown recipients are:

  1. Set maximumNumberOfBuiltInElementGrammars to 0 (zero).

  2. Set maximumNumberOfBuiltInProductions to 0 (zero).

  3. Set localValuePartitions to 0 (zero).

    Note:

    The localValuePartitions parameter does not have any effect if either valueMaxLength and/or valuePartitionCapacity is set to 0 (zero).

In regard to restricted device classes it may be sensible to also investigate into the right EXI1.0 options such as:

  1. strict

  2. preserve

  3. valueMaxLength and valuePartitionCapacity

  4. blockSize

Note that if the recipients capabilities are known, these known capabilities should be used to offer the best possible compression and processing performance.

C Header Considerations (Non-Normative)

C.1 General Considerations

The processing of the EXI header may require memory allocation to handle grammar learning and value partition. It is advised that the processing of the EXI header does not go over the memory requirements used for the EXI body processing. In particular:

  1. Built-in grammar learning should be kept to a minimum, optimally not requiring the storage of an element grammar besides the scope of this element.

  2. String partitions should not be needed to correctly parse the corresponding header.

C.2 Implementation Strategies

Depending on the use-case various strategies are feasible to properly parse the EXI Options of the header. Note that a full EXI Options parser is not needed nor sensible in any case.

For device classes that support only one (or a restricted set) set of EXI options it may be possible to follow the approach of bit matching. Bit matching means that the received header could be compared to a known option set that is supported. If the expected “bits” match the decoder is able to process the EXI body. If the bits don’t match the stream is encoded with unsupported options.

A more sophisticated approach is to realize a partial EXI Options grammar parser. Let’s assume an EXI processor supporting profile parameters or strict encodings. One way to implement a partial parser could be as follows:

Example C-1. Partial EXI Options Parser
/* Pseudo Code */
int nbit = readNBit(1);
if( nbit != 0 ) {
    /* SE(*) in root element */
    return false;
}
/* header */
nbit = readNBit(2);
switch( nbit ) {
case 0:
    /* lessCommon */
    nbit = readNBit(2);
    /* 0: uncommon */
    /* 1: preserve */
    /* 2: blockSize  */
    /* 3: EE */
    If ( nbit == 0 ) {
        /* lessCommon */
        nbit = readNBit(3);
        /* 0: alignment */ 
        /* 1: selfContained */
        /* 2: valueMaxLength */
        /* ... */
        /* 5: any, e.g, profile */ 
        if ( nbit == 5 ) {
            /* userDefinedMetaData */
            /* uri hit for "http://www.w3.org/2009/exi": 5 in 3 bits */
            /* local-name miss for "p": 2 in 1 Byte and 112 in 1 Byte */
            /* ... */
        }
    } else {
        /* not supported */
        return false;
    }
    break;
case 1:
    /* common */
    return false;
case 2:
    /* strict, set EXI processor accordingly */
    break;
case 3:
    /* EE, EXI options finished */
    break;
default:
    /* invalid */
    return false;
}  
                    

Depending on the application one of the presented approaches may be feasible. Moreover, other strategies or also a mix between strategies may be sensible.

C.3 Element exi:p integration

Inserting an exi:p element as part of the user-defined metadata section of a EXI options element is invalid according to the XML Schema for EXI Options Document defined in Efficient XML Interchange (EXI) Format 1.0. The exi:p element is matched against a ##other wildcard according the XML schema for EXI Options Document. While the intent of the metadata section is to allow any possible content, the target namespace of the related wildcard cannot be '##any' due to XML schema specific rules.

Using the EXI namespace to qualify the exi:p element is useful for several reasons:

  • The namespace is already populated by every EXI processor which can parse EXI Options documents.

  • The namespace is reserved for the purpose of EXI applications.

  • The EXI format treats ##other wildcards as ##any wildcards.

These potential benefits, in particular the compactness of the exi:p element, are believed to outweight the exi:p element validation issue.

D Prefix Workarounds (Non-Normative)

It is advised for users that want to preserve prefixes while capping memory requirements to properly set their namespace declarations so that:

  1. QName prefixes are encoded as 0 bit

  2. Prefixes in NS events are always encoded literally

In those conditions, an EXI processor does not need to build the prefix indexing tables. In addition, application layers that provide APIs such as SAX or StAX, often store namespace mapping information. In that case, prefixes can be unambiguously retrieved from their associated namespace URI from the application layer.

E Name Table Workarounds (Non-Normative)

Name tables cannot be restricted without breaking the compatibility with the EXI 1.0 specification. This section describes how an implementation may circumvent this issue by bringing the knowledge of the application to the EXI processor.

E.1 Name Table Encoder Workarounds

The application may declare to the EXI encoder all the QNames it is aware of. Amongst several possible optimizations, this allows the EXI encoder to pre-allocate statically the memory used to store all the possible QNames, be they pre-populated using schema knowledge or not. In addition the string representation of these QNames may also be shared between the application and the EXI processor.

If all the QNames used by the application are declared to the EXI encoder, no name table entry will be dynamically inserted and no additional memory allocation may be needed for the name tables. The EXI encoder will assign indexes dynamically for names that are not pre-populated by the schema knowledge to keep the compatibility with the EXI 1.0 specification. It will also need to keep track of the number of entries that are indexed to compute the binary representation size of entry indexes.

E.2 Name Table Decoder Workarounds

The application may declare to the EXI decoder all the QNames it is interested in. This allows the EXI decoder to pre-allocate the memory used to store these QNames, be they pre-populated using schema knowledge or not. In addition the string representation of these QNames may also be shared between the application and the EXI processor.

When a litterally encoded URI or local name string happens in an EXI stream, the EXI decoder may check whether it is of interest to the application. If so, the EXI processor will be able to assign dynamically to that string an index without allocating any new memory to store this name table entry.

When a URI or local name string happens in an EXI stream that was NOT declared of interest by the application, the EXI processor may have a behavior specified by the application: adding it to the name tables as defined in the EXI 1.0 specification if there is enough memory, incrementing the corresponding name table counter but not storing the string and skipping the corresponding event, raising an error...

Note:

Under the circumstance that local value string entries are not disabled the EXI processor needs to store the number of local value partition entries.

F Grammar Restriction Considerations (Non-Normative)

F.1 Grammar Restriction Encoder Considerations

For a given QName, EXI encoders will generally need to retain some information related to the state of the corresponding built-in element grammar. In particular, it may need to know the following information for each QName that has no associated schema-informed grammar:

  • The number of top level productions of the grammar associated to that QName

  • Whether a xsi:type attribute must be inserted after an SE event of that QName

  • Whether an xsi:type event was already encoded using the grammar of that QName

Note that in the case of no grammar learning at all, this information may be stored as a boolean, representing whether that QName was already encoded as part of a SE event.

In the case of built-in element grammars for which one or more productions were inserted before the grammar learning is disabled and for which a production should have been inserted after the grammar learning is disabled for a given element E, encoders must use second level productions to encode all remaining events of that given element E that cannot be represented using already inserted productions. It should be noted that every AT(*) or SE(*) second level production encoded will cause EXI decoders that are unaware of the EXI profile to insert a production in the corresponding grammar. The production insertion number will grow with the number of attributes or start element events that cannot be represented using already inserted productions. This number may be arbitrarly large. Encoders need to take that into account when encoding documents so that not-EXI profile decoders can still properly decode the documents. For instance, encoders may decide to use xsi:type based grammar switching to limit the size of such a grammar.

F.2 Grammar Restriction Decoder Considerations

Once grammar learning is disabled, the EXI decoder will not need to create any new built-in element grammar. For any such grammar, the EXI decoder can directly deduce the corresponding grammar state from the EXI stream given that the xsi:type attribute is always represented by the AT(*) production whose event code length is 2:

In bit-packed mode:

  • If the first bit after the SE event of a QName is set to 0, the corresponding grammar is a new built-in element grammar with no top-level production at all.

  • If the first bit after the SE event of a QName is set to 1, the corresponding grammar is a built-in element grammar with one top level production, AT(xsi:type).

In byte-aligned mode:

  • The first byte after the SE event of a QName must be equal to 1.

  • If the second byte after the SE event is 1, the corresponding grammar is a built-in element grammar with one top level production, AT(xsi:type).

  • Otherwise, the second byte after the SE event is equal to 3 and the corresponding grammar is a new built-in element grammar with no top-level production at all.

The second part of the encoded production is equal to 1 to refer to a AT(*) production. When prefix preservation is on, the second part of the encoded production may be equal to 2 to refer to NS productions that may appear between SE and AT(xsi:type) events.

In the case of built-in element grammars for which one or more productions were inserted before the grammar learning is disabled and for which a production should have been inserted after the grammar learning is disabled, the EXI decoder may only need to store the number of top-level productions of this grammar in addition to the already inserted productions. It will also need to check whether the top-level productions already include a AT(xsi:type) production. The first part of the encoded production must be equal to the number of top-level productions. The second part of the encoded production must be equal to 1, so as to refer to a AT(*) production.

It is thought as good practice for an EXI profile decoder to strip from the infoset any xsi:type attribute with the xsd:anyType value that corresponds to the grammar learning disabling mechanism.

G Workaround for Even More Restricted Devices (Non-Normative)

In the case of very restricted devices, it is important to evaluate how useful data sent to these devices is. The fact that data can be decoded by these devices (using the EXI profile or not) does not mean that these devices can actually process it. For example, a very simple thermostat may receive "temperature setpoint" data encoded using a standardized schema. Sending additional data (for example, weather forecast information) encoded with or without a dedicated schema does not help the thermostat. It is good practice to refrain from sending to these devices data not defined in the schema. This leads to better use of resources (bandwidth and memory) and simpler implementations.

Such very restricted devices may collocate within a system with fully conformant EXI devices or EXI profile conformant devices. A first option is to use schema-informed strict mode for the very restricted devices and schema-informed default mode with profile for the other devices. A second option is to use the schema-informed default mode for both sets of devices. This reduces the overall complexity and lowers hardware requirements such as ROM/flash sizes. To cope with very restricted devices constraints, it may be prohibited to send data not defined in the schema: this removes the need for these devices to support built-in grammars at all.

H Specification Changes (Non-Normative)

H.1 Changes from Candidate Recommendation

  • The parameter empty exi:p element that has been considered to be at risk has been entirely removed. An empty exi:p element was a syntactic shortcut to a particular exi:p element with a value equal to +1.1.

  • Clarified that a built-in element grammar is said to be instantiated even if no production is actually inserted in it.

  • Clarified that, even though the insertion of a exi:p element within a EXI options element is schema invalid as per EXI header options schema, it can be represented in EXI strict mode and was found implementation-wise.

  • Updated the appendix F.2 to cope with NS events that may appear between SE and AT(xsi:type) events.