Public document·View comments·Disposition of Comments·
Efficient Extensible Interchange Working Group Other specs in this tool
In the table below, red is in the WG decision column indicates that the Working Group didn't agree with the comment, green indicates that a it agreed with it, and yellow reflects an in-between situation.
In the "Commentor reply" column, red indicates the commenter objected to the WG resolution, green indicates approval, and yellow means the commenter didn't respond to the request for feedback.
| Commentor | Comment | Working Group decision | Commentor reply |
|---|---|---|---|
LC-2104
Mohamed ZERGAOUI <innovimax@gmail.com> (archived comment) |
|
One of the advantages of using a base 10 representation is that it avoids rounding issues when moving floating point data between EXI and text XML and between EXI and an application that use the standard XML interfaces. XML, EXI and the standard XML interfaces all use a base 10 representation for floating point numbers, so no rounding issues will occur in these circumstances. You are correct that rounding issues may occur when moving floating point data between EXI and a base 2 representation. These rounding issues will be identical to those that occur moving floating point data between text XML and a base 2 representation, so EXI maintains the same behavior as XML in these cases. As such we avoid introducing any *new* rounding issues. Any work-arounds developed to address rounding issues for text XML will continue to work for EXI. |
no |
LC-2175
FABLET Youenn <Youenn.Fablet@crf.canon.fr> (archived comment) |
|
We feel that EXI should fit smoothly into the XML stack. To avoid any problems EXI supports type information in the same fashion as XML does. The groups intention is not to super-set XML. XML allows one to specify typed-values with the attribute xsi:type. EXI provides the same mechanism and both work on elements only. EXI makes use of XML Schema and its types form the basis for typed values in the entire W3C environment. Introducing new types or another type system is not the scope of our group and might affect XML processing in general. A newly invented typing system would be processed in the context of EXI only. Such type information would be lost when documents are converted from EXI to XML then back to EXI. This may cause an issue, since it may look like XML cannot preserve what EXI is describing. The solution for arbitrary types in EXI (as well as in XML) is using schema information. Please note that partial schema information are sufficient for EXI. In the use-case you provided a ds:SignatureValue element with the according type information for content and attributes achieves the desired result. An EXI processor picks the given type information for element content as well as the attributes. |
tocheck |
LC-2164
ISHIZAKI Tooru <ishizaki.tooru@canon.co.jp> (archived comment) |
|
You are correct in that the four options you present are the only possible ones from the alignment-compression pair. The reason why alignment and compression are separated as options is to achieve better compactness for the EXI Options header. It is expected that EXI compression is used often, so it is placed in the "common" part of the Options header, whereas the non-default alignment options are expected to be used only in very special circumstances and are therefore placed in the "uncommon" part. This reduces the size of the Options header in the usual case when the default alignment is used. But none of this prescribes any particular implementation strategy. An implementation is free to adopt the approach that you suggest, including in its API, as long as it produces and is able to consume correct Options headers. |
tocheck |
LC-2166
ISHIZAKI Tooru <ishizaki.tooru@canon.co.jp> (archived comment) |
|
Value qnames are represented as strings in EXI format. Because prefixes used in an instance and its corresponding schema are declared in each document independently from the other, enumerated values in schema are of little use for types derived from xsd:QName since value qnames in EXI streams are strings. Note, however, that markup qnames which occur as the names of elements and attributes are represented using built-in QName datatype representation [1]. The rationale for the use of strings as the representation of value qnames is primarily so as to avoid the anticipated processing efficiency overhead that is likely to be involved if we used built-in QName datatype representation for value qnames. This concern of processing efficiency overhead is particular to value qnames, and is not foreseen for markup qnames. Therefore we use built-in QName datatype representation for markup qnames. |
yes |
LC-2103
Mohamed ZERGAOUI <innovimax@gmail.com> (archived comment) |
|
Thank you for your suggestion regarding the order of the bits in the 2-bit unsigned Integer representation used to distinguish between lexical variances of the Boolean type. The current representation allows bit-testing of the first bit to provide the value of the Boolean. It is not clear that changing the representation to allow bit-testing of the second bit rather than the first bit would provide a notable benefit. Both representations seem equally capable, so our current plan is to keep the current representation. Please let us know if we've misunderstood some aspect of your suggestion. |
tocheck |
LC-2106
Mohamed ZERGAOUI <innovimax@gmail.com> (archived comment) |
|
When initially discussing the valuePartitionCapacity option, the working group analyzed several different potential methods for treating new strings when the maximum capacity is reached. These methods included Move to front, which you proposed. The group does not believe that Move to front can be sufficiently efficient to serve as the replacement method, based on analysis of the possible algorithms in this context. The string table is used constantly when processing an EXI document, so it needs to be as efficient as possible, and we do not believe that the potential improvement in compactness is worth the amount of decreased processing efficiency that Move to front is estimated to result in. |
tocheck |
LC-2187
TAMIYA Keisuke <tamiya.keisuke@canon.co.jp> (archived comment) |
|
We understand the concern you have raised over the issue grammar learning behaviour may pose on the size of memory EXI processor consumes and requires. While acknowledging it is a potential issue, we had stopped short of adding a mechanism to restrain grammar learning at the time we dicussed the feature a while back, based on implementation report of the feature shared by the WG memebers, which suggested otherwise that the issue did not surface in real deployments. One of the traits that documents for use in exchange in general is that grammars are learned a lot up front at a pace close to linear rate, and the learning becomes less and less frequent, eventually saturating to a convergence. This observation helps explain why the issue is not very likely to emerge in practice. Based on these observation and analysis, as well as the consideration of the cost that would otherwise be incurred if a restraining mechanism was introduced both in terms of compactness and runtime efficiency, we stay cautious and may require further arguments involving specific use cases before we consider a way to alleviate the potential issue. |
yes |
LC-2190
TAMIYA Keisuke <tamiya.keisuke@canon.co.jp> (archived comment) |
|
The EXI format itself offers only a hook to support selfContained subtrees, meaning that an EXI processor may access such an independent fragment without reading what came first. The working group expects many different use cases for this feature and does not restrain its use to a specific mechanism which is likely not be suitable for many environments. It is therefore up to an application or system to make use of this feature and build an index or any other solution on top of it. |
tocheck |
LC-2171
Thomas Hornig <t.hornig@highQ.de> (archived comment) |
|
The group believes that EXI should integrate well with the family of XML technologies, and therefore we would rather not define EXI-specific encryption or digital signature methods but use the existing XML Encryption and Signature specifications, applying them to EXI by using their defined extension points. As you note, the recommended rule is to first compress (or encode in EXI) and then encrypt. Accomplishing this with XML Encryption requires some additional specification work, but very little. We have already talked with the XML Security working group on the best way to accomplish this and believe we understand how to do it in a way that fits into the XML Encryption specification. After this work is done, we will publish it in one of our documents (we have not yet determined which one). |
tocheck |
LC-2168
UCHIDA Hitoshi <uchida.hitoshi@canon.co.jp> (archived comment) |
|
We understand that the IEEE float representation is one of the most frequently requested changes to the EXI specification. We would like to use this opportunity to better explain our position on this issue and share with you the rationale that supports it. As you will find outlined below, the WG had assessed both its advantages and disadvantages in the context of EXI, as well as alternative ways to assuage the concerns. The WG came to believe that the use of IEEE float fits better with the user-defined datatype representation capability [1] of EXI than is integrated as an additional mandatory physical representation of character information items that are typed as xsd:float or xsd:double. There are some aspects of IEEE float that do not lend natureally themselves to the primary role of EXI as an efficient exchange format of XML, particularly when EXI is required to remain intrinsically compatible with existing XML family of standards written to XML infoset, such as XML signatures. The issues the WG looked at in particular are, less compactness and less amenability to compression, as well as rounding issues in serialization (base-2 to base-10) and the cost of stringification required when used with the typical XML APIs (SAX, DOM, etc.) which are undeniably predominantly text-based. However, by saying this, it is not our intention to derogate the value nor it is to neglect its wide-spread use in diverse range of applications. We do not intend to dispute the merit IEEE float brings to some applications, and understand EXI might be seen as the perfect opportunity to see that merit in a way fitted well to XML technologies. Though the EXI spec calls for a deliberate assessment when considering the use of user-defined datatype representation, obviously IEEE is one of such cases that warrant the use of the facility without any question. It was not an easy discussion within the WG that led to the position mentioned above. Some argued EXI is not XML enough, and others contended EXI as not binary enough. We respected both views to find a difficult balance and better serve both needs. In a way, we do not see EXI as static, rather it is a core foundation on which things can be further built to suit use cases, which will make the use of EXI more perfect. The two notable points of innovation provided by EXI are user-defined datatype representation, and the provision that permits the definition of schema binding for other schema languages. We therefore encourage you to seek the best way to leverage user-defined datatype definitions to represent float and double data as IEEE floats. If you find any suggestions that would make it easier to use IEEE float through user-defined datatype representation, we would love to hear them. Such suggestion may well find its way into a collateral documents such as FAQ or Best Practices. [1] http://www.w3.org/TR/exi/#datatypeRepresentationMap |
tocheck |
LC-2169
UCHIDA Hitoshi <uchida.hitoshi@canon.co.jp> (archived comment) |
|
http://lists.w3.org/Archives/Public/public-exi-comments/2009Sep/0003.html | tocheck |
LC-2170
UCHIDA Hitoshi <uchida.hitoshi@canon.co.jp> (archived comment) |
|
EXI already provides one feature that can be used to achieve the functionality you request. Namely, it is possible to encode a series of XML documents as an EXI fragment, which will retain both the string table and any learned grammar content between individual documents of the series. In general, the working group believes that being able to preserve any learned content from one encoded document to the next is primarily applicable to situations where there is already an externally-defined ordering among the documents, such as over a communication channel that guarantees in-order delivery. For such situations, the solution using fragments and relying on the external ordering seems adequate, and thus we have no intention of defining an ordering internal to EXI. |
yes |
LC-2133
Yuri Delendik <yury_exi@yahoo.com> (archived comment) |
|
The XML Schema mapping to EXI grammar terms described in section 8.5.4 [2] is a normative mapping. A conformant EXI decoder must support this specified mapping, as stated in the conformance section [3]. As noted in your comment, it is possible to specify mappings from other schema languages. Requiring all conformant EXI processors to implement the W3C XML Schema mapping feature will facilitate interoperability, and we picked XML Schema because it is a W3C Recommendation. We do not intend to make any judgement or preference over various schema languages. Other mappings might be done by the EXI Working Group (or another group) in the future but it is not planned at this time. For those reasons, as well as since we do not see real semantic benefit by the changes, we feel that this section is worth staying in the main body of the specification. A second part of your comment suggests that we use EXI grammar terms to describe EXI Options. We will make it clear that the XML schema for Options is provided for clarity of the specification, and it is not required for the EXI processors to read an Options schema. Therefore we think that adding an equivalent EXI grammar, which would be more verbose, is not needed. The last part of your issue is about the use of "strict" mode. The Working Group believes that the use of the strict vs. non-strict mode is not equivalent to the use of a different schema. The processor uses the same schema but handles deviations from that schema when in non-strict mode. This enables an use case where sender and receiver agrees to a schema, and still the encoder has a liberty of arbitrarily using "strict" option instance-by-instance basis, without pre-agreed schemaID. |
tocheck |
LC-2108
<pub@upokecenter.com> (archived comment) |
|
We excluded those regular expressions that contain either wildcard (".") or negative character groups, since those expressions tend to result in large set of characters. Even those rare cases that are not the case often have better alternative ways to specify the same effect, such as [^0-] can be expressed otherwise simply as [�-/]. On the other hand, character class subtraction is retained because, unlike wildcard or negative character groups, the operation always results in a number of characters smaller than that of the 1st operand. We also found that character class subtraction does not add much computational burden if it is properly implemented. Please also note that it is our expectation that schema authors can provide some help by being aware of the general cost of each operation and specifying patterns in ways more friendly to EXI processing. |
tocheck |
LC-2172
FABLET Youenn <Youenn.Fablet@crf.canon.fr> (archived comment) |
|
The EXI encoding of simple type data, and the use and disuse of certain facets has been decided on the basis of both empirical observation of merits and its implications. We had looked at facets related to length to see whether EXI should be aware of those facets to improve the simple type value encodings. The result of implementation experiment shared by WG members indicated that the effects of leveraging those facets were not substantive enough to make it convincing to include the function into the format specification, given that the addition of which implies for EXI processors to check the presence of those facets for every occurrence of schema-bound strings before determining the method of representing the length field. In this particular incident, the concerns about the potential processing efficiency penalty outweighted the benefit observed. Encoding rules associated with strings, such as string tables, are so defined as to be adequately simple because it is a known hot spot in the processor execution, and any subtle overhead can accue into a noticeable cost in performance. Also, please note that at least for repeated strings, string tables kick in after the first occurrence of that string value. Therefore, the effect would be limited to the first occurrence of a particular string value only. We presume that this is one of the reasons why we did not see the level of benefits we originally expected as you do, out of the length-related facets. |
tocheck |
LC-2173
FABLET Youenn <Youenn.Fablet@crf.canon.fr> (archived comment) |
|
It seems that there are two aspects to your question. One pertains to design issues involved in creating XML Schemas for optimizing EXI encoding compactness. The other relates to the internal representation that an EXI implementation might use to perform schema-based encoding. Concerning designing a schema to get more compact EXI encodings, the general rule would be, "More precise content constraints and more deterministic structure lead to a more compact EXI encodings." Examples of applying this general rule are: -Carefully think about minOccurs and maxOccurs of particles, and requirement (vs. optionality) of attributes. The more you exploit those properties, the more compactness you can achieve. -Use substitutionGroup, instead of depending on xsi:type mechanism, whenever possible. -Use wildcard only when absolutely necessary. Use concrete elements and attributes if possible. The above are simple guidelines. However, some questions related to the general rule are not so simple. An example of such a consideration is: Where is the right location for a required element in a model group? For example, in the following schema fragment, element "D" comes first. But placing "D" after "C" (as shown in the second schema fragment, below) would increase determinism. The reason for this relates to the number of possible alternatives for elements which may follow "B". So, in the first example, the elements "C", "W", "X", "Y" and "Z" may each appear after "B". However, in the second example, only element "D" may follow "B". The second example is more deterministic and should result in a more compact encoding. <sequence> <element name="D" /> <element name="B" minOccurs="0" /> <element name="C" minOccurs="0" /> <choice> <element name="W" /> <element name="X" /> <element name="Y" /> <element name="Z" /> </choice> </sequence> <sequence> <element name="B" minOccurs="0" /> <element name="C" minOccurs="0" /> <element name="D" /> <choice> <element name="W" /> <element name="X" /> <element name="Y" /> <element name="Z" /> </choice> </sequence> Regarding the grammar size relative to schema size (which is one of the issues we believe you raise), this is the nature of schema-processing. EXI is no different from schema-validation of XML parsers using XML schemas. In principle, grammar size is proportional to the number of definitions and constructs used in the schema. This should be a consideration when generated schema-informed EXI encodings. The question of how much schema information should be stored and applied is important, because in some cases there are two conflicting goals. The first is the desire to limit grammar footprint and the second is the desire to increase determinism of the grammar. So this is a practical issue that each use case has to cope with, rather than an inherent issue with EXI. We have not, as a group, discussed this, nor was it in our plans to develop these kinds of implementation guidelines. While such instances of schemas may cause footprint issues for a straight-forward implementation of the specification, there have been implementation reports from within the WG that there are implementation techniques that resolve such footprint issues. We may look into schema modeling guidelines, but only after other charter-driven tasks are completed, and if time allows. It seems that guidelines addressing issues in implementing an XML schema validation parser would be pertinent to your questions (especially with respect to your question about grammar size). Considering this, the XML Schema group may have better insight than the EXI group can provide at this time. |
tocheck |
LC-2227
Gengo Suzuki <suzuki.gengo@lab.ntt.co.jp> (archived comment) |
|
http://lists.w3.org/Archives/Public/public-exi-comments/2009Jul/0002.html | tocheck |
LC-2165
ISHIZAKI Tooru <ishizaki.tooru@canon.co.jp> (archived comment) |
|
The advantage of the local-element-ns flag is improved compactness for representing element QNames when Preserve.prefixes is true. Without the local-element-ns flag, the prefix encoding for each element QName would have to account for the possibility that the prefix is declared by an NS event that follows the associated SE event (i.e., a prefix that has not yet been encountered in the stream). In the common case, where there is only one prefix declared for each namespace, the prefix encoding would increase from zero bits per element to one bit per element. If there were two prefixes declared for each namespace, the prefix encoding would increase from one bit per element to two bits per element. Since there can be a lot of SE events in a document, this can have a significant impact on compactness. |
tocheck |
LC-2167
ISHIZAKI Tooru <ishizaki.tooru@canon.co.jp> (archived comment) |
|
http://lists.w3.org/Archives/Public/public-exi-comments/2009Mar/0001.html | tocheck |
LC-2192
Jochen Darley <joda@upb.de> (archived comment) |
|
The blocksize of EXI compression is constant for a single EXI stream so that an EXI decoder get an approximate idea of how much memory is necessary for buffering events before it starts decompression. One of the inclinations of EXI design was enabling those devices with limited hardware resources, often with very limited network resources to successfully tap into XML family of technologies. For the scenario you described, the best solution may be to feed a flow of separate EXI documents or EXI fragments, each of which has been compressed independently. The other thing that may be relevant and worth mentioning is that EXI compression is designed to cost much less CPU cycle than gzip does, yet consistently with better results. This trait is known to enable certain scenarios to apply EXI compression for their output feeds, wherein application of gzip has been perceived prohibitive given the exhaustion of server CPU resources due to the high processing cost entailed in compression. |
tocheck |