The terminology for describing languages, producers, consumers, information, constraints, syntax, evolvability etc. follows. Let us consider an example. Two or more systems need to exchange name information. Names may not be the perfect choice of example because of internationalization reasons, but it resonates strongly with a very large audience. The Name Language is created to be exchanged. A producer is an agent that creates text. Continuing our example, Fred is a producer of Name Language text. An Act of Production is the creation of text.. A producer produces text for the intent of conveying information. When Fred does the actual creation of the text, that is an act of production. A consumer is an agent that consumes text. We will use Barney and Wilma as consumers of text. An Act of Consumption is the processing of text of a language. Wilma and Barney consume the text separately from each other, each of these being a consumption event. A consumer is impacted by the instance that it consumes. That is, it interprets that instance and bases future processing, in part, on the information that it believes was present in that instance. Text can be consumed many times, by many consumers, and have many different impacts.

A Language consists of a set of text, any syntactic constraints on the text, a set of information, any semantic constraints on the information, and the mapping between texts and information. Text is a specific, discrete sequence of characters. Given that there are constraints on a language, any particular text may or may not have membership in a language. Indeed, a particular string of characters may be a member of many languages, and there may be many different strings of characters that are members of a given language. The text of the language are the units of exchange. Documents are texts of a language. The Name Language consists of text set that have 3 terms and specifies syntactic constraints: that a name consists of a given and a family. A language has a set of constraints that apply to the set of strings in the language. These constraints can be defined in machine processable syntactic constraint languages such as XML Schema, microformats, human readable textual descriptions such as HTML descriptions, or are embodied in software. Languages may or may not be defined by a schema in any particular schema language. The constraints on a language determine the strings that qualify for membership in the language. Vocabulary terms contribute to the set of strings, but they are not the only source of characters to the set of strings in a given language. The language strings may include characters outside of terms, such as punctuation. One reason for additional characters is to distinguish or separate terms, such as whitespace and markup.

Name examples. Dave Orchard name="Dave Orchard" Dave Orchard urn:nameschem:given:Dave:family:Orchard]]>

The set of information in a language almost always has semantics. In the Name Language, given and family have the semantics of given and family names of people. The language also has the binding from the items in the information set to the text set. Any potential act of interpretation, that is any consumption or production, conveys information from text according to the language's binding. The language is designed for acts of interpretation, that being the purpose of languages. In our example, this mapping is obvious and trivial, but many languages it is not. Two languages may have the exact same strings but different meanings for them. In general, the intended meaning of a vocabulary term is scoped by the language in which the term is found. However, there is some expectation that terms drawn from a given vocabulary will have a consistent meaning across all languages in which they are used. Confusion often arises when terms have inconsistent meaning across language. The Name terms might be used in other languages, but it is generally expected that they will still be "the same" in some meaningful sense.

These terms and their relationships are shown below

There are many different systems for exchanging texts in languages, such as SQL, Java, XML, ECMAScript, C#. We will briefly describe some key refinements to our lexicon for XML. An XML language has a vocabulary that may use terms from one or more XML Namespaces (or none), each of which has a namespace name. An XML language is an identifiable set of vocabulary terms with defined XML syntactic and semantic constraints. By XML language, we mean the set of elements and attributes, or instances, used by a particular application. The Name Language - consisting of name, given, family terms - has a namespace for the terms. We use the prefix "namens" to refer to that namespace. The Name Language could consist of terms from other vocabularies, such as Dublin Core or UBL. These terms each have their own namespaces, illustrating that a language can comprise vocabularies from multiple namespaces. An XML Namespace is a convenient container for collecting terms that are intended to be used together within a language or across languages. It provides a mechanism for creating globally unique names.

We shall use the term instance when speaking of sequences of characters (aka text) in XML. An instance is a specific, discrete Text in XML format. Documents are instances of a language. In XML, they must have a root element. A name document might have a name element as the root element. Alternatively, the name vocabulary may be used by a language such as purchase orders. The purchase order documents may contain name elements. Thus instances of a language are always part of a document and also may be the entire document. XML instances (and all other instances of markup languages) consist of markup and content. In the name example, the given and family elements including the end markers are the markup. The values between the start and end markers are the content. An instance has an information model. There are a variety of data models within and without the W3C, and the one standardized by the W3C is the XML infoset.

The XML related terms and their relationships are shown below

As documents, or messages, are exchanged between applications, they are processed. Most applications are designed to discriminate between valid and invalid inputs. In order to have any sort of interoperability, a language must be defined or described in some normative way so that the terms invalid and valid have meaning.

There are a variety of tools that might be employed for this purpose (DTDs, W3C XML Schema, RELAX NG, Schematron, etc.). These tools might be augmented with normative prose documentation or even some application-specific validation logic. In many cases, the schema language is the only validation logic that is available.

It is almost unheard of for a single version of a language to be deployed without requiring some kind of augmentation. Invariably, the original language designer did not include certain terms and constraints. In fact, good designers should not try to define all the possible terms and constraints. This is sometimes called boiling the ocean. Knowing that a language will not be all things to all people, a language designer can allow parties to extend instances of the language or the language itself. Typically the tools will allow the language designer to specify where extensions in the instance and extensions in the language are allowed. Of note, we do not call extending an instances of a language a new version. This limits our discussion of versioning to changes in a language, not changes to instances.

Whether you've deployed ten services, or a hundred, or a million, if you change a language in such a way that all those services will consider instances of the new language invalid, you've introduced a versioning problem with real costs.

Once a language is used outside of its development environment, there will be some cost associated with changing it: software, user expectations, and documentation may have to be updated to accomodate the change. Once a language is used in environments outside of a single realm of control, any changes made will introduce multiple versions of the language.

There are many reasons why a different version of a language may be needed. A few of them include:

Whatever the cause, over time, different versions of the language exist and designing applications to deal with this change in a predictable, useful way requires a versioning strategy.

The primary motivation to allow instances of a language to be extended is to decentralize the task of designing, maintaining, and implementing extensions. It allows producers to change the instances without going through a centralized authority. It means that changes can occur at the producer or consumer without the language owner approving of them. Consider the effort that the HTML Working Group put into modularity of HTML. Without some decentralized process for extension, every single variant of HTML would have to be called something else or the HTML Working Group would have to agree to include it in the next revision of HTML.

At the most basic level, languages can change in only a few ways:

Of course, the difference between two versions of a language can be an arbitrary number of these changes.

One of the most important aspects of a change is whether or not it is backwards or forwards compatible.

Some typical backwards- and forwards-compatible changes:

Some typical forwards-compatible changes:

Some typical backwards-compatible changes:

Some typical incompatible changes:

Ultimately, there are different kinds of languages. The versioning approaches and strategies that are appropriate for one kind of language may not be appropriate for another. Among the various kinds of vocabulares, we find:

This is by no means an exhaustive list. Nor are these categories completely clear cut. MathML can certainly be used standalone, for example, and languages like SVG are a combination of standalone, containers, and mixtures.

It is rarely desirable to prevent 3rd parties from extending languages, but it does happen. An example may be a tightly constrained security environment where distributed authoring is considered a “bug” rather than a feature.

If so, a substitution mechanism is required for forwards compatibility. If an older consumer has no mechanism for dealing with new content, then forwards-compatible evolution isn't possible. One simple substitution mechanism is simply ignoring the unrecognized components.

If so, and if compatible extensions are also possible, then it must be possible to identify incompatible changes so that they can override the substitution mechanism used for extensible changes.

In environments where unrecognized components are ignored, a “must understand” component can be added to identify incompatible changes.

If compatible changes are not possible, then incompatible changes simply become the default. For example, WS-Security mandates that 3rd parties can only provide incompatible extensions. Unlike most languages, a security language has unique requirements where the consequences of ignored data can be severe. WS-Security accomplished this by specifying that all extensions are required to be understood and there is no substitution mechanism.

As with 3rd parties compatible extensions, a substitution mechanism for the designer’s extensions is required for forwards compatibility.

In XML, the designer can always do this by using new namespace names, element names or version numbers. In other languages, this may not be possible because there is no mechanism for indicating the incompatible change.

A part of this question is whether the language depends on another language. That determines which, if any, facilities are provided for the containing language and what must be provided by a contained language.

SOAP is an example of a container language. The SOAP processing model applies uniformly to all headers, which may employ soap:mustUnderstand to identify incompatible changes, even though the contents of the SOAP headers are languages independent from SOAP.

Languages can be expressed in text, comma separted values, XML, SGML, binary, source code, and almost any kind of form. See the Architecture of the World-Wide Web section on data formats for more information - http://www.w3.org/TR/2004/REC-webarch-20041215/#formats.

Choosing a schema language or languages guides the language design in many ways. Some features, particularly extensibility, must be anticipated in the first version of a language in order to take advantage of the features of some schema languages.

In addition, various features may be incompatible across different languages. For example, writing a V2 compatible schema in W3C XML Schema requires special design, which is not required in a schema language such as RELAX NG. Some of the language design choices mandated by W3C XML Schema are discussed in other sections of this Finding.

The ability to write a schema for extensions or versions is directly affected by the schema design and the compatibility desires.

If the language can be extended in a compatible way, then a few specific schema design choices must be followed.

Wildcards are used to provide extensibility in XML Schema. If revisions to the Schema are to support substitution, specific schema designs must be used in conjunction with the wildcard. The main choices are: provide wildcards, provide Extension elements, or provide delimiter elements. Extension and delimiter elements are described in the new components in existing or new namespaces section.   If Extension/delimiter elements are not provided, then a compatible V2 Schema cannot be written.

Forwards compatibility can only be achieved by providing a substitution mechanism for Version 2 instances or Version 1 extensions to V1 without knowledge of V2.  A V1 consumer must be able to transform any instances, such as V1 + extensions, to a V1 instance in order to process the instance. The “Must Ignore unknown” rule is a simple substitution mechanism. This rule says that any extensions are “ignored”. Using it, a V1 + extensions document is transform into a V1 document by ignoring the extensions. Others substitution mechanisms exist, such as the fallback model in XSLT.

The identification of components into language versions or extensions has a variety of general mechanisms related to namespaces. These are detailed in the Versioning section.

The identification of versions is covered by language identification, but 3rd parties cannot arbitrarily change versions or change namespaces. They may need a mechanism to indicate that an extension is an incompatible change. A couple of mechanisms are a “Must Understand” identifier (such as a flag or list of required namespaces) or requiring that extensions are in substitution groups.

Versioning is a broad and complex issue. Different communities have different notions about what constitutes a version, what constitutes a reasonable policy, and what the appropriate behavior is in the face of deviations from that policy. Historically, it has always proved more complicated in practice than in theory.

In broad terms, the approaches to versioning fall into a number of classes ranging from none to a big bang:

There's no single approach that's always correct. Different application domains will choose different approaches. But by the same token, the approaches that are available depend on other choices, especially with respect to namespaces. This dependency makes it imperative to plan for versioning from the start. If you don't plan for versioning from the start, when you do decide to adopt a plan for versioning, you may be constrained in the available approaches by decisions that you've already made.

A language commonly goes through a lifecycle of iterative development followed by deployment followed by deployment of new versions. The point in the lifecycle will affect the selection of the versioning strategy for the language

Just as there are a number of approaches, there are a number of strategies for implementing an approach. The internet - including MIME, markup languages, and XML languages have succesfully used various strategies, either singly or in combination. Summaries of strategies and requirements have been produced for earlier technologies and guided XML Namespaces and Schema, such as .

For any given approach, some strategies may be more appropriate than others. Among the strategies we find:

Languages can choose a mixture of approaches. For example, XSLT provides both an explicit fallback mechanism for some conditions and explicit testing for others. The SOAP specification, another example, specifies Must Ignore as the default strategy and the ability to dynamically mark components as being in the Must Understand strategy.

Different kinds of languages and different versioning strategies expose different problems. If you don't have a strategy at all, you are effectively choosing the no versioning strategy.

It's probably obvious that attempting to deploy a system that provides no versioning mechanism is frought with peril. Putting the burden of version discovery on consumers is probably impractical in anything except a closed system.

At the other end of the spectrum is the "big bang" approach which is also problematic.

Big bang is a very coarse-grained approach to versioning. It establishes a single version identifier, either a version number or namespace name, for an entire document.

The semantics of the "big bang" are that applications decide on the basis of the document version whether or not they know how to process that document. If the version isn't recognized, the entire document is rejected.. Typically, when introducing a new version using the big bang approach, all of the software that produces or consumes the instance documents is updated in a sweeping overhaul in which the entire system is brought down, the new software deployed and the system is restarted. This big bang approach to versioning is practical only in circumstances where there is a single controlling authority, and even in that case, it carries with it all manner of problems. The process can take a considerable amount of time, leaving the system out of commission for hours if not days. This can result in significant losses if the system is a key component of a revenue generating business process and the cost of coordinating the system overhaul can also be quite costly as well.

The "big bang" approach is appropriate when the new version is radically different from its predecessor. But in many cases, the changes are incremental and often a consumer could, in practice, cope with the new version. For example, it might be that there are many messages that don't use any features of the new version or perhaps it is appropriate to simply ignore elements that are not recognized.

For example, consider two services exchanging messages. Imagine that some future version of the language that they are using defines a new priority element. Because producers and consumers are distributed, it may happen that an old consumer, one unprepared for a priority element, encounters a message sent by a newer producer.

If big bang versioning is used, old systems will reject the new message. However, if the versioning strategy allowed the old consumer to simply ignore unrecognized content, it's quite possible that other components of the system could simply adapt to the previous behavior. In effect, the old system would ignore the priority element and its descendents so it would see a message that looks just like the old format it is expecting.

For the producer, the result would be that the request is fulfilled, though perhaps in a more or less timely fashion than expected. In many cases this may be better behavior than receiving an error. In particular, producers using the new format can be written to cope with the possibility that they will be speaking to old consumers.

If the new system needs to make sure that priority is respected, then it can change the purchase order's name or namespace to indicate that the new behavior is not considered backwards compatible.

What is needed is some sort of middle ground solution. An evolving system should be designed with backwards and forwards compatibility in mind.

One approach to mitigate against a complete overhaul is to run multiple versions of the system. One variant is offering both versions of the system, for example by using different URIs for the old and new resources. The request to one resource gets mapped to the other resource behind the scenes using a proxy or gateway. This "alternative" approach works when the intermediary can completely handle or generate the new information (for backwards compatibility) or ignore the new information (for forwards compatibility). For example, adding SSL security to a resource changes the URI but a Web server can typically handle mapping the https: URI to the older http: URI. If both URIs are maintained, then the addition is a compatible change. Another example is where new information is required, such as the priority, and the intermediary can apply a default value to provide the required priority. However, this too has its costs as multiple versions of the software must be supported and maintained over time and there is the added cost of developing the proxy or gateway between the two environments. Further, this does not work in scenarios where the intermediary cannot generate the new required content. For example, if a middle name is required in V2, a middle cannot be generated from just a family and a given name.

Having multiple versions naturally leads to identifying versions. One approach to version identification is to use version numbers, with a goal of using major version changes for incompatible changes and minor version changes for compatible ones. The version numbers can be contained in the instance documents, in the protocol messages containing in the instance document, or the address for the protocol messages (ie http://example.com/foo/v2).

Unfortunately, version numbers often wind up looking very similar to the big bang approach. In many approaches, each language is given a version identifier, almost always a number, that's incremented each time the language changes. Although it's possible to design a system with version numbers that enables both backward and forward compatibility - for example XSLT - typically a version change is treated as if that the new language is not backwards compatible with the old language.

Some efforts, such as HTTP, try to have the best of both worlds by allowing for extensibility (in HTTP's case, via headers) as well as version numbers that explicitly identify when a new version is backwards compatible with an old version.

One argument in favor of version numbers is that they allow one to determine what is a 'new version' and what is an 'old version'. But in practice this is not necessarily true. For example, RSS has 0.9x, 1.x, and 2.x versions, all being actively developed in parallel. In effect the version numbers, even though they appear to be ordered, are simply opaque identifiers. Using version numbers does not gaurantee that version 1+x has any particular relationship to version 1.

The self-describing and extensible nature of XML markup, and the addition of XML Namespaces, provide a much better framework for developing languages that can evolve.

The following names would be valid:

The 2^nd example shows all the components in the same new namespace. The 3rd and 4^th example show an additional middle element in 2 different namespace names. The 3rd example comes from a namespace name that is in the same domain as the name element’s new namespace name. One reason for 2 namespaces is to modularize the language. The 4^th example shows a namespace name from a different domain for the middle. It is probable that the mid:middle was created by the name author, and the middiffdomain:middle was created by a 3rd party.

In this strategy, the following names would be valid:

The 2^nd and 3^rd example show an additional middle element in 2 different namespace names. The first middle, the 2^nd example, comes from a namespace name that is in the same domain as the name element’s namespace name. The 3^rd example shows a complete different namespace name for the middle. It is probable that the mid:middle was created by the name author, and the middiffdomain:middle was created by a 3rd party.

In this strategy, the following names would be valid:

The 2^nd example shows the use of the optional middle name in the name namespace. The 3^rd and 4^th example show an additional middle element in 2 different namespace names. The first middle, the 3rd example, comes from a namespace name that is in the same domain as the name element’s namespace name. The 4^th example shows a complete different namespace name for the middle. It is probable that the mid:middle was created by the name author, and the middiffdomain:middle was created by a 3rd party.

Using a version identifier, the name instances would change to show the version of the name they use, such as:

The last two examples show that the middle is now a mandatory part of the name. This is indicated by just the version number or a new namespace plus version number.

A significant downside with using version identifiers is that software that supports both versions of the name must perform special processing on top of XML and namespaces. For example, many components “bind” XML types into particular programming language types. Custom software must process the version attribute before using any of the “binding” software. In Web services, toolkits often take SOAP body content, parse it into types and invoke methods on the types. There are rarely “hooks” for the custom code to intercept processing between the “SOAP” processing and the “name” processing. Further, if version attributes are used by any 3rd party extensions—say mid:middle has a version—then the schema cannot refer to the correct middle.

Using a version identifier, the name instances would change to show the version of the name they use, such as:

The 2^nd example shows that the middle is an optional part of the name. The last example shows that the middle is a mandatory part of the name.

A downside with using new namespace names is that some tools, like XPath, can be harder to use in the face of new namespace names. Software that extracts the given and family name based upon the expanded name will often break if a new namespace name is used.

The various schema languages have a variety of defined mechanisms to indicate extensions that are incompatible. For example, XML Schema provides Type Extension and Substitution Groups. These mechanisms are described in Part 2.