As texts, or messages, are exchanged between applications, they are processed. Most applications are designed to discriminate between valid and invalid inputs. In order to have any sort of interoperability, a language must be defined or described in some normative way so that the terms invalid and valid have meaning.

There is a range of mechanisms that can be employed for defining what constitutes the validity of a language. For example, DTDs, W3C XML Schemata, RELAX NG Schemata, methods based on Schematron and so on. These approaches might be augmented with normative prose documentation or even some application-specific validation logic. In many cases, the schema language is the only validation mechanism that is available.

It is almost unheard of for a single version of a language to be deployed without requiring some kind of augmentation. Invariably, the original language designer did not include certain terms and constraints. In fact, good designers should not try to define all the possible terms and constraints. This is sometimes called boiling the ocean. Knowing that a language will not be all things to all people, a language designer can allow parties to extend instances of the language or the language itself. Typically the tools will allow the language designer to specify where extensions in the instance and extensions in the language are allowed. Of note, we do not call extending a text of a language a new version. This limits our discussion of versioning to changes in a language, not changes to instances.

Whether you've deployed ten resources, or a hundred, or a million, if you change a language in such a way that all those resources will consider instances of the new language invalid, you've introduced a versioning problem with real costs.

Once a language is used outside of its development environment, there will be some cost associated with changing it: software, user expectations, and documentation may have to be updated to accommodate the change. Once a language is used in environments outside of a single realm of control, any changes made will introduce multiple versions of the language.

There are many reasons why a different version of a language may be needed. A few of them include:

Whatever the cause, over time, different versions of the language exist and designing applications to deal with this change in a predictable, useful way requires a versioning strategy.

At the most basic level, languages can change in only a few ways:

Of course, the difference between two versions of a language can be an arbitrary number of these changes.

One of the most important aspects of a change is whether or not it is backwards or forwards compatible.

Some typical backwards- and forwards-compatible changes:

Some typical forwards-compatible changes:

Some typical backwards-compatible changes:

Some typical incompatible changes:

Ultimately, there are different kinds of languages. The versioning approaches and strategies that are appropriate for one kind of language may not be appropriate for another. Among the various kinds of vocabularies, we find:

This is by no means an exhaustive list. Nor are these categories exclusive. Many languages have mixed modes. For example, XQuery has a text mode and an XML mode.

Different kinds of languages and different versioning strategies expose different problems. If you don't have a strategy at all, you are effectively choosing the no versioning strategy.

It's probably obvious that attempting to deploy a system that provides no versioning mechanism is fraught with peril. Putting the burden of version discovery on consumers is probably impractical in anything except a closed system.

At the other end of the spectrum is the "big bang" approach which is also problematic.

Big bang is a very coarse-grained approach to versioning. It establishes a single version identifier, either a version number or namespace name, for an entire text.

The semantics of the "big bang" are that applications decide on the basis of the text version whether or not they know how to process that text. If the version isn't recognized, the entire text is rejected. Typically, when introducing a new version using the big bang approach, all of the software that produces or consumes the texts is updated in a sweeping overhaul in which the entire system is brought down, the new software deployed and the system is restarted. This big bang approach to versioning is practical only in circumstances where there is a single controlling authority, and even in that case, it carries with it all manner of problems. The process can take a considerable amount of time, leaving the system out of commission for hours if not days. This can result in significant losses if the system is a key component of a revenue generating business process and the cost of coordinating the system overhaul can also be quite costly as well.

The "big bang" approach is appropriate when the new version is radically different from its predecessor. But in many cases, the changes are incremental and often a consumer could, in practice, cope with the new version. For example, it might be that there are many messages that don't use any features of the new version or perhaps it is appropriate to simply ignore elements that are not recognized.

Recall our Name example and consider a producer and a consumer exchanging name messages. Imagine that some future version of the name language defines a new middle element. Because producers and consumers are distributed, it may happen that an old consumer, one unprepared for a middle element, encounters a message sent by a newer producer.

If big bang versioning is used, old systems will reject the new message. However, if the versioning strategy allowed the old consumer to simply ignore unrecognized content, it's quite possible that other components of the system could simply adapt to the previous behavior. In effect, the old system would ignore the middle element and its descendents so it would see a message that looks just like the old format it is expecting.

For the producer, the result would be that the request is fulfilled, though perhaps not quite the way it may have hoped. For example, a request that results in a name response may return a name without the middle. In many cases this may be better behavior than receiving an error. In particular, producers using the new format can be written to cope with the possibility that they will be speaking to older consumers.

If the new system needs to make sure that middle is respected, then it can change the language in an incompatible way to indicate that the new behavior is not considered backwards compatible.

Often, what is needed is some sort of middle ground solution.

For any given strategy, there are various designs that achieve the strategy, and some may be more appropriate than others. Among them we find:

Languages can be expressed in text, comma separated values, XML, SGML, binary, source code, and almost any kind of form. See the Architecture of the World-Wide Web section on data formats for more information - http://www.w3.org/TR/2004/REC-webarch-20041215/#formats.

It is sometimes desirable to prevent 3rd parties from extending languages, but it does happen. An example may be a tightly constrained security environment where distributed authoring is considered a "bug" rather than a feature.

If so, a substitution mechanism is required for forwards compatibility. If an older consumer has no mechanism for dealing with new content, then forwards-compatible evolution isn't possible. One simple substitution mechanism is simply ignoring the unrecognized components.

If so, and if compatible extensions are also possible, then it must be possible to identify incompatible changes so that they can override the substitution mechanism used for extensible changes.

In environments where unrecognized components are ignored, a "must understand" component can be added to identify incompatible changes.

If compatible changes are not possible, then incompatible changes simply become the default. For example, WS-Security mandates that 3rd parties can only provide incompatible extensions. Unlike most languages, a security language has unique requirements where the consequences of ignored data can be severe. WS-Security accomplished this by specifying that all extensions are required to be understood and there is no substitution mechanism.

As with 3rd parties compatible extensions, a substitution mechanism for the designer’s extensions is required for forwards compatibility.

In XML, the designer can always do this by using new namespace names, element names or version numbers. In other languages, this may not be possible because there is no mechanism for indicating the incompatible change.

A part of this question is whether the language depends on another language. That determines which, if any, facilities are provided for the containing language and what must be provided by a contained language.

SOAP is an example of a container language. The SOAP processing model applies uniformly to all headers, which may employ soap:mustUnderstand to identify incompatible changes, even though the contents of the SOAP headers are languages independent from SOAP.

Choosing a schema language or languages guides the language design in many ways. Some features, particularly extensibility, must be anticipated in the first version of a language in order to take advantage of the features of some schema languages.

In addition, various features may be incompatible across different languages. For example, writing a V2 compatible schema in W3C XML Schema requires special design, which is not required in a schema language such as RELAX NG. Some of the language design choices mandated by W3C XML Schema are discussed in other sections of this Finding.

The ability to write a schema for extensions or versions is directly affected by the schema design and the compatibility desires.

Every language design will make decisions about these requirements. These requirements can be expressed in a table form:

Requirement
Language form
Schema Lang
3rd party compatibly extend
3rd party incompatibly extend
Designer incompatibly extend
stand-alone

If the language is intended to be capable of compatible extensibility, then a few specific schema design choices must be followed.

Forwards compatibility can only be achieved by providing a substitution mechanism for Version 2 instances or Version 1 extensions to V1 without knowledge of V2.  A V1 consumer must be able to transform any instances, such as V1 + extensions, to a V1 instance in order to process the instance. The "Must Ignore unknown" rule is a simple substitution mechanism. This rule says that any extensions are "ignored". Using it, a V1 + extensions text is transformed into a V1 text by ignoring the extensions. Others substitution mechanisms exist, such as the fallback model in XSLT.

The identification of components into language versions or extensions has a variety of general mechanisms related to namespaces. These are detailed in the Versioning section.

The identification of versions is covered by language identification, but 3rd parties cannot arbitrarily change versions or change namespaces. They may need a mechanism to indicate that an extension is an incompatible change. A couple of mechanisms are a "Must Understand" identifier (such as a flag or list of required namespaces) or requiring that extensions are in substitution groups.

Every language design will make a decision in these areas. These designs can also be expressed in a table form:

Design
Schema design
Substitution Mechanism
Component Identification
Incompatible Ext identification

Having multiple versions naturally leads to the need to identify versions. Version identification has traditionally been done with a decimal separating the major versions from the minor versions, ie "8.1", "1.0". Often the definition of a "major" change is that it is incompatible, and the definition of a "minor" change is that it is forwards- and/or backwards - compatible. Usually the first broadly available version starts at "1.0". A compatible version change from 1.0 might be identified as "1.1" and an incompatible change as "2.0".

The version numbers can be contained in the texts, in the protocol messages containing in the text, or the address for the protocol messages. Some examples are shown below: Name examples. Dave Orchard Dave Orchard urn:nameschemev2:given:Dave:family:Orchard GET /name/123456789 HTTP/1.1 GET /name/v2/123456789/ HTTP 1.1]]>

It should be noted that associating version number changes with compatibility changes may be idealistic as there abundant cases where this system does not hold. New major version identifiers are often aligned with product releases, or incompatible changes identified as a "minor" change. A good example of an incompatible changed identified as a minor change is XML 1.1. XML 1.0 processors cannot process all XML 1.1 documents because XML 1.1 extended XML 1.0 where XML 1.0 does not allow such extension.

Unfortunately, version numbers often wind up looking very similar to the big bang approach. In many approaches, each language is given a version identifier, almost always a number, that's incremented each time the language changes. Although it's possible to design a system with version numbers that enables both backward and forward compatibility - for example XSLT - typically a version change is treated as if that the new language is not backwards compatible with the old language.

Some efforts, such as HTTP, try to have the best of both worlds by allowing for extensibility (in HTTP's case, via headers) as well as version numbers that explicitly identify when a new version is backwards compatible with an old version.

One argument in favor of version numbers is that they allow one to determine what is a 'new version' and what is an 'old version'. But in practice this is not necessarily true. For example, RSS has 0.9x, 1.x, and 2.x versions, all being actively developed in parallel. In effect the version numbers, even though they appear to be ordered, are simply opaque identifiers. Using version numbers does not gaurantee that version 1+x has any particular relationship to version 1.

Version numbers typically work best when versioning and extending a language is done in a centralized and linear manner. The makeup of each version can then be consistent and well described.

There are many cases where decentralized and non-linear versioning is desired. The desire for decentralized and non-linear versioning and extensibility was a large motivator for XML and for XML Namespaces. The self-describing and extensible nature of XML markup, and the addition of XML Namespaces, provides a framework for developing languages that can evolve in a decentralized manner. XML Namespaces [ XML Namespaces 1.0] provide a mechanism for associating a URI with an XML element or attribute name, thus specifying the language of the name. This also serves to prevent name collisions.