Grammar productions are used to describe objects (values, types, [XPath/XQuery] expressions, etc.) manipulated by the Formal Semantics. The Formal Semantics makes use of several kinds of grammar productions: productions from the [XPath/XQuery] grammar itself, productions for a subset of the [XPath/XQuery] language called the XQuery Core which is used throughout this document, and other productions used for formal specification only such as for the XQuery type system.

XQuery grammar productions describe the XQuery language and expressions. XQuery productions are identified by a number, which corresponds to their number in the document, and are marked with (XQuery). For instance, the following production describes FLWOR expressions in XQuery.

For the purpose of this document, the differences between the XQuery 1.0 and the XPath 2.0 grammars are mostly irrelevant. By default, this document uses XQuery 1.0 grammar productions. Whenever the grammar for XPath 2.0 differs from the one for XQuery 1.0, the corresponding XPath 2.0 productions are also given. XPath productions are identified by a number, which corresponds to their number in , and are marked with (XPath). For instance, the following production describes for expressions in XPath.

XQuery Core grammar productions describe the XQuery Core. The Core grammar is given in . Core productions are identified by a number, which corresponds to their number in , and are marked with (Core). For instance, the following production describes the simpler form of the FLWOR expression in the XQuery Core.

The Formal Semantics manipulates objects (values, types, expressions, etc.) for which there is no existing grammar production in the document. In these cases, specific grammar productions are introduced. Notably, additional productions are used to describe values in the , and to describe the [XPath/XQuery] type system. Formal Semantics productions are identified by a number, and are marked by (Formal). For instance, the following production describes global type definitions in the [XPath/XQuery] type system.

Note that grammar productions that are specific to the Formal Semantics (i.e., marked with (Formal)) are not part of [XPath/XQuery]. They are not accessible to the user and are only used in the course of defining the languages' semantics.

Grammar non-terminals are used extensively in this document to represent objects in judgments (see the next section). As a convenience, non-terminals used in judgments link to the appropriate grammar production.

The basic building block of the formal specification is called a judgment. A judgment expresses whether a property holds or not.

For example:

A judgment may hold (if it is true) or not hold (if it is false). For instance '1 is a positive integer' holds and '-1 is a positive integer' does not hold.

A judgment can contain symbols and patterns.

Symbols are purely syntactic and are used to write the judgment itself. Symbols are chosen to reflect a judgment's meaning, and are written in bold fonts. For example, 'is a positive integer', '=>' and ':' are symbols, the second and third of which should be read yields, and has type respectively.

Patterns are used to represent objects, constructed from a given grammar production. In patterns, italicized words usually correspond to non-terminals in the grammar. The name of those non-terminals is significant, and may be instantiated only to an object (a value, a type, an expression, etc.) that can be substituted legally for that non-terminal. For example, 'Expr' is a pattern that stands for every [XPath/XQuery] expressions, 'Expr1 + Expr2' is a pattern that stands for every addition expression, 'element a { Value }' is a pattern that stands for every value in the [XPath/XQuery] data model that is an 'a' element.

Non-terminals in a pattern may appear with subscripts (e.g. Expr1, Expr2) to distinguish different instances of the same sort of pattern. In some cases, non-terminals in a pattern may have a name that is not exactly the name of that non terminal, but is based on it. For instance, a BaseTypeName is a pattern that stands for a type name, as would TypeName, or TypeName2. This usage is limited, and only occurs to improve the readability of some of the inference rules.

When instantiating the judgment, each pattern must be instantiated to an appropriate sort of object (value, type, expression, etc). For example, '3 => 3' and '$x+0 => 3' are both instances of the judgment 'Expr => Value'. Note that in the first judgment, '3' corresponds to both the expression '3' (on the left-hand side of the => symbol) and to the value '3' (on the right-hand side of the => symbol).

In some cases, inference rules may need to use the fact that a certain judgment does not hold. not(Judgment) holds if and only if Judgment does not hold.

In some cases, a pattern may be instantiated to a value within a finite set of pre-determined values. We may write that set of possible values using the in judgment. For instance, the judgment

holds if the pattern Color has either the value blue or the value green.

In some cases, a judgment may use the = sign to indicate that a given value is equal to another value, or that a pattern is equal to a given value. For instance, the judgment

holds if the pattern Color has the value blue.

An index to all the judgments used in this specification is provided in .

An environment component is a dictionary that maps a symbol (e.g., a function name or a variable name) to an object (e.g., a function body, a type, a value). One can access information in an environment component or update it.

If envComp is an environment component, then envComp(symbol) denotes the object to which symbol is mapped. The notation is intentionally similar to function application, because an environment component can be considered a function from the argument symbol to the object to which the symbol is mapped.

This document uses environments that group related environment components. If env is an environment containing the environment component envComp, that environment component is denoted env.envComp. The value that symbol is mapped to in that environment component is denoted env.envComp(symbol).

The two main environments used in the Formal Semantics are: a dynamic environment (dynEnv), which models the [XPath/XQuery] dynamic context, and a static environment (statEnv), which models the [XPath/XQuery] static context. Both are defined in .

For example, dynEnv.varValue denotes the dynamic environment component that maps variables to values and dynEnv.varValue(Variable) denotes the value of the variable Variable in the dynamic context.

Environments are used in a judgment to capture some of the context in which the judgment is computed, and most judgments are computed assuming that some environment is given. This assumption is denoted by prefixing the judgment with env |-. The |- symbol is called a turnstile and is used in almost all inference rules.

For instance, the judgment

is read as: Assuming the dynamic environment dynEnv, the expression Expr yields the value Value.

Environments can be updated, using the following notation:

Updating an environment creates a copy of the original environment and overrides any previous binding that might exist for the same name and the same component in that environment. Updating the environment is used to capture the scope of a symbol (e.g., for variables, namespace prefixes, etc). For instance, in the following expression

each let expression changes the dynamic context by binding a new variable to a new value. Each different context is represented by a different environment. The original environment, in which the expression 1 is evaluated, does not contain any binding for variable $x. This environment is updated a first time with a binding of variable $x to the value 1, and this new environment is used for the evaluation of the expression $x + 2. Then this second environment is updated with a binding of variable $x to the value 3, and this environment is used for the evaluation of the expression $x - 3.

Also, note that there are no operations to remove entries from environments. This is never necessary as updating an environment effectively creates a new extended copy of the original environment, leaving the original environment accessible wherever it is in scope along with the updated copy.

Inference rules are used to specify how to infer whether a given judgment holds or not. Inference rules express the logical relation between judgments and describe how complex judgments can be concluded from simpler premise judgments.

A logical inference rule is written as a collection of premises and a conclusion, written respectively above and below a dividing line, as follows:

All premises and the conclusion are judgments. From a logical point of view, an inference rule is a deduction that if the premises hold, then the conclusion holds as well. In that sense, the previous inference rule has a similar meaning as the following logical statement.

IF premise1

AND ...

AND premisen

THEN conclusion

Here is a simple example of inference rule, which uses specific instances of the example judgment 'Expr => Value' from above:

This inference rule expresses the following property: if the variable expression '$x' yields the value '0', and the literal expression '3' yields the value '3', then the expression '$x + 3' yields the value '3'.

An inference rule may have no premises above the line, which means that the expression below the line always holds. For instance:

This inference rule expresses the following property: evaluating the literal expression '3' always yields the value '3'.

The two above rules are expressed in terms of specific expressions and values, but usually rules are more abstract. That is, the judgments are not fully instantiated. Here is a rule that says that for any variable $VarName that yields the integer value Integer, adding '0' yields the same integer value:

Each occurrence of a given pattern in a particular inference rule must be instantiated to the same object within the entire rule. This means that, in the context of a particular instantiation of a rule, one can talk about the value of $VarName instead of the value bound to the first (second, etc) occurrence of $VarName.

Here is an example of a rule occurring later in this document.

This rule is read as follows: if two expressions Expr1 and Expr2 are known to have the static types Type1 and Type2 (the two premises above the line), then it is the case that the sequence expression Expr1 , Expr2 has the static type Type1, Type2, which is the sequence of types Type1 and Type2. Note that this inference rule does not modify the static environment.

The following rule defines the static semantics of a let expression. The binding of the new variable is captured by an update to the varType component of the original static environment.

This rule is read as follows: First, because the variable is a QName, it is first expanded into an expanded QName. Second, the type Type1 for the let input expression Expr1 is computed. Then the let variable with expanded name, expanded-QName with type Type1 is added into the varType component of the static environment statEnv. Finally, the type Type2 of Expr2 is computed in that new environment.

In some cases, ellipses may be used in inference rules to handle an arbitrary number of judgments. In those cases, some of the patterns may have indices as subscript. If the same index is used several times within the same rule, the number of judgment in each case must be the same. For instance, the following rule holds for any number of expressions, from Expr1 to Exprn, with the same number of types Type1 to Typen.

This inference rule is equivalent to having an unbounded number of rules, the first of which has 1 judgment, the second of which has 2 judgments, etc. For instance, the above rule holds if and only if one of the following rules hold.

or

etc.

When ellipses are used, the value for the index always ranges from 1 to an arbitrary number n.

In isolation, each inference rule describes a fragment of the semantics for a given judgment. Put together, inference rules describe possible inferences that can be used to decide whether a particular judgment holds.

For a given judgment, if that judgment can be inferred to be true by applying any sequence of inferences based on premises which are known to be true, the inference succeeds. In most cases, the inference will proceed by proving intermediate judgments, following the consequences from one judgment to the next by applying successive inference rules.

Such inference is a mechanism which can be used to describe both static type analysis and dynamic evaluation. More specifically, performing static typing consists in proving that the following judgment holds for a given expression Expr.

If the judgment holds for a given type Type, this type is a possible static type for the expression. If there exists no type for which this judgment holds, then static typing fails and a static type error is returned to the user.

Consider the following expression.

Using the static typing rules given for expressions in the rest of this document, one can deduce that the expression is of type xs:integer through the following inference.

Conversly, consider the following expression.

Using the static typing rules given for expressions in the rest of this document, one can apply inference rules up to the following point.

However, there is no rule that can infer the type of fn:nilled((1,2,3)), because the static typing rules for function calls will only hold if the type of the function parameters is a subtype of the expected type. However, here (xs:integer,xs:integer,xs:integer) is not a node type, which is the expected type for the function fn:nilled.

Note that in some cases, the inference can only proceed through the appropriate changes to the environment. For instance, consider the following expression.

Using the static typing rules given for expressions in the rest of this document, one can deduce that the expression is of type (xs:integer,xs:integer) through the following inference.

This example illustrates how each rule is applied to individual sub-expressions, and how the environment is used to maintain the relevant context information.

A value is a sequence of zero or more items. An item is either an atomic value or a node.

An atomic value is a value in the value space of an atomic type, labeled with the name of that atomic type. An atomic type is either a primitive or derived atomic type according to XML Schema , xs:untypedAtomic, or xs:anyAtomicType.

A node is either an element, an attribute, a document, a text, a comment, or a processing-instruction node.

Element nodes have a and contain a complex value or a simple value. Attribute nodes have a and contain a simple value. Text nodes always contain one string value of type xs:untypedAtomic, therefore the corresponding type annotation is omitted in the formal notation of a text node. Document nodes do not have a type annotation and contain a sequence of element, text, comment, or processing-instruction nodes.

A simple value is a sequence of atomic values.

A complex value is a sequence of attribute nodes followed by a sequence of element, text, comment, or processing-instruction nodes.

A can be either the QName of a declared type or an anonymous type. An anonymous type corresponds to an XML Schema type for which the schema writer did not provide a name. Anonymous type names are not visible to the user, but are generated during schema validation and used to annotate nodes in the data model. By convention, anonymous type names are written using the fs: Formal Semantics prefix: fs:anon0, fs:anon1, etc.

Formal values are defined by the following grammar.

Element (resp. attributes) without type annotations, are assumed to have the type annotation xs:anyType (resp. xs:anySimpleType). Atomic values without type annotations, are assumed to have a type annotation which is the base type for the corresponding value. For instance, "Hello, World!" is equivalent to "Hello, World!" of type xs:string.

Untyped elements (e.g., from well-formed documents) have the xs:untyped, untyped attributes have the xs:untypedAtomic, and untyped atomic values have the xs:untypedAtomic.

An element has an optional nilled marker. This marker is present only if the element has been validated against an element type in the schema which is nillable, and the element has no content and an attribute xsi:nil set to "true".

An element also has a sequence of namespace bindings, which are the set of in-scope namespaces for that element. Each namespace binding is a prefix, URI pair. Elements without namespace bindings are assumed to have an empty set of in-scope namespaces.

A well-formed document

In the absence of a Schema, this document is represented as

A document before and after validation.

The formal model for values can represent values before and after validation. Before validation, this element is represented as:

After validation, this element is represented as:

An element with a list type

Before validation, this element is represented as:

Assume the following Schema.

After validation against this Schema, the element is represented as:

An element with an anonymous type

Before validation, this element is represented as:

Assume the following Schema.

After validation, this element is represented as:

where fs:anon1 stands for the internal anonymous name generated by the system for the sizes element.

A nillable element with xsi:type set to true

Before validation, this element is represented as:

Assume the following Schema.

After validation against this Schema, the element is represented as:

An element with a union type

Before validation, this element is represented as:

Assume the following Schema:

After validation against this Schema, the element is represented as:

The [XPath/XQuery] type system is based on and . and specify normatively the type information available in [XPath/XQuery]. We define the formal notation that is used in this document to describe and manipulate types in inference rules. Formal types are used for specification purposes only and are not exposed to the [XPath/XQuery] user.

Representation of content models. For the purpose of static typing, the [XPath/XQuery] type system only describes minOccurs, maxOccurs, and minLength, maxLength on list types for the occurrences that correspond to the DTD operators +, *, and ?. Choices are represented using the DTD operator |. All groups are represented using the interleaving operator (&).

Representation of anonymous types. To clarify the semantics, the [XPath/XQuery] type system makes all anonymous types explicit.

Representation of XML Schema simple type facets and identity constraints. For simplicity, XML Schema simple type facets and identity constraints are not formally represented in the [XPath/XQuery] type system. However, an [XPath/XQuery] implementation supporting XML Schema import and validation must take simple type facets and identity constraints into account.

This document describe types in the [XPath/XQuery] types system, as well as the operations and properties over those types which are used to define the [XPath/XQuery] static typing feature. The two most important properties are whether a data instances matches a type, and whether a type is a subtype of another. Those properties are described in . This document does not describe all other possible properties over those types.

The mapping from XML Schema into the [XPath/XQuery] type system is given in . The rest of this section is organized as follows. describes item types, describes content models, and describe top-level type declarations.

An item type is either an atomic type, an element type, an attribute type, a document node type, a text node type, a comment node type, or a processing instruction type. We distinguish between document nodes, attribute nodes, and nodes that can occur in element content (elements, comments, processing instructions, and text nodes), as we need to refer to element content types later in the formal semantics.

An element or attribute type has a name or wildcard, and an optional type reference. A name alone corresponds to a reference to a global element or attribute declaration. A name with a type reference corresponds to a local element or attribute declaration. "element *" or "attribute *" alone refers to the wildcard types for any element or any attribute. In addition, an element type has an optional nillable flag that indicates whether the element can be nilled or not.

A document type has an optional content type. If no content type is given, then the type is treated as being the wildcard type for documents, i.e., a sequence of text and element nodes. For consistency with element nodes, PIs and comments are not indicated in that wildcard type, but may occur in instances.

Following XML Schema, types in [XPath/XQuery] are composed from item types by optional, one or more, zero or more, all group, sequence, choice, empty sequence (written empty), or empty choice (written none).

The type empty matches the empty sequence. The type none matches no values. none is the identity for choice, that is (Type | none) = Type. The type none is the static type for .

The [XPath/XQuery] type system includes three binary operators on types: ",", "|" and "&", corresponding respectively to sequence, choice and all groups in Schema. The [XPath/XQuery] type system includes three unary operators on types: "*", "+", and "?", corresponding respectively to zero or more instances of the type, one or more instances of the type, or an optional instance of the type.

The "&" operator builds the interleaved product of two types. The type Type1 & Type2 matches any sequence that is an interleaving of two sequences of items, Value1 and Value2, with Value1 matching Type1 and Value2 matching Type2. The interleaving of two sequences of items Value1 and Value2 is any sequence Value0 such that there is an ordered partition of Value0 into the two sub-sequences Value1 and Value2. The interleaved product captures the semantics of all groups in XML Schema, but is more general as it applies to arbitrary types. All groups in XML Schema are restricted to apply only on global or local element declarations with minOccurs 0 or 1, and maxOccurs 1.

For example, consider the types Type1 = xs:integer,xs:integer,xs:integer and Type2 = xs:string,xs:string. Value1 = (1,2,3) matches the type Type1 and Value2 = ("a","b") matches the type Type2. Any of the following Value0 are interleavings of Value1 and Value2, and therefore match the type (Type1 & Type2):

Types precedence order. To improve readability when writing types, we assume the following precedence order between operators on types.

Parenthesis can be used to enforce precedence. For instance

is equivalent to

and a different precedence can be obtained by writing

Top level definitions correspond to global element declarations, global attribute declarations and type definitions in XML Schema.

A type definition has a name (possibly anonymous) and a type derivation. In the case of a complex type, the derivation indicates whether it is derived by extension or restriction, its base type, and its content model, with an optional flag indicating if it has mixed content.

Note the type system allows recursive types, following the rules defined in .

Global element and attribute declarations always have a name and a reference to a (possibly anonymous) type. A global element declaration also may declare a substitution group for the element and whether the element is nillable.

Here is a schema describing purchase orders from .

Here is the mapping of the above schema into the [XPath/XQuery] type system.

Note that the two anonymous types in the item element declarations are mapping to types with names fs:anon1 and fs:anon2.

The following additional definitions illustrate how more advanced XML Schema features (a complex type derived by extension, an anonymous simple type derived by restriction, and substitution groups) are represented in the [XPath/XQuery] type system.

The above definitions are mapped into the [XPath/XQuery] type system as follows:

statEnv denotes the environment available during static analysis. Static analysis may extend parts of the static environment. The static environment is also available during dynamic evaluation.

If analysis of an expression relies on some component of the static context that has not been assigned a value, a static error is raised.

The following environment components are part of the static environment:

Note that the boundary-space behavior is not formally specified in this document.

An initial environment is set up when [expression/query] processing begins, containing, for example, the function signatures of all built-in functions. The initial values for the static context are defined in and and is denoted by statEnvDefault in the Formal Semantics.

Here is an example that shows how the static environment is modified in response to a namespace definition.

This rule reads as follows: the phrase on the bottom (a namespace declaration in the query prolog followed by an expression) is well-typed (accepted by the static typing rules) within an environment statEnv if the expression above the line is well-typed in the environment obtained from statEnv by adding the namespace declaration.

The helper function fs:active_ns(statEnv) returns all the active in-scope namespaces in the given static environment.

For each attribute and element node in Value, such that the node has name expanded-QName in the namespace AnyURI, the helper function fs:get_static_ns_from_items(statEnv, Value) returns the in-scope namespace that corresponds to AnyURI. This is a reverse-lookup on statEnv.namespace by AnyURI.

dynEnv denotes the environment available during dynamic evaluation. Dynamic evaluation may extend parts of the dynamic environment.

If evaluation of an expression relies on some component of the dynamic context that has not been assigned a value, a dynamic error is raised.

The following environment components are part of the dynamic environment:

The initial values for the dynamic context are defined in and . The corresponding initial dynamic environment is denoted by dynEnvDefault in the Formal Semantics.

The following Formal Semantics variables represent the context item, context position, and context size properties of the dynamic context:

Within this document, variables with the "fs" prefix are reserved for use in the formal specification. Values of $fs:position and $fs:last can be obtained by invoking the fn:position and fn:last functions, respectively. Note that the type for those variables is obtained as for any other variables. As expected the type of $fs:position and $fs:last is always xs:integer while the type of $fs:dot depends on the context in which it is being used.

The following figure depicts the [XPath/XQuery] processing model

Normalization is specified using mapping rules, which describe how a [XPath/XQuery] expression is rewritten into an expression in the [XPath/XQuery] Core. Mapping rules are also used in to specify how XML Schemas are imported into the [XPath/XQuery] type system.

The static semantics is specified using static typing rules, which relate [XPath/XQuery] expressions to types and specify under what conditions an expression is well typed.

The dynamic, or operational, semantics is specified using dynamic evaluation rules, which relate [XPath/XQuery] expressions to values, and in some cases specify the order in which an [XPath/XQuery] expression is evaluated.

The static environment is used in certain cases (e.g. for type matching) during evaluation. To keep the notation simpler, the static environment is not written in the dynamic evaluation rules, but it is assumed to be available.

The inference rules used for dynamic evaluation, like those for static typing, follow a recursive structure, computing the value of expressions from the values of their sub-expressions.

Document order is defined in .

Atomization converts an item sequence into a sequence of atomic values and is implemented by the fn:data function. Atomization is applied to a value when the value is used in a context in which a sequence of atomic values is required.

If a sequence of items is encountered where a boolean value is expected, the item sequence's effective boolean value is used. The fn:boolean function returns the effective boolean value of an item sequence.

[XPath/XQuery] has several functions that provide access to input data, described in . These functions are of particular importance because they provide a way in which an expression can reference a document or a collection of documents. The dynamic semantics of these input functions are described in more detail in .

In certain places in the XQuery grammar, a statically known valid absolute URI is required. These places are denoted by the grammatical symbol URILiteral, and are treated as described in .

All the built-in types of XML Schema are recognized by [XPath/XQuery]. In addition, [XPath/XQuery] recognizes the predefined types xs:anyAtomicType, xs:untypedAtomic and xs:untyped and the duration subtypes xs:yearMonthDuration and xs:dayTimeDuration . The definition of those types in the [XPath/XQuery] type system is given below.

The following type definition of xs:anyType reflects the semantics of the Ur type from Schema in the [XPath/XQuery] type system.

The following type definition of xs:anySimpleType reflects the semantics of the Ur simple type from Schema in the [XPath/XQuery] type system.

The name of the Ur simple type is xs:anySimpleType. It is derived by restriction from xs:anyType, its content is a sequence any atomic types.

The following type definition of xs:anyAtomicType reflects the semantics of xs:anyAtomicType in the [XPath/XQuery] type system.

The following type definitions of the XML Schema primitive types reflect the semantics of the primitive types from Schema in the [XPath/XQuery] type system.

All of those primitive types derive from xs:anyAtomicType. Note that the value space of each atomic type (such as xs:string) does not appear. The value space for each type is built-in and is as defined in .

The type xs:untypedAtomic is defined as follows.

Note that this rule does not indicate the value space of xs:untypedAtomic. By definition, xs:untypedAtomic has the same value space as xs:string.

The following example shows two atomic values. The first one is a value of type string containing "Database". The second one is an untyped atomic value containing "Database".

The type xs:untyped is defined as follows.

The following type definitions of the XML Schema derived types reflect the semantics of the XML Schema types derived by restriction from another atomic type.

Three XML Schema built-in derived types are derived by list, as follows. Note that those derive directly from xs:anySimpleType, since they are derived by list, and that their value space is defined using a one or more occurrence indicator.

For example, here is an element whose content is of type xs:IDREFS.

Note that the type name xs:IDREFS derives from xs:anySimpleType, but not from xs:IDREF. As a consequence, calling the following three XQuery functions with the element a as a parameter succeeds for f1 and f2, but raises a type error for f3.

The totally ordered duration types, xs:yearMonthDuration and xs:dayTimeDuration , are derived by restriction from xs:duration.

In addition, the Formal Semantics uses the additional type fs:numeric. This type is necessary for the specification of some of XPath type conversion rules. It is defined as follows.

The typed value of a node is computed by the fn:data function, and the string value of a node is computed by the fn:string function, defined in . The normative definitions of typed value and string value are defined in .

Empty parentheses () always have the empty type. Remember that it is a static type error for most expressions other than () to have the empty type (see for the complete rule.)

Note that this section uses some auxiliary judgments which are defined in .

The corresponding Section in the [XPath/XQuery] document just contains examples.

[XPath/XQuery] provides several operators for combining sequences of nodes.