Model-Based UI XG Final Report

2.2.1 The Context of Use

Context is an all-embracing term. Composed of “con” (with) and “text”, context refers to the meaning that must be inferred from the adjacent text. As a result, to be operational, context can only be defined in relation to a purpose, or finality [CRO02]. In the field of context-aware computing a definition of Context that has been largely used is provided by [DEY00]: Context is any information that can be used to characterize the situation of entities (i.e. whether a person, place or object) that are considered relevant to the interaction between a user and an application, including the user and the application themselves. Context is typically the location, identity and state of people, groups and computational and physical objects.

While the above definition is rather general, thus encompassing many aspects, it is not directly operational. Hence, we hereby define the Context of Use of an interactive system as a dynamic, structured information space that includes the following entities:

• a model of the User, U, (who is intended to use or is actually using the system)
• the hardware-software Platform, P, (which includes the set of computing, sensing, communication, and interaction resources that bind together the physical environment with the digital world)
• the social and physical Environment, E, (where the interaction is actually taking place).

Thus, a context of use is a triple composed by (U, P, E)

The User represents the human being (or a human stereotype) who is interacting with the system. The characteristics modelled or relevant can be very dependant on the application domain. Specific examples are age, level of experience, the permissions, preferences, tastes, abilities and disabilities, short term interests, long term interests, etc. In particular, perceptual, cognitive and action disabilities may be expressed in order to choose the best modalities for the rendering and manipulation of the interactive system.

The Platform is modeled in terms of resources, which in turn, determine the way information is computed, transmitted, rendered, and manipulated by users. Examples of resources include memory size, network bandwidth, and input and output interaction devices. [CCB02] distinguishes between elementary platforms (e.g. laptop, PDA, mobile phone), which are built from core resources (e.g. memory, display, processor) and extension resources (e.g. external displays, sensors, mice), and clusters, which are built from elementary platforms. Resources motivate the choice for a set of input and output modalities and, for each modality, the amount of information made available. W3C's Delivery Context Ontology [DCONTOLOGY] is intended to define a concrete Platform Model.

The Environment denotes the set of objects, persons and events that are peripheral to the current activity but that may have an impact on the system and/or users behavior, either now or in the future (Coutaz and Rey, 2002). According to our definition, an environment may encompass the entire world. In practice, the boundary is set up by domain analysts whose role is to elicit the entities that are relevant to the case at hand. Specific examples are: user's location, ambient sound, lighting or weather conditions, present networks, nearby objects, user's social networks, level of stress ...

The relationship between a UI and its contexts of use leads to the following definitions:

Multi-target (or multi-context) UI

A multi-target (or multi-context) UI supports multiple types of users, platforms and environments. Multi-user, multi-platform and multi-environment UIs are specific classes of multi-target UIs which are, respectively, sensitive to user, platform and environment variations. [CCTLBV03]

Adaptive UI

An Adaptive UI refers to a UI capable of being aware of the context of use and to (automatically) react to changes of this context in a continuous way (for instance, by changing the UI presentation, contents, navigation or even behaviour).

Adaptable UI

An Adaptable UI can be tailored according to a set of predefined options. Adaptability normally requires an explicit human intervention. We can find examples of UI adaptability on those word processors where the set of buttons contained by toolbars can be customized by end users.

Plastic UI

A Plastic UI is a multi-target UI that preserves usability across multiple targets. Usability is not intrinsic to a system. Usability can only be validated against a set of properties set up in the early phases of the development process. [CCTLBV03]

2.2.2 Abstraction Levels

The CAMELEON Reference Framework, structures the development life cycle into four levels of abstraction, from the task specification to the running interface (see Figure 1):

• The Task and Concepts level (corresponding to the Computational-Independent Model–CIM–in MDE) which considers: (a) the logical activities (tasks) that need to be performed in order to reach the users’ goals and (b) the domain objects manipulated by these tasks. Often tasks are represented hierarchically along with indications of the temporal relations among them and their associated attributes.
• The Abstract User Interface (AUI) (corresponding to the Platform-Independent Model–PIM– in MDE) is an expression of the UI in terms of interaction spaces (or presentation units), independently of which interactors are available and even independently of the modality of interaction (graphical, vocal, haptic …). An interaction space is a grouping unit that supports the execution of a set of logically connected tasks.
• The Concrete User Interface (CUI) (corresponding to the Platform-Specific Model–PSM– in MDE) is an expression of the UI in terms of “concrete interactors”, that depend on the type of platform and media available and has a number of attributes that define more concretely how it should be perceived by the user. "Concrete interactors" are, in fact, an abstraction of actual UI components generally included in toolkits.
• The Final User Interface (FUI) (corresponding to the code level in MDE) consists of source code, in any programming language or mark-up language (e.g. Java, HTML5, VoiceXML, X+V, ...). It can then be interpreted or compiled. A given piece of code will not always be rendered on the same manner depending on the software environment (virtual machine, browser …). For this reason, CAMELEON considers two sublevels of the FUI: the source code and the running interface

These levels are structured with a relationship of reification going from an abstract level to a concrete one and a relationship of abstraction going from a concrete level to an abstract one. There can be also a relationship of translation between models at the same level of abstraction, but conceived for different contexts of use. These relationships are depicted on Figure 1.

2.4.1 Transformation Steps

[LV09] describes different kinds of transformation steps:

• Reification covers the inference process from high-level abstract descriptions to run-time code. The CAMELEON Reference Framework recommends a four-step reification process: a Concepts-and-Tasks Model is reified into an Abstract UI which in turn leads to a Concrete UI. A Concrete UI is then turned into a Final User Interface, typically by means of code generation techniques.
• Code generation is a particular case of reification which transforms a Concrete UI Model into compilable or interpretable code.
• Translation is an operation that transforms a description intended for a particular target into a description at the same abstraction level but aimed at a different target.
• Reflection transforms a UI representation at a given level of abstraction to another UI representation at the same level of abstraction for the same context of use.
• Abstraction is an operation intended to map a UI representation from one non-initial level of abstraction to a higher level of abstraction. In the context of reverse engineering, it is the opposite of reification.
• Code reverse engineering is a particular case of abstraction from executable or interpretable code to models.

Figure 2 - Forward and Reverse Engineering Development Paths

2.4.2 Development Paths

Transformation types have been introduced in the previous section. These transformation types are instantiated into development steps. These development steps may be composed to form development paths. Several types of development paths are identified by [LV09]:

• Forward engineering (Figure 2 left) is a composition of reification and code generation enabling a transformation of a high-level viewpoint into a lower-level viewpoint.
• Reverse engineering (Figure 2 right) is a composition of abstractions and code reverse engineering enabling a transformation of a low-level, viewpoint into a higher level viewpoint.
• Context of use adaptation is a composition of a translation with another type of transformation enabling a viewpoint to be adapted in order to reflect a change in the context of use of a UI.
• Middle-out development: This term refers to a situation where a programmer starts a development by a specification of the UI (no task or concept specification is priorly built). Several contributions have shown that in reality, a development cycle is rarely sequential and even rarely begins by a task and domain specification. Literature in rapid prototyping converges with similar observations. Middle-out development shows a development path starting in the middle of the development cycle e.g., by the creation of a CUI or AUI model. After several iterations of this level (more likely until customer's satisfaction is reached) a specification is reverse engineered. From this specification the forward engineering path is followed.
• Retargeting: This transition is useful in processes where an existing system should be retargeted, that is, migrated from one source computing platform to another target computing platform that poses different contraints. Retargeting is a composition of reverse engineering, context adaptation and forward engineering. In other workds a Final UI code is abstracted away into a CUI (or an AUI). This new CUI and/or AUI is reshuffled according to specific adaptation heuristics. From this reshuffled CUI and/or AUI specification a new interface code is created along a forward engineering process.

The CAMELEON Reference Framework promotes a four-step forward engineering development path starting with domain concepts and task modelling. Although research in HCI has promoted the importance of task modelling, practicioners often skip this stage, and directly produce CUIs using protyping tools such as Flash because of the lack of tools allowing rapid prototyping from task models. This practice corresponds to the last two steps of the reification process recommended in the reference framework. Nonetheless, the framework can be instantiated with the number of reification steps that fits designers culture. In other words, designers can choose the entry point in the reification process that best fits their practice. If necessary, the missing abstractions higher in the reification process can be retrieved through reverse engineering

3.1.1 Context of Use Model (NEXOF-RA)

Figure 4 depicts graphically the "Context of Use" Model proposed by the NEXOF-RA Project [NEXOF-RA]. Such a Model captures the Context of Use in which a user is interacting with a particular computing platform in a given physical environment in order to achieve an interactive task.

Context of Use is the main entity which has been modelled as a an aggregation of User, Platform and Environment. They are all Context Elements. A Context Element is an instance of Context Entity. A Context Property represents a characteristic of a Context Element or information about its state. A Context Property might be associated to zero or more instances of Context Value. Examples of Context Property instances are: 'position', 'age' or 'cpuSpeed'. There can be Context Property instances composed by other sub-properties. For example, the 'position' property is typically composed by: 'latitude', 'longitude' and 'altitude'. Both a Context Property and a Context Value can be associated to different metadata represented by the Context Property Description and Context Value Description classes respectively. A Context Property can be obtained from different Context Providers. A Device Description Repository (DDR) [DDR-REQUIREMENTS] [DD-LANDSCAPE] is a Context Provider of information about the "a priori known" characteristics of Platform Components, particularly devices or web browsers.

The model below also describes a simple, abstract conceptual model for the Platform. A Platform can be represented as an aggregation of different Aspect [DDR-SIMPLE-API] instances (device, web browser, network, camera, ...), which are called Components. To this aim we have splitted this relationship into three different aggregations: active, default and available. active indicates that a Component is "running". For example, if a camera is "on" such a Component is said to be "active". default conveys what is the referred Aspect instance when there is no an explicit mention of a specific one. Finally, available represents what are the "ready to run" Components. For example, when a device has more than one web browser installed, the "available web browsers" property value will be a list containing a reference to the web browsers that could (potentially) be put into running.

Figure 4 - Context of Use Model (NEXOF-RA Project)

3.1.2 Platform Model : W3C's Delivery Context Ontology

The Delivery Context Ontology (DCO) [DCONTOLOGY] is a W3C specification (work in progress) which provides a formal model of the characteristics of the environment in which devices interact with the Web or other services. The Delivery Context, as defined by [DI-GLOSS], includes the characteristics of the Device, the software used to access the service and the Network providing the connection among others. DCO is intended to be used as a concrete, standard Platform Model, even though due to convenience reasons it also models some Environment entities.

Figure 5 gives an overview of the main entities modelled by DCO. The root entity is the DeliveryContext class which is linked to the currentDevice, currentUserAgent, currentNetworkBearer and currentRuntimeEnvironment. The former are in fact active Components from the point of view of the Context of Use Model presented on the previous section. The Device class has been modelled as an aggregation of DeviceSoftware and DeviceHardware. In addition DCO also models some elements of the Environment such as the Location or the Networks present.

Figure 5 - Delivery Context Ontology : Main Entities

3.1.3 GUMO and UserML

GUMO and UserML are two formalisms proposed by [HCK06] in order to deal with the problem of representing generic user models. The SITUATIONALSTATEMENTS and the exchange language UserML work on the syntactical level, while the general user model ontology GUMO has been developed (using OWL) on the semantical level. SITUATIONALSTATEMENTS represent partial descriptions of situations like user model entries, context information or low-level sensor data. SITUATIONALSTATEMENTS follow a layered approach of meta level information arranged in five boxes: mainpart , situation, explanation, privacy and administration. These boxes have an organizing and structuring functionality. An example of a SITUATIONALSTATEMENT represented in the UserML language can be seen below

    <statement>
        <mainpart>
            <subject>Peter</subject>
            <auxiliary>hasProperty</auxiliary>
            <predicate>walkingSpeed</predicate>
            <range>slow-medium-fast</range>
            <object>fast</object>
        </mainpart>
        <situation>
            <start>2010-04-09T19:20</start>
            <end>?</end>
            <durability>few minutes</durability>
            <location>airport.dutyfree</location>
            <position>X,Y,Z</position>
        </situation>
        <explanation>
            <source>sensor.repository</source>
            <creator>sensor.PW</creator>
            <method>Bayes</method>
            <evidence>LowLevelData</evidence>
            <confidence>0.8</confidence>
        </explanation>
        <privacy>
            <key>?</key>
            <owner>Peter</owner>
            <access>friends-only</access>
            <purpose>research</purpose>
            <retention>1 week</retention>
        </privacy>
    </statement>

The main conceptual idea in the approach of SITUATIONALSTATEMENTS that influences the construction of the GUMO ontology is the division of descriptions of the user model dimensions into three parts: auxiliary, predicate and range as shown in the example above. As a matter of fact, the range attribute offers a new degree of freedom to the ontology definition: it decouples the definition for the predicate from possible range scales.

For example if one wants to say something about the user’s interest in football, one could divide this so-called user model dimension into the auxiliary part “has interest”, the predicate part “Football” and the range part “low-medium-high” as shown in the figure below. Likewise, If a system wants to express something like the user’s knowledge about Beethoven’s Symphonies, one could divide this into the triple “has knowledge”, “Beethoven’s Symphonies” and “poor-average-good-excellent”

Peter {hasInterest,Football,low-medium-high} low
Peter {hasKnowledge,Beethoven's Symphonies,poor-average-good-excellent} good

The implication for the general user model ontology GUMO of these examples above is the clear separation between user model auxiliaries, predicate classes and special ranges. What leads to a tricky problem is that actually everything can be a predicate if the auxiliary is “interest” or “knowledge”.

Information in the situation box is responsible for the temporal and spatial embedding of the whole statement in the real physical world. With this open approach one can handle the issue of the history in user modeling and context-awareness. Particularly the attribute durability carries the qualitative time span of how long the statement is expected to be valid (minutes, hours, days, years). In most cases when user model dimensions or context dimensions are measured, one has a rough idea about the expected durability, for instance, emotional states change normally within hours, however personality traits will not change within months.

The GUMO Ontology defines both User Model Dimensions (e.g. hasInterest, hasKnowledge, hasProperty, hasBelieve, hasPreference) and User Model Auxiliaries (e.g. Ability And Proficiency, Personality, Emotional State, Physiological State, Mental State). However, as stated above, it turned out that actually any concept in the whole world can be needed to express user model data. To overcome such an issue the GUMO authors propose to join in any other OWL ontology, Linked Data, or to use the UBISWORLD Ontology. UBISWORLD can be used to represent some parts of the real world like an office, a shop, a museum or an airport. It represents persons, objects, locations as well as times, events and their properties and features.

3.2.1 ConcurTaskTrees (CTT)

CTT is a notation for task model specifications which has been developed to overcome limitations of notations previously used to design interactive applications. Its main purpose is to be an easy-to-use notation that can support the design of applications of any degree of complexity.

The main features of CTT are:

• Hierarchical structure, providing different levels of granularity and allowing large and small task structures to be reused, at both a low and a high semantic level.
• Graphical syntax, CTT task models are represented as icons and trees.
• Concurrent notation, operators for temporal ordering are used to link subtasks at the same abstraction level. Such semantic ordering determines the user tasks that should be active at any time.
• Focus on activities, thus it allows designers to concentrate on the most relevant aspects when designing interactive applications that encompass both user and system-related aspects avoiding low levels implementation details that at the design stage would only obscure the decisions to take.

A set of tools to develop task models in ConcurTaskTrees, to analyse their content and to generate corresponding UIs are being developed and are available at [CTTE].

The figure below shows a CTT task model which describes an ATM UI. It has been modelled as two different abstract tasks (depicted as a cloud): EnableAccess and Access. There is an enabling temporal relationship (>>) between them, which indicates that the Access task can only be performed after the successful completion of the EnableAccess task. If we have a look at the EnableAccess task it can be seen that it have been splitted into: two interaction tasks (InsertCard, EnterPassword) and an application (system-performed) task (RequirePassword). Likewise, the Access task has been decomposed into: WithdrawCash, DepositCash and GetInformation. These tasks are related by means of the choice ([]) operator which indicates that different tasks can be chosen, but once a task is chosen the other will not be available until the former is finished. It can be observed, particularly, the DecideAmount task which it is a user task, representing a cognitive activity. The []>> symbol indicates an enabling with information passing relationship, which means that there also exists an information flow between the concerned tasks.

Figure 7 - CTT Task Model for the User Interface offered by an ATM

3.2.2 ANSI/CEA-2018 Task Model Description Standard

ANSI/CEA-2018 [Rich09] defines an XML-based language for task model descriptions. The standard was published by CEA in Nov. 2007 [CEA2007], and by ANSI in March 2008. In this standard a task model is defined as a formal description of the activities involved in completing a task, including both activities carried out by humans and those performed by machines. The standard defines the semantics and an XML notation for task models relevant to consumer electronics devices, but nothing prevents anybody from using it in a broader domain.

The figure below shows an XML excerpt of an ANSI/CEA-2018 task model for playing music on an entertainment system consisting of a media server and a media player [MBUI-CEA2018]. It can be observed that the main task is decomposed into different subtasks (steps). Steps are sequential by default, in the order as defined in the XML structure. Tasks have input and output slots, representing the data to be communicated with other tasks. Restrictions over such data are expressed by means of Preconditions and Postconditions. Bindings specify the data flow between the input and output slots of a task and its subtasks, and those between the subtasks.

3.3.1 MARIA AUI

MARIA [PSS09] Model-based lAnguage foR Interactive Applications, is a universal, declarative, multiple abstraction level language for service-oriented applications in ubiquitous environments. It provides a flexible dialogue and navigation model, a flexible data model, which allows the association of various types of data to the various interactors, and support for more recent techniques able to change the content of UIs asynchronously respect to the user interaction.

Figure 9 shows the main elements of the abstract user interface metamodel (some details have been omitted for clarity). As can be seen, an interface is composed of one data model and one or more presentations. Each presentation is composed of name, a number of possible connections, elementary interactors, and interactor compositions. The presentation is also associated with a dialog model which provides information about the events that can be triggered at a given time. The dynamic behavior of the events, and the associated handlers, is specified using the CTT temporal operators (for example, concurrency, or mutually exclusive choices, or sequentiality, etc.).

Figure 9 - MARIA AUI Meta-Model

When an event occurs, it produces a set of effects (such as performing operations, calling services, etc.) and can change the set of currently enabled events (for example, an event occurring on an interactor can affect the behavior of another interactor, by disabling the availability of an event associated to another interactor). The dialog model can also be used to describe parallel interaction between user and interface. A connection indicates what the next active presentation will be when a given interaction takes place. It can be either an elementary connection, a complex connection (when Boolean operators compose several connections), or a conditional connection (when specific conditions are associated with it). There are two types of interactor compositions: grouping or relation. The latter has at least two elements (interactor or interactor compositions) that are related to each other.

In MARIA an interactor can be either an interaction object or an "only output" object. The first one can be one of the following types: selection, edit, control, interactive description, depending on the type of activity the user is supposed to carry out through such objects. The control object is refined into two different interactors depending on the type of activity supported (navigator: navigate between different presentations; activator: trigger the activation of a functionality). An only output interactor can be object, description, feedback, alarm, text, depending on the supposed information that the application provides to the user through this interactor.

It is worth pointing out that further refinement of each of these interactors can be done only by specifying some platform-dependent characteristics, therefore it is specified at the concrete level.

3.3.2 UsiXML AUI Meta-Model

UsiXML [LVMBL04] [UsiXML-Proj] is an XML-compliant markup language, which aims to describe the UI for multiple contexts of use. UsiXML adheres to MBUI by having meta-models describing different aspects of the UI. At the time of writing a new version of UsiXML is under development thanks to the support of an Eureka ITEA2 project [UsiXML-Proj].

The figure below depicts the current version of the UsiXML Meta-Model for AUI description (work in progress). The class AUIobject is at the top of the hierarchy representing the elements that populate an AUI Model. AUIInteractor and AUIContainer are subsumed by AUIObject. The latter defines a group of tasks that have to be presented together and may contain both AuiInteractors or other AuiContainers. An association class AuiRelationship allows to define the kind of relationship (Ordering, Hierarchy, Grouping, or Repetition) between an object and its container. AUIInteractionUnit is an aggregation of AUIObject and Behaviour specified by means of Listener, Event and Action. AuiInteractor has been splitted into DataInteractor (for UI data input/output) or TriggerInteractor (for UI command). Selection,Input and Output are data interactors. Concerning trigger interactors, Command is intended to launch any kind of action within the UI whilst Navigator allows to change the interaction unit.

Figure 10 - Abstract User Interface Meta-Model in UsiXML

3.4.1 UsiXML CUI Meta-Model

Figure 11 is the graphical representation of the UsiXML Meta-model for the Concrete UI (work in progress). The root entity is CUIObject which has been subclassed in CUIInteractor and CUIContainer. The relationship between Interactors and Containers is captured by the 'contains' relationship and the CUIRelationship association class. It is important to note that the meta-model includes specializations for the different modalities (graphical, tactile, vocal), as a CUI Model is modality-dependent. The Style class is intended to capture all the presentational attributes for a CUI Object. This design pattern decouples the model from 'presentational vocabularies'.

Figure 11 - UsiXML CUI Meta-Model

Figure 12 depicts the hierarchy of GraphicalInteractor modelled by UsiXML. As it can be observed, the typical graphical interactors found in conventional toolkits are included.

Figure 12 - UsiXML CUI Graphical Interactors

The UsiXML meta-models presented above are still under development, thus many issues are open. For example, layout representation, bindings between the Domain Model and the AUI / CUI, model modularization and extension, etc.

3.6.1 Research-Driven

• RIML [DWGSZ03] is a markup language based on W3C standards that allows document authoring in a device independent fashion. RIML is based on standards such as XFORMS. Special row and column structures are used in RIML to specify content adaptation. Their semantics is enhanced to cover pagination and layout directives in case pagination needs to be done. Due to the use of XForms, RIML is device independent and can be mapped into a XHTML specification according to the target device. RIML semantics is enhanced to cover pagination and layout directives in case pagination needs to be done, in this sense it was possible to specify how to display a sequence of elements of the UI.
• SHDM [ROS07] [MOU04] is a model-driven approach to design web applications through six different steps: Requirements Gathering, Domain Model Design, Navigational Design, Behavior Design , Abstract Interface Design and finally implementation. Each phase focuses on a particular aspect and produces artifacts (models) detailing the application to be run on the web. Accordingly, in SHDM a Web application is defined as a navigational view over some ontology that describes the problem domain (i.e., a domain model). An abstract interface is essentially a composition of abstract widgets, which may be of four possible types: Exhibitors, used to present information to the user; Capturers, used to receive input from the user; Activators, used to signal the occurrence of an event to the application, and Composites, which are aggregations of other widgets.
• TERESA [PSM08], is a XML-based language for describing UIs, which has an associated authoring environment, Multimodal TERESA. It provides designers with the possibility of designing interfaces for a wide set of platforms, which support various modalities.
• UIML [APB99] (User Interface Markup Language) was one of the first model-based languages for describing UIs. A UI is decomposed into structure, style, contents, and behaviour. It is however only partially compliant with the CAMELEON Reference Framework (e.g., it does not have any task or context model) and it has not been applied to obtain multi-target user interfaces or to context-aware adaptation.
• useML is a notation for specifying enhanced task models in industrial environments and is part of the user-centered Useware-engineering development process [ZT08]. Originally developed in 2003 [MT08], useML was enhanced in 2009 with several aspects concerning temporal operators, conditions and optionality of tasks. useML has shown its applicability and usefulness in several other domains e.g. automotive or medicine [MTK07]. UseML is embedded in a model-based architecture for developing multimodal and multiplatform UIs. This model-based architecture was developed as an instance of the CAMELEON Reference Framework using different abstraction layers and different UIDLs. For editing useML the graphical useML-Editor (Udit) [MSN09] was developed. Furthermore useML was extended to work in ambient intelligence factory environments like e.g. the SmartFactoryKL [BMGMZ09] which enables the run-time generation of graphical UIs.
• XIML [EVP00], [EVP01] is composed of four types of components: models, elements, attributes, and relations between the elements. The presentation model is composed of several embedded elements, which correspond to the widgets of the UI, and attributes of these elements representing their characteristics (color, size…). The relations at the presentation level are mainly the links between labels and the widgets that these labels describe. XIML supports design, operation, organization, and evaluation functions; it is able to relate the abstract and concrete elements of an interface; and it enables knowledge-based systems to exploit the captured data.

3.6.2 Industry-Driven

• Collage IBM's Collage [Collage] is a declarative programming language and runtime for cumulative building of data-centric, reactive systems composed by distributed web components. Nodes of the underlying RDF data model are dynamically typed, interpreted and updated in response to occurrence of external and internal events (user input, service responses, data computations etc.). Collage's recursive MVC-approach allows for arbitrary detail of UI-specification ranging from abstract UI primitives (adopted from XForms) to concrete layout overlays.
• Flex Adobe's Flex [Flex] comprises an open source framework for creation and deployment of Flash-based applications. While leveraging ActionScript for implementation Flex offers an higher-level XML-syntax (MXML) for declarative specification of application and user interface components. This covers features like web service requests, data-binding and validation and a rich, extensible library of UI-controls, containers and animation effects. MXML files are compiled into Flash bytecode (SWF) for platform-neutral execution on client side within browsers (via Flash Player) or as standalone desktop applications (via Adobe AIR runtime).
• Open Laszlo Open Laszlo [OpenLaszlo] is an open source platform for development and delivery of rich internet applications (RIA). These are defined in LZX and JavaScript and deployed either as static, pre-compiled binaries (DHTML, Flash) or rendered dynamically by the OpenLaszlo Server into a device-specific OpenLaszlo client application (DHTML, Flash). The XML dialect LZX resembles HTML, while supporting high level UI (sliders, trees, grids) and action elements (animator) an XML-based data model and declarative dependencies at view level (constraints) or data level (data pathes).
• XAML Microsoft's XAML [XAML] serves several purposes within the .NET framework: a declarative definiton of visual user interfaces for desktop applications (via Windows Presentation Foundation) and web (RIA via Silverlight) comprising a hierarchical model of 2D, 3D objects and media, flow control, data binding, eventing, transformations and styling through a templating mechanism. It may as well describe long-running processes executed via Windows Workflow Foundation.
• XForms [XFORMS11] XForms is a widely adopted W3C-standard targeting the next generation of (web) form applications. Following the MVC-design pattern these operate on a model comprising data-oriented XML-instances enhanced e.g. by validity restrictions, node computations expressed in XPath [XPATH] and a variety of data submission models. The event-based controller layer leverages XML-Events [XML-EVENTS] and a rich set of predefined actions eliminating the need for imperative programming (Javascript). The generic, device-independent and extensible set of user controls support advanced interactions like tabs and wizard-like page flow. For rendering purposes XForms markup is embedded into a presentation oriented host language (HTML, SVG) and additionaly formated via CSS. A powerful, thoroughly XML-based architecture arose from the combination of XForms clients and native XML-database servers exposing (stored) XQuery [XQUERY] statements through a REST-interface (XRX).
• XUL XUL [XUL] is a component of the Mozilla browser and related applications and is available as part of Gecko [Gecko]. With XUL and other Gecko components, developers can create sophisticated applications without special tools. XUL was designed for creating the user interface of Mozilla applications including the web browser, mail client and page editor. In XUL developers can describe a concrete UI using a markup language, use CSS style sheets to define appearance and JavaScript for behavior. Programming interfaces for reading and writing remote content over the network and for calling web services are also available. Unlike HTML, however, XUL provides a powerful set of interactors for creating menus, toolbars, tabbed panels, and hierarchical trees to give a few examples.