Report for the Workshop on Future Standards for Model-Based User Interfaces

13-14 May 2010, Rome

Hosted by the CNR-ISTI HIIS Laboratory

Call for Participation | Statements of Interest | Workshop Program

Contents

Executive Summary

This report covers the W3C Workshop on Future Standards for Model-Based User Interfaces, which was held in Rome on 13-14 May 2010. The Workshop was timed to follow the publication of the report of the W3C Model-Based UI Incubator Group. Participants presented model-based approaches from a variety of perspectives, reflecting many years of research work in this area. The Workshop's final session looked at the opportunity for launching standards work on meta-models as a basis for exchanging models between different markup languages. The work could be done with the support of the European SERENOA research project, itself part of the Framework 7 programme.

Background

Web application developers face increasing difficulties due to wide variations in device capabilities, in the details of the implementation languages they support, the need to support assistive technologies for accessibility, the demand for richer user interfaces, the suites of programming languages and libraries, and the need to contain costs and meet challenging schedules during the development and maintenance of applications.

Research work on model-based design of context-sensitive user interfaces has sought to address the challenge of reducing the costs for developing and maintaining multi-target user interfaces through a layered architecture that separates out different concerns:

  1. Application task models, data and meta-data
  2. Abstract Interface (device and modality independent, e.g. select 1 from N)
  3. Concrete Interface (device and/or modality dependent, e.g. use of radio buttons, but implementation language independent)
  4. Implementation on specific devices with wide variations in hardware and software capabilities (e.g. HTML+JavaScript, SVG, Java, .NET or Flash)

This architecture focuses on design and separates off the implementation challenges posed by specific delivery channels. The architecture enables developers to work top-down or bottom-up. The implementation or "Final-UI" can be generated automatically, subject to developer preferences. This includes the notion of UI skins where a particular style is applied to the models defined by the Concrete UI. This framework is slightly different from the MDA OMG one, which only distinguishes between two levels, and thus merges implementation and platform-dependent.

High level development suites can be provided to shield developers from the underlying representations. For example, a data model could be manipulated as a diagram, while the user interface could be defined via drag and drop operations together with editing values in property sheets. The development suite is responsible for maintaining the mappings between layers and verifying their consistency. Developers can choose to provide alternative mappings as needed to address different delivery contexts (contexts of use).

During the last year the W3C Model-Based UI XG has evaluated research on model-based user interfaces, including end-to-end models that extend beyond a single Web page, and has assessed its potential as a framework for developing context-sensitive Web applications.

Workshop Overview

The Workshop started with a presentation on the MBUI Incubator Group which had recently closed after publishing its final report. We then heard about the role of model-based approaches for interactive application design, including UsiXML, useML, UIML, the use of RDF in the Semantic Hypermedia Design Method (SHDM), and the authoring of service-based applications through a tool also able to transform task models expressed as ConcurTaskTrees (CTT) into UI descriptions represented in MARIA XML.

This was followed by a session on run-time adaptation using model-based aproaches. This caters for the case where the devices vary considerably and the precise capabilities are unknown at design time. Likewise for the users and the environment. A model-based approach enables the design rationale to be preserved at run-time. MyMobileWeb approaches this with IDEAL2 for device adaptation and SCXML for modeling application flow. Another case is with migratory interfaces where the users have the freedom to switch from one device to another during the use of an application and continue from the point where they left off.

After lunch, we switched to a discussion of the role of formal semantics, and ontologies for user interfaces and interactions. We heard about LEONARDI, a model driven-framework for composing business applications. This was followed by a session on user interface models and related standards.

XML is a popular basis for model-based approaches, starting with native XML databases, and using XQuery and XSLT for processing. SCXML can be used for work flow definition, and XForms for UI with work arounds for synchronization. A strong justification for declarative techniques is the reduction in development time compared with procedural approaches where the designers intent is implicit in the code, thereby increasing the likelihood of bugs, in particular when multiple, platform-dependent versions of an application should be delivered.

Declarative approaches also lend themselves to increased accessibility of applications because they make the design choices more explicit, and model-based techniques retain this benefit even when the final UI doesn't itself use declarative representations. For example, WAI/ARIA includes model-based concepts, such as that of role and state. Model-based approaches are also applicable to multimodal interaction. EMMA is an XML language for annotating interpretations of user input from one or more modes e.g. speech and gestures. This can be used as part of loosely coupled architectures involving multiple devices.

We also heard about work on MDA (Model-Driven Architecture) including the use of UML for domain models and state chart diagrams for behavior. The ISO/IEC 24752 standard deals with universal remote consoles. It defines a pluggable XML language that enables mobile phones, computers and other devices to act as controllers for compatible devices over a variety of communications media, e.g. TVs, DVD players and media servers. CEA 2018 could be used as a possible task model engine.

In earlier work on model-based design, models are used as part of the initial design phase, and compiled into code stubs for programmers to elaborate. For model-based techniques to be effective throughout the life cycle of an application, it is necessary to retain the means to move between different levels of models in both directions.

Current practice in Web engineering often lacks a systematic approach, with no notion of context and limited if any concepts for adaptation. This is heightened by the advent of large very high resolution displays. Web pages designed for smaller displays look awkward on these larger displays. Adaptation is needed for content and navigation. Work arounds include libraries such as jQuery Masonry using JavaScript and CSS. Multicolumn layout in CSS3 will help, but further work is needed on high level authoring frameworks, which should allow people who are not expert in programming to participate in work on design and development.

MBUI has great promise for new kinds of applications made possible through an ever increasing range of networked devices. What kinds of models are needed for the Web of Things? This includes rich models of users, devices and the environment in which they reside. Such models need to be live so that applications can respond dynamically to changes. The models also need to support distributed applications that couple sensors and actuators into the Cloud with intermediate servers acting as gateways and localized controllers.

Multimedia and synchronization present further challenges. Can we combine temporal models, multimodality models and user interface models? Other challenges include the role of model-based techniques for testing, e.g. for applications in safety critical situations.

For more details and access to the indvidual presentations and minutes, see the workshop program.

Recommended Next Steps

The Workshop's final session focused on next steps, and the opportunity for launching new standards work at W3C. This should address model-based techniques for Web applications with a view to encouraging interoperability between a new generation of authoring tools, and aimed at reducing the costs and development time whilst supporting design and run-time adaptation to the context.

The advent of rich web applications and increasing variety of devices make this especially timely after some 20 years of research, and given that other organizations have been working in related areas for some time, e.g. MDA at OMG, and task models at ANSI (CEA 2018).

Scope

The main focus should be on meta-models as a basis for exchange between different markup languages at different layers in the Cameleon reference framework. This should address the needs for adaptation, validation, accessibility and multimodal interaction. Further considerations include migratory user interfaces, and user interfaces involving multiple devices and distributed applications. A key target for the final UI would be HTML5, SVG and associated JavaScript libraries, e.g. jQuery and Dojo. Target devices would include mobile, notepad, desktop, and large screens on tables or walls, together with gestural input.

Stakeholders and Benefits

These would include tool developers, application developers, and website owners/operators. Benefits include improved quality, reduced development time and cost, improved accessibility, more effective communication between members of development teams, easier testing, and reduced effort for adapting to changing requirements.

Who would join the Working Group?

The following is indicative and subject to review of the proposed charter by each organization:

Potential Liaisons

within W3C:

externally:

Next steps

The Workshop participants recommended that a small group of volunteers be formed with a view to writing a draft charter in September 2010. Out-reach to potential Working Group participants will be important, and one avenue is through related conferences.

Further details are given in the minutes of the Workshop's final session.

Acknowledgements

The co-chairs and workshop participants would like to express their gratitude to W3C and to the CNR-ISTI HIIS Laboratory for hosting the workshop.

photo of participants

Call for Participation | Statements of Interest | Workshop Report

Valid XHTML Basic 1.0 Dave Raggett and Fabio Paternò, Workshop co-Chairs

$Id: report.html,v 1.21 2010/06/28 11:49:22 dsr Exp $