Interaction management for Web Applications

Abstract

The World Wide Web is entering its adolescence, and there are many opportunities for new kinds of applications, each demanding different styles of interaction. This paper covers just a few applications, and indicates how they can be realized with a combination of existing and new standards.

Introduction

The Web has made do with HTML, CSS, scripting and plugins for the last ten years. The time has come to look at fresh possibilities. This paper takes a look at a small number of web applications that between them illustrate a wide range of behaviors. In so doing we will address the following ideas:

Travel expense claim spreadsheet

Most of us loath having to fill out expense claims after a business trip. Rather than struggling with Excel or a poorly designed Java based form, wouldn't it be nice to be able to use an attractive and well thought out XML-based Web application? This application uses SVG for an attractive presentation along with XForms for a clean separation between the presentation and the data logic.

The problem with traditional spreadsheets is that they lack an explicit data model, and everything is mixed up with the layout in rows and columns. In doing a make-over, the first step is to define the XML we would like to use to represent expense claims. This combines information about the person making the claim and the trips covered, together with a number of individual expenses. Each expense has a date, a type (taken from an enumeration), a free form description, the amount being claimed and optionally, the currency and exchange rate. The application should auto-number items and show the accumulated totals. XForms provides a means for describing the data in terms of fixed and repeating elements, together with any integrity constraints. This can be combined if needed with an XML Schema that can subsequently be used to validate expense claims submitted by the application.

The presentation could be represented directly in SVG together with a script written in ECMAScript to implement the interaction behavior. For instance, when the input focus is moved to the expense type, the user should be able to click on a drop down menu of allowed types. A decent application should perhaps autofill the field based on the previous choice, and further present the n-most recently used types for quick selection, to avoid the need for repeatedly scrolling down a long list of types. Users should also be able to use keystrokes to type values with an autocompletion mechanism, since this avoids the need to lift your hands from the keyboard.

Autofill and autocompletion can also be used for other fields, such as the expense description, the currency code and exchange rate. Because there can be many expense items, there needs to be a way to add new items into the presentation. One way is to automatically add a new blank row whenever the last row isn't empty. Another is to have a single set of widgets for entering an expense item, together with a read-only presentation of the items already entered. This is slightly complicated by the need to be able to revise or delete previously entered items. A couple of buttons and a reasonable choice of keystroke bindings for the enter and delete keys solves this problem nicely.

XForms provides a declarative alternative to hacking the interface with ECMAScript. In addition to providing a way to describe the data, it provides a set of abstract user interface components using XPath to bind to that data, and XML Events as a means to bind handlers for DOM2 events. This makes it very easy to constrain the expense type to an enumerated set, or to show the accumulated total for the claim. Unfortunately, the XForms 1.0 Recommendation doesn't specify how to bind these abstract controls to SVG.

What's missing?

There are a number of ways we could go. One possibility would be to use a mix of markup for SVG and XForms controls. This would be messy, and it would be preferable to have a way to separate the data binding from the details of how each control is rendered with SVG along with its detailed interaction behavior.

Another issue is the means for authors to specify their layout intentions. Cascading Style Sheets (CSS) evolved to describe the presentation and layout features supported by HTML browsers. CSS offers a mix of fixed and scalable measures for units of length, but lacks policy based layout mechanisms common to many modern windowing environments, although there have been proposals for adding these, see for example: Frame-based layout via Style Sheets, by Bert Bos, David Raggett, and Håkon Wium Lie, from June 1996.

SVG itself allows for very rich presentations, but as previously noted, it would be desirable to separate the layout intentions from the details of the presentation. In other words, to be able to use clean markup that describes the layout and user interface controls, but which leaves the detailed presentation of both to a separate presentation sheet. This draws on inspiration from modern object oriented graphical interface libraries like Gnome and GTK+ and UI design tools like Glade. These provide a suite of layout controls and widgets, along with a clean separation from their presentation. The presentation is controlled via theme engines that interpret declarative representations of the behavior and presentation for each kind of control.

Some people might argue that the markup used for the form should be free of all layout information, and that that should be entirely defined in a style sheet. For instance, to use div elements for all such containers. Some authoring tools go to the other extreme, embedding all of the presentation information into the markup. Most people will feel more comfortable with the middle ground, where the markup indicates layout intentions, but presentation details are held separately and can be easily modified without changes to the rest of the application.

We should also consider what is needed to provide really ergonomic interfaces. For example, how best to support the desired autofill and autocompletion mechanisms for the travel expense application. For drop-down selection menus autocompletion can be simply provided by the implementation based upon the set of choices available. What about autocompletion on the expense description fields?

Current spreadsheets build a dictionary of the text entered by the user across all cells. That doesn't require any effort on the behalf of the application author. A further idea is to build a dictionary for all input fields bound to the same element in the XForms data model, i.e. to restrict autocompletion in expense description fields to values previously entered into such fields. This too needn't be something that the author is responsible for.

A similar argument applies to tracking a list of favorites for fields of the same type, which can be presented as part of a drop down menu, or used to guide autocompletion by computing the most likely completion based upon previous inputs. This would appear to be an area where implementations can compete to provide the most ergnonomic interfaces. A closing thought is how to preserve the dictionaries and statistics from one use of an application to the next. In principle, this could be cached based upon the URI of the web application.

Industrial process real-time simulator

Ever wondered how people learn to control the nuclear power plant in a submarine, or chemical plant in an oil refinery? Making a mistake on the job could cost lives and cause an ecological disaster. It therefore pays to learn the ropes on a software simulation, where you can get to grips with all kinds of eventualities, without fear of the consequences of failure. This application uses SVG for a pretty animated presentation. The simulation is described in XML coupled with Java classes that implement the real-time calculations involved.

The idea for this use case is to use the XForms binding mechanism to bind the UI to the domain data model, together with the use of application specific markup and Java classes. The novel part is the use of a mechanism that binds each element to a Java class, and the unusual nature of the UI controls themselves.

Engine fault diagnosis assistant

Want to learn how to diagnose faults in the complex engines in an oil tanker, a thousand miles from land? Fault diagnosis is a collaborative exercise where the application and user work together to understand and fix the problems. This application makes use of domain specific models expressed in XML to drive the dialog between the application and user. These models describe how to breakdown tasks into others, the causal links between different subsystems, and domain specific data.

The idea for this use case is to focus on the value of domain specific models beyond mere data models. In other words, XForms has shown the value of domain specific data models, and now it is time to look at other kinds of models e.g. causal dependencies between subsystems, and task models for fault diagnosis. These models would be interpreted by Java classes that drive collaborative dialog. The input would be simple forms with output as text and graphics with links to animations showing how to perform specific procedures.

Mathematician's note pad

Remember Hari Seldon's note pad in Isaac Asimov's Foundation? You are holding a lightweight note pad. You use a stylus to quickly sketch mathematical expressions. As you do so, the hand-drawn squiggles are replaced by neatly typeset symbols. At the same time, the application builds up the underlying representation in MathML. With a quick flourish of the stylus, you can invoke the full power of the server's mathematical skills to assist you, step by step as you work through the problem that has been bugging you all night. This application uses spatial grammars to guide recognition of mathematical input, and a state machine to drive the interaction with the server.

The idea for this use case is to illustrate an interesting multimodal application with grammars for pen and speech, as well as an XML representation of an event driven state machine for the behavior. Spoken commands such as: simplify, factor, integrate, differentiate, solve, chart, etc.