Setting the scope for light-weight Web-based applications

Definition

The light-weight, Web-based applications (“webapps”) of this essay are small, platform-independent programs that are downloaded on demand and execute inside a client program, such as a browser. They are thus like Java applets, but more "script-like" than "program-like" and therefore easier to write in many cases (though harder in others). They have a clearly separated user interface, that allows webapps to be easily adapted to different devices.

A trivial example is a Celsius-Fahrenheit converter: it runs inside your browser window and contains some code that calculates a number for every number you give it. Its UI might be two sliders, or two text input fields, or a text field and two buttons. You stop it by pressing the browser's back-button or maybe the program itself has a stop button.

Webapps are to be distinguished from Web documents, which are static, except for hyperlinks. (Note that a hypertext may still include certain visual effects, depending on the style sheet that is applied, such as tooltips, pop-ups and collapsing/expanding text and even animations.) Webapps don't have a document-like interface, but a program-like interface (a GUI/WIMP in the visual case).

Webapps are also different from Web-based forms, whose UI is still primarily that of a documents and whose logic is executed on a server (apart from superficial, UI-related logic, such as unchecking one radio button when another one is clicked). A webapp could be made to visually mimick a form and could communicate with a server, but its purpose is to do computations on the client-side and it lacks the typographical primitives of style sheets such as CSS and XSL

The difference between Java applets and webapps is the type of applications and type of programmers they target. Java is a general-purpose 3rd generation programming language. It has objects, modules and strong typing, which make it suitable for complex applications, but require a relatively experienced programmer. Webapps are for simple applications, for which you would use script-like languages (shell scripts, batch files, Perl, or maybe Basic) if the code had been for a single platform. The programming language, though not yet known at this point, will probably be interpreted and have implicit typing. There will probably be other differences in the underlying technology.

In summary, the characteristics of a webapp are the following:

• A webapp is an application that is downloaded on demand, not installed. Unlike a plug-in or a stand-alone application, a webapp disappears after use, just like a Web page. (Although it may of course be cached by the browser or other UA that fetched it.)
• Webapps execute on a platform provided by a UA (User Agent) on the client machine. The UA is something like a Web browser. Indeed, it may be an actual Web browser. Thus, the webapps can be executed on all machines for which a webapps UA is available, independent of the OS or underlying machine architecture.
• Webapps are “light-weight” in the sense that it is easy to create small applications with them. They are more like a “script” than a “program.” The overhead to create a webapp from scratch is minimal, just like in a shell script or batch file, and unlike in a typical programming language: a webapp probably doesn't need to set up libraries, declare variables and define types, or create header files; it is probably interpreted raher than compiled and linked.
• A webapp can itself be a front-end for an application on a server, such as a database (or in general: a Web service). In other words, it has access to the network. On the other hand, not all webapps will need a network. Some will contain everyhting they need once they are downloaded. For example, a solitaire game contains all the rules and data and only needs to interact with the user.
• A webapp has access to a number of high-level functions, that are typically also found in a browser: it can do a GET, POST or PUT on a URL (at least HTTP, other protocols may be available if the webapp UA supports them); it can invoke a browser to display resources it cannot display itself, such as HTML documents. The webapp UA also provides a number of traditional programming libraries to the webapp, such as string functions, trigonometric functions, regular expressions, etc.
• A webapp may not have access to the local file system (depending on how well the user or the UA trusts the webapp), but it may need some form of permanent storage. This could be a virtual file system (one for each webapp), provided by the webapp UA, or it could be a “cookie.”
• A webapp should be small (relative to the extent of its functionality), because it has to be downloaded. This could be achieved with compact code or high-level languages (i.e., many functions already provided by the webapp UA).
• Even though a webapp is platform independent (i.e., runs on every platform for which a webapp UA has been written), it looks and feels like a platform-native application: menus and buttons look and behave the same as other applications on the same platform. It will automatically use the standard dialog boxes of the platform and the standard keyboard shortcuts.
• A webapp can be a component in a compound document. E.g., it is expected that a webapp can be embedded in an HTML document with the help of HTML's OBJECT element.

The following may or may not be characteristics of webapps, depending on the technology chosen to implement them.

• A webapp could itself be a server for another webapp that is a client, thus allowing a distributed application to be created, e.g., a peer-to-peer system.
• A webapp may have an interchangable UI. If the webapp technology is designed as hinted further down in this essay, a webapp will have two parts: the UI-independent program logic and the UI. The latter can be replaced with a different one, that reacts to the same messages from the program. This should especially be easy if the UI part is written in a simple language, that is not a full programming language and that is amenable to WYSIWYG editing.
• A webapp may have a digital signature, showing that it came from where it says it came from.
• The webapp UA may be able to play sound and video. (I.e., it may provide UI objects for sound and video to the webapp, for use in its UI.)

Accessibility has to be designed into webapps from the start. Exactly how is not clear yet. The ability to provide several UIs for the same webapp allows specialized UIs for different devices, but few webapps will have UIs for all devices, just as few Web documents currently have style sheets for multiple devices. Some automatic adaptation is needed. Depending on the platform, the UI objects from which the UI is built may already have built-in accessibility APIs.

Example applications

Some examples of applications that could be implemented as webapps.

Celsius-Fahrenheit converter

A very simple application, that needs no network and whose program logic probably takes no more than two lines of code. The user can input a number and have the webapp convert it from Celsius to Fahrenheit or vice-versa.

The program logic simply gets a command and a number on its input and returns a number and a unit on its output.

The UI code is probably a dozen lines or so. The UI could be a slider and two buttons (“C to F” and “F to C”), or a text field and two buttons, or just two sliders (if one is moved, a command is sent to the application, which returns a number, which cause the other slider to move to that number).

In fact, the webapp could be delivered as one program with three UIs, allowing the user to select his preferred UI, through a menu in the webapp UA. (This is similar to how a document with alternative style sheets allows a user to select his preferred one.)

Digital clock

This webapp also doesn't require network access, but it shows the need for access to the clock of the computer. The UI is simply a text label and the application behind it sends new text to that label at regular intervals.

How the API to the clock is defined is not yet known. It could be that the application enters a loop that queries the current time, sleeps for while and then repeats the process. Or it could be that the clock is modeled as something that generates events, coming in on a stream/socket, just like input may arrive from the user in other programs.

Analog clock

This webapp is similar, but also shows the need for a graphics canvas. Whether this canvas can draw shapes (circles, lines, text, etc.) or only shows prerecorded images is yet to be defined.

Tax form simulator

This is a large application, that contains the rules for tax paying, either all rules or the rules as they apply in some domain, e.g., house owners. It is not a “real” tax form, in the sense that it can't be used to submit the filled-in form to the tax collectors, but it allows somebody to do “what-if” simulations and prepare all the calculations ahead of filling in a real form.

The UI will probably show several screens, one at a time, that each are reminescent of some fragment of a real tax form. Filling in a field or checking a box results in the application performing a recalculation of the amount to pay, which the UI then displays.

The webapp might have a function to print a form or a summary.

Tax form that also submits the form

This is a webapp that the ministry of finance might develop and offer to people who want to fill in their tax forms on-line. Like the previous example, it can be used for “what-if” simulations, but the goal of this webapp is that the user clicks “submit” on the last screen.

A requirement for such a webapp is that the user can somehow sign the form that he filled in. This might be achieved with an HTTP password that has been previously agreed upon or a GPG signature or some other means.

Design your kitchen

This is an application with a GUI that is a bit different from most other programs. The main part is a 3D rendering (possibly associated with a 2D map) of a kitchen, in which the user can place furniture with the mouse, change colors and texture of surfaces, add light points, etc.

This might be associated with a function to order the designed kitchen, or at least get a price quote on it.

Taking a snapshot (a virtual photograph) of the finalized kitchen and printing it (or storing it on disk) is also a possible function.

The 3D rendering is obviously not a predefined object of the webapp UA. It will have to made out of a graphics canvas on which the UI (in response to calculations by the application) places images or geometric shapes.

IRC client

This webapp looks like a typical IRC client: a text screen showing the conversation up til now, an input field for entering a line of text and some buttons (set the title, private conversation, etc.).

The interesting aspect of this webapp is, that it has an open socket to some server. I.e., it not just issues GET and PUT on URLs, but manages a protocol by itself (at least at the application level).

Similar applications: Usenet news reader and telnet client.

An internet radio tuner

This webapp could be implemented in two ways: it could decode the audio stream itself and send data to the UA's sound API, or it could launch a player program (VLC, realplayer, or similar) and control it via interprocess communication.

The former raises two questions: is the programming language of the webapp fast enough (since it is probably an interpreted language, like Javascript, or one that runs on a virtual machine) and does the webapp have the right to reach outside of its “sand box” and contact servers?

The second also raises two questions: how does the webapp know what media players are available and does it have the right to start programs on the client's machine?

The internet radio tuner, like the Celsius-Fahrenheit converter, could come with several UIs. One could be a big interface, high on graphics, with drawn buttons. One could be small and minimal. One might even be multi-modal, accepting user input simultaneously from the mouse, the keyboard and the microphone.

Weblog authoring tool

This is a dedicated editor that sends HTML fragments to a weblog server. The editor part may be a standard text widget, but the webapp has special functions for adding links and predefined HTML fragments.

Calculator

A very basic webapp: it shows the face of a traditional pocket calculator. The user presses buttons for digits and operations and after each press the application computes the number to display in the single-line output field.

Solitaire games

These are also fairly simple webapps, without need for network communication. Think of patience, four-in-a-row, 16-puzzle, etc.

These might have a need for persistent storage, so that the user can see his previous scores or maybe even continue a saved game.

Apple's Sherlock and Watson

The applications running on the Sherlock or Watson application platforms on the Mac are typically applications with a little bit of program logic, that derive most of their functionality from communicating with a server. Apple calls Sherlock a “Web services client.” They could almost be done as an HTML page with a form, except that the little bit of additional logic and the presentation as a GUI instead of a document make the applications much easier to use.

The applications typically have one or more screens, among which the user can select (tabbed interface). When they are started, they get data from a server. Depending on the user's action, they may then get more or more detailed data from the server. E.g., one interface shows movie listings. Initially, it shows a list of locations to choose from and the other parts of the interface are empty. When the user selects a location, detailed information about the films playing at that location are downloaded. The locations remain displayed and can be selected from at again at any moment, however, since this is a “direct manipulation” interface, not a series of forms.

The name

In this essay, I've used the name “webapp” as a convenient short name to refer to the technology. But other names are possible. Here are some suggestions:

• webapp, WebApp, Webapp, Web-app
• applet (but might be too much associated with Java)
• mobile apps, mobile code
• client-side apps
• thin apps

What it's not

Here are some of the technologies (partly still unsolved problems), that were mentioned as related to webapps and possibly solved by it, but which on closer inspection are actually not related.

XML namespaces

Webapps technology won't solve the problem of defining the meaning of documents with multiple XML namespaces.

<foo xmlns:a="http://example.org/a"
     xmlns:b="http://example.org/b">
  <a:bar>This "bar" is different from...</a:bar>
  <b:bar>... this "bar."</b:bar>
</foo>

Such namespaces can be used in XML formats that are created by combining other XML formats. They make it clear which parts come from which format. Namespaces are a syntactic construct, they have no meaning in themselves. Thus, any format that uses namespaces has to define what they mean in the context of that format.

Webapps are programs, and therefore have a specific meaning, but webapps technology is not by itself meant to attach meaning to namespaces. It is conceivable that the meaning of some XML format is defined by a mapping of elements to webapps (e.g., with a technology similar to Mozilla's XBL), but this would be a separate technology, which uses webapps, but is not itself part of webapps.

Plug-in API

Plug-ins are a way to extend a browser with support for new formats, new protocols or other new functionality (bookmarks, search, etc.) They implement a certain API, by which the browser can invoke them and the browser implements a complementary API, which a plug-in uses to communicate with the browser.

There are similarities between a plug-in and a webapp, but also differences: a webapp is transitory, a plug-in is installed permanently; a webapp is platform-independent, a plug-in is binary code for a specific platform; a webapp is untrusted code running in a “sand box”, a plug-in is an integral part of the browser. This means that the APIs often have similar functionality, but are not the same.

For example: both a plug-in and a webapp can ask the UA to retrieve a document, given a certain URL. But the webapp may restricted to URLs on the server it itself was downloaded from. Bot a plug-in and a webapp can open a dialog box, but the webapp calls a platform-independent function that is translated by the UA to the platform-native call, while the plug-in can call the native graphics library directly.

Compound documents

A compound document is a document made up of semi-independent parts, that are displayed together, but can also be used independently or as part of another compound document. The parts are referenced, rather than included. In Ted Nelson's words: a compound document is a document that has other documents transcluded.

As said above in the definitions section, a webapp can be a component in a compound document. E.g., it is expected that browsers will be able to run a webapp inside an OBJECT element in an HTML document.

A webapp itself is not a compound document, primarily because it is not a document. As part of its execution, it may open other documents or cause documents and other webapps to be displayed in a browser, and thus it may at some point almost look like a compound document to a user. But since an application cannot be easily inspected and analysed by a program (such as search engines or link checkers), it cannot take the place of real compound document formats, such as HTML.

Web-based forms

A webapp can obviously be made to look and act superficially like a form (with selection boxes, text fields, etc.), that undergoes some minimal processing before being sent to a server, but if the processing doesn't need client-side processing, it is better to use a declarative (typically HTML-based) form.

Webapps are less accessible, harder to maintain and harder to adapt to user preferences than HTML documents. It is harder for a user to do a “view source” or to find out how to submit the form without using a browser. Forms done as HTML documents probably also look better, since forms are text-based and CSS can provide the typography for them, while text in webapps is most likely in the form of label objects, with very primitive control over formatting.

There are various mark-up-based technologies for Web-based forms, that are easy to use, powerful, or even both. A general guideline should be to use one of those (without any Javascript), when possible, and only write a webapp when the mark-up-based solutions don't work:

• HTML 4 forms: the traditional, fairly simple, but limited forms support in HTML.
• XForms: powerful, but complex to author. Support for using the XForms + XHTML document format is being developed, with notably a draft of CSS features to select and style XForms elements.
• Opera's (Ian Hickson's) “Web forms” (a proposal at this time): a generalization of HTML 4's forms, incorporating some of the features of XForms, while keeping the simplicity of use of HTML.

Dangers

The reason to send an application (procedural knowledge) rather than a document (declarative knowledge) is normally because that solves the receiver's problem in the easiest or quickest way.

As a trivial example, assume somebody wants to know how much tax was included in the 100 € that he paid. One solution to that is just to tell him: "13.04 €." That solves his immediate problem, but next time he wants to know the tax for 150 € and he has to ask again. You can also give him a calculator that accepts a number and outputs the tax. That way, he will never have to ask again. But one day, he may want to know how much the price would be if the tax was 15 €. The calculator doesn't have that function. What he could do is try a number of inputs until he finds the one that returns 15. Or, you can give the person the formula, then he has full knowledge and can from now on calcalute forwards, backward or inside out, without any more help.

The first of these solutions, giving the answer in this concrete case, is the solution via a server-side application. This tells the user almost nothing beyond his particular question. The second, giving the application itself, is the webapps solution. It provides the user with procedural knowledge. He is now able to solve a whole class of similar problems by himself. The third solution is by means of declarative knowlege. The user now has the means to answer this and many other questions by himself. On the other hand, if this is the only time he needs the answer or this is the only kind of problem he ever solves, one of the two earlier solutions would have solved his problem quicker. It is only in the re-use that the declarative knowledge wins.

Another reason to send procedural knowledge (i.e., a webapp) might be that the provider of the information himself doesn't understand the full solution. He knows a method that gives a solution and passes it on, but he doesn't know why it works. This is like somebody who programs a videorecorder: he has figured out that it works if he presses button A, counts to 5, presses button B and then turns knob C. What each of those buttons do, he doesn't know, but at least he solved his problem...

There is, unfortunately, another reason information providers might send a webapp instead of a document: because they don't know they are withholding information that way. Which is a pity, because making a webapp represents considerable work, while it is unlikely to be useful as widely or for as long a time as the author hoped. A webapp will never be as device-independent as a document.

And, there is the case that information providers may want to hide knowledge, to bind customers to their service or because they consider the information their property. There isn't much that can be done about that.

When sending a program, there is a difference between sending a “binary” and sending the source. The former hides the embedded knowledge much more and makes reuse of the procedures that are contained in the code hard. Sending source code (which is interpreted or compiled on the client side) is preferrable: it allows the receiver to fix bugs, use (parts of) the code in his own application and thus also helps to teach and spread the webapps technology.

History

[ Active-X, Javascript, XBL/XUL]

Java (1995) is a 3rd generation programming language. It is object-oriented, strictly typed and borrows much of its syntax from C.

Java derives its portability from the existence of a virtual machine and a large set of libraries (file I/O, networking, GUIs, database access, etc.), some of which are bundled with browsers, so that the byte code of a Java program can run inside many browsers on many platforms. Java has an extensive security model, to allow control over what mobile code (Java applets) can do when they run on a user's machine.

Java is currently the most widely used example of mobile code. Java is suitable for large and complex programs. On the other hand, it does not separate program logic from UI and it is not very suitable for “quick and dirty” scripts.

Flash is best known as a format for animated graphics. The typical Flash “movie” on the Web shows some cartoon characters or geometric shapes moving and making sound, possibly reacting to mouse events. But Flash has a programming language (Javascript/ECMAscript), which is interpreted by the plug-in running in a browser, and it has several libraries, so it is possible to program simple applications in it, without any cartoon graphics.

Water (2001) is a language with similar scope as PHP or JSP. The logic is all on the server, the client just sees an HTML page with some form elements. Like in PHP and JSP, program logic and UI (i.e., HTML elements) are mixed, although programmers can use functions to better separate them.

CURL (1998) is a LISP-like language (but with curly braces, hence the name). Curl programs are compiled into native code on the client side. Curl builds interfaces with primitives like vertical and horizontal boxes (as in T_EX), buttons, text input fields, multi-line text paragraphs, lines and curves. Curl programs can be distributed: partly on the client and partly on the server. Simple Curl applications look a bit like PHP or JSP, in the sense that the program logic and the UI layout are mixed. Larger programs use functions and objects. Strong typing is supported, but not required.

Konfabulator (2003) is a program that executes applets (called "widgets"), that are written in Javascript and which have a GUI defined in an XML-based format. Konfabulator provides a set of APIs, that allow the widgets to access system resources, such as the clock, the file system, the battery status, an HTTP handler, etc. Many people have contributed widgets and a few hundred are available for download. Note that widgets are downloaded and installed, like plug-ins, not downloaded on demand, like webapps.

Gist (1990) is a language for rapid prototyping of GUIs, that I developed and implemented for my PhD, between 1988 and 1993. It is designed to allow writing GUI front-ends for command-line programs, such as the Unix shell, or any programs that accepts text on standard input and outputs other text on standard output.

Gist is interpreted, object-oriented (the objects are GUI widgets) and is a pattern-action style language. It has no loops, but supports recursion. The patterns can either be events (mouse clicks, key presses, etc.) or regular expressions that match outputs from the back-end program. The actions either set properties of the object or send messages (commands) to other objects or to the back-end application.

Gist doesn't have a predefined set of GUI widgets, but is linked to a platform-dependent set of widgets. (The implementation under X worked with XToolkit based widgets, such as the Motif set.) The idea to separate the program logic and the GUI logic and use a different language for each comes from Gist.

[Opera's initiative]

Syntax and implementation

I think the ideal technology would consist of two languages:

1. one language that describes the UI logic, and
2. one that describes the program logic and handles network communication.

A packaging format is also needed, since webapps, like most programs, will typically consist of several files.

Window main        # Window object called "main"
  title "Main"     # Initial value of attribute
  width 600px      # Initial value of attribute
  /message (.*)/:  # Pattern of a received message
      status $1.   # Action: send message to status
  /quit/:
      self halt.
Label status
  background #CCCCCC
  /.*/:
      self.value $1.