A Graphic Design Perspective on Processing Delivery Context

Lynda Hardman and Jacco van Ossenbruggen
Centrum voor Wiskunde en Informatica (CWI), Amsterdam
{Lynda.Hardman, Jacco.van.Ossenbruggen}@cwi.nl


For delivery contexts to become fully integrated into the Web environment, design trade-offs, such as those made by human designers, have to be taken into account within the automated process. We sketch views of document processing from document engineering and graphics design perspectives and discuss the implications for processing delivery contexts.

Design Perspectives on Web Presentations

We compare two different perspectives of creating a presentation: document engineering and graphic design. The former assumes that the authoring process can be broken down into a sequence of sub-processes which are able to operate independently to generate the end result. The latter assumes a content-provider with a message to be communicated to a target audience, both of which the designer has to understand exactly before creating the appropriate mix of graphics and text to effectively communicate the client's message. Both perspectives are valid and need to be understood.

Document Engineering Perspective

From a document engineering perspective, it is important to separate content from style information. The underlying principle is that the essence of the message is contained in the (XML-structured) text and remains unchanged when style parameters, such as screen width or font size, are varied. This principle allows the creation of an infrastructure where the file containing the content, the XML file, can be created and maintained separately from a style file, such as a CSS stylesheet. The advantages of this approach are well-known within the Web community. These include the reuse of the same content in different contexts and the enforcement of consistent styles across different sets of content [1].

A presentation, however, involves more than applying an appropriate style to the selected content. A third, and essential, ingredient is the structure of the presentation. The simple separation of content and style as described above suffices only when the presentation structure is similar to the content structure in the underlying XML. If this is not the case, then a transformation step, such as enabled by XSLT, is needed to convert the content structure to the desired presentation structure. For example, the lexical order in a source XML document might need to be transformed to the order that is most appropriate in the text-flow of the target HTML presentation or into an XSL formatting object tree.

The document engineering process of creating Web presentations can be summarized in three steps:

  1. select or create the content (typically structured using XML);
  2. define a mapping from content to the presentation structure that defines, among other things, the most appropriate order (e.g. by using XSLT);
  3. (optionally) refine this presentation structure by applying preferred style parameters (e.g. by using CSS).
Essential in this approach is the assumption that the three steps can be carried out independently. Content can be entered into a database by a content-provider. This content can then be extracted from the database in the desired order by a server-side script written by a Web-site programmer. Finally, the preferred style parameters can be determined (server-side) by a graphic designer's and/or (client-side) by the end-user's stylesheet. For many content-driven Web sites, this assumption holds. Furthermore, the current Web infrastructure, with its large number of XML-related tools, is well equipped to support this process.

Graphic Design Perspective

From a graphic design perspective of creating a presentation, aspects of content, presentation structure and style depend on each other in ways that are generally ignored in the document engineering perspective. This is not to say that document engineering tools are not useful, but rather that the extra dependencies which make the task of good design so complex require more complex solutions. Since adapting presentations to any particular delivery context requires finding solutions within this design space, the three aspects of content, presentation structure and style need to be expressed and manipulated explicitly.

External forces that influence design originate directly from the different interests of the parties involved. To determine the requirements of an automated system, we use the following motivating example, based on a typical scenario with three main parties: a content-provider who wishes to effectively communicate a message to a user, aided by a skilled designer.

Examples of forces that originate from the content provider include the mission of the content provider's organization (e.g. making profit by selling books online), the limited availability of resources (e.g. the amount of time and money the organization is willing to spend on the design, the amount of disk space or bandwidth that is available at the server), and the content provider's preferences (e.g. the use of company colors in the Web forms).

Examples of forces that originate from the user include the user's needs (e.g. the desire to buy a book), the limitations imposed by the user's delivery context (e.g. the user could be driving a car, have a low bandwidth connection or have strict time constraints), and the user's personal preferences (e.g. user could prefer visual to textual information, dislike fast cuts in video material, prefer soft colors to primary colors).

Given a good understanding of the type of forces that play a role, it is the task of the designer to come up with a design that best matches the needs of the content provider and the user. In addition to the forces originating from the content-provider and the user, there are additional forces originating from the designer, whose resources are also limited and might also have personal preferences. Many of these forces could give rise to conflicts and will require the designer to make balanced trade-offs. For example, the designer might decide not to use the soft colors of the organization's company logo for users that need to fill in Web-forms while working in bad lighting conditions.

Automatic adaptation also needs to be able to deal with forces originating from content-provider and user, as well as with forces originating from the adaptation process itself (e.g. limited computing resources). We do not claim that an automated system could make such decisions as well as a professional designer. Future adaptation systems should, however, be able to make acceptable design decisions when dealing with these types of trade-offs. Their intelligence requires explicit knowledge about the design space dependencies and external constraints, combined with an adequate search strategy. These characteristics require that adaptation be more than the application of a simple mapping from source to destination format. Rather, it requires heuristic reasoning to find an optimal solution to balance the forces involved.

Beyond CSS and XSLT style and transformation rules

Despite the proven advantages of the document engineering approach, it has significant limitations. Specifically, in our own work on automatically adapting multimedia presentations to a variety of delivery contexts, generic XML tools proved to be inadequate [2]. Current tools are unable to deal with multimedia content for which it is not known a priori which transformation and stylesheet are suitable for displaying the content in a particular context. In online multimedia databases, for example, multimedia presentations can be generated from the media items returned by a database query. Since information about the media items such as quantity, type, size and size is not known in advance, template-based solutions cannot be used for determining a suitable presentation structure.

Adaptation engines need to be able to search in the design space and make trade-offs, rather than applying a number of functional transformations. This type of decision process is hard to define using the simple ``if selector matches then apply rule body'' type of current style and transformation rules.

Experiments with the Cuypers system [2] allowed us to analyze the adaptation process of multimedia presentations for which the quantity, type and size of the media items were not known until run-time. We found that for these applications, automatic adaptation also requires the ability to verify the presentations that result from applying a set of transformation rules. The system employs backtracking to search for alternative rules when the end result does not meet the constraints imposed by the available resources. For example, even when a specific rule is applied only for target screens with a certain width, that condition in itself will not guarantee that the presentation resulting from applying the rule to media content of unknown size will indeed meet the maximum width constraints. What is needed is a means of evaluating the actual width of the final presentation, and a means of trying alternative rules when the generated presentation does not meet the constraints.

While CSS and XSLT rules cannot be used to specify the required search strategies, this type of processing is vital for intelligent adaptive behavior on the Web. The Web thus requires more sophisticated ways of transforming the combined information provided by delivery contexts, metadata and the content into meaningful presentations.

In addition to improved transformation processes, we also need to develop better abstractions to reason about the ``soft'' constraints imposed by the preferences of the parties involved. This type of reasoning requires explicit knowledge of the dependencies among content, presentation structure and style. Taking these preferences and the associated dependencies into account will have a large impact on the perceived overall quality and design of automatic Web presentations. Currently, style rules work only on the basis of individual style properties. For example, one can specify the font type or color of a specific XML element. To what extent the application of these individual rules yield the desired overall result is hard to predict in advance, especially when dealing with more complex publishing systems that feature dynamic content, XSLT transformations, transcoding proxies and CSS stylesheets. After this process, the font and color of two XML elements positioned together in the final presentation might not go well together. Within the graphic design profession, style guidelines and checklists have been developed that can be used to avoid such design mistakes. It should be possible to build on this body of knowledge, and at least check the overall presentation against the most common design flaws. In addition to graphic design, similar checks could be developed for checking the design of the overall temporal flow of, and synchronization within, the presentation [3], and for checking the design of the navigation and interaction schemes of the final presentation.


Current Web technology addresses the problem of multiple delivery contexts through the use of CSS and XSLT stylesheets. These can be used for transforming presentation-independent XML documents to specific presentation formats, such as XHTML, SVG or SMIL. This is sufficient for dealing with a limited number of delivery contexts per stylesheet but is inadequate for adapting content to the plethora of delivery contexts for different devices, network resources and user groups. To solve this problem the Web requires intelligent transformation methods that are able to take into account a wide variety of delivery contexts to generate a presentation corresponding to a particular delivery context. While it is unrealistic to expect that even an intelligent stylesheet would be sufficiently powerful to cater for any given delivery context, our claim is that the current transformation technologies can be significantly improved in order to allow a substantial increase in flexibility.

The main problem with our vision is the large amount of high quality design and domain knowledge that it requires. We do not intend to replace human designers, but strive for providing applications with sufficient design knowledge when design decisions cannot be make by humans. It will require a large amount of human effort to make this knowledge explicit and it will require even more work to maintain it and keep it up to date. Given the problems most authors already have when they are forced to move from the ``what you see is what you get'' paradigm of desktop publishing to the ``structured document'' paradigm of XML-based Web publishing, this will not be an easy job. We do not, however, need to build it overnight: just as the current Web, we can create delivery context processing layers on top of the existing XML and RDF-based layers. Content-providers will start to use these new layers as soon as there are sufficiently large economic reasons (e.g. attracting more customers by making their site accessible from new mobile devices) or legal incentives (e.g. laws that require sites -- including multimedia content -- to be accessible for users with disabilities).


J. van Ossenbruggen, Processing Structured Hypermedia -- A Matter of Style.
PhD thesis, Vrije Universiteit, Amsterdam, The Netherlands, April 10, 2001.
Also available on http://www.cwi.nl/$\sim$jrvosse/thesis/.

J. van Ossenbruggen, J. Geurts, F. Cornelissen, L. Rutledge, and L. Hardman, ``Towards Second and Third Generation Web-Based Multimedia,'' in The Tenth International World Wide Web Conference, (Hong Kong), pp. 479-488, IW3C2, May 1-5, 2001.

F. Bes, M. Jourdan, and F. Khantache, ``A Generic Architecture for Automated Construction of Multimedia Presentations,'' in Proceedings of the International Conference on Multimedia Modeling 2001 (MMM01), (CWI, Amsterdam, The Netherlands), pp. 229-246, November 5-7, 2001.