Lately I’ve been working on a couple of exciting customer projects, involving HTML5-based UIs on embedded (TV) platforms. Due to the fabulous work some of my colleagues put into QtWebkit and QtDirectFB, and the ongoing work from MIPS Technologies on their Webkit JIT, the paint performance is very good, and Javascript is getting faster and faster.
However, while working on these projects, I’ve ran into a few issues that were not solvable by JIT-accelerated JavaScript or by hardware accelerated painting.
The main problem was that when you write a dynamic UI for a full application, the complexity starts to be difficult to manage. I started by dividing the app to a few HTML pages, but that of course created unacceptable delays when switching between the pages. So I looked online for dynamic web-app solutions. Those include the amazing Javascript libraries Dojo and JQuery. The problem with those is that they manage the complexity by selecting elements with custom-built CSS-like selectors (which are slow) and by modifying the HTML DOM tree in runtime (which is slower).
An alternate way to handle HTML performance is with the HTML5 canvas element (which is nicely accelerated) - the problem is that canvas API is currently too low level, and the widget libraries on top of it are not mature yet, definitely not in time for my customer projects.
So I asked myself: what components are really needed in order to make a dynamic UI?
Looking at the Qt Kinetic project, I need 4 main elements:
- Animations
- State Machine
- Graphic effects
- Declarative Syntax
Now HTML and CSS are already a declarative syntax, and CSS3 (supported in webkit) contains all the animations and graphic effects I currently need. The missing element, which is also my personal favorite, is the State Machine.
Why do I keep going on and on about state-machines?
If you think about it, both CSS, HTML, and Qt widgets are optimized to handle complexity in space. State machines, however, are optimized to handle complexity in time. An application can be dynamic only if it’s rich both in space and in time. That’s why to me state-machines are of outmost importance for dynamic UIs.
Enough philosophy, what’s this about?
Now I’ve had the pleasure to work with the SCXML standard in the last year, both by prototyping the Qt-SCXML library (which later helped with the evolution of the Qt State Machine Framework), and by joining the w3c group that defines the SCXML standard.
While working on my latest HTML-based project, I was thinking - what if I could define the flow of my web-app with an SCXML statechart? Wouldn’t that allow for managing the complexity of my web app, without incurring the performance overhead of modifying and re-modifying the DOM?
Introducing Statechartz
So I took the time to write an SCXML library for Javascript. I basically had to copy the SCXML algorithm to Javascript syntax, which took a while, but has already saved me hours of work by having the code of my web-apps cleaner and smaller.
The result can be found at the statechartz branch on the SCXML lab GIT repo., and the demo can be viewed online. Note that the library is currently only tested on webkit-based browsers (QtWebkit/Arora, Safari, Chrome), but porting it to other browsers shouldn’t be too difficult.
Looking a bit at the HTML code of the demo, you’ll notice the following:
<link rel="statechart" href="flow.scxml" />
This is the line that connects an HTML page with an SCXML statechart, defining the page’s flow. The state-machine would start running as soon as the page is loaded.
But hey! rel=”statechart” isn’t standard!
Well, I was surprised to learn that HTML5 allows you to add your own link types, which become legal if you register them in the WHATWG Wiki, which I did; so now <link rel=”statechart” href=”somefile.scxml” /> is completely standard 
Though using the link tag would cover most cases, it’s completely optional and the Statechartz library can be used more dynamically, either by loading an SCXML file whenever wanted:
var myStateMachine = Statechartz.loadScxml(someURL);
or by using a dynamic javascript syntax that looks like this:
var myStateMachine;
with (Statechartz) {
myStateMachine = build(
State("root",
Entry(function() { doSomething(); }),
Parallel("s1",Initial,
Exit(function()(doSomethingElse(); }),
Transition(Event("EVENT.QUIT"), Target("exit"))
),
Final("exit")
)
);
}
Another thing to look at, is the DOM:
<div id="screen_weather" class="screen">
<script type="text/javascript" src="..."></script>
</div>
<div id="screen_shopping" class="screen">
<iframe src="..." height="240" width="320" scrolling="no" style="overflow:hidden" frameborderwidth="0"></iframe>
</div>
<div id="screen_calc" class="screen">...</div>Even though screen_weather, screen_shopping and screen_calc exist in the DOM to begin with, only one of them is shown at a time. Also, there’s no particular Javascript code that shows and hides them. That’s done by a trick I added to the Statechartz library: the CSS class of the document BODY element changes when the state changes. So, if I’m in a state with ID “calc”, the body would have a CSS class “state_calc”, and now all it takes is adding this to the CSS (it’s a little different in the actual CSS, but I’m trying to make a point):
#screen_calc {
left: -350px;
-webkit-transition-duration: 600ms;
opacity: 0.5;
}
.state_calc #screen_calc {
left: 0px;
-webkit-transform: rotate(360deg) scale(1);
opacity: 1;
}
This code, like states and transitions in Qt 4.6, would animate the opacity and geometry of my calculator screen, and would create an animated transition when the calculator is entered to / exited from.
The Demo, and performance
The
- Dynamic UIs with states and transitions (switching between the 3 widgets)
- Managing UI complexity (e.g. the instructions popup)
- Business logic (the calculator ‘engine’)
Note that the demo specifically is too animation-heavy to run well on embedded platforms, and of course webkit could use some more optimizations. but the concepts (don’t mess with the DOM, use Statecharts to manage time) are the same, and are working on other webkit-based projects with a cvery nice performance gain.
The next step on the SCXML path
After this project, SCXML is now supported in 3 implementations: Qt C++ (with scc), QtScript (with QScxml), and web (with Statechartz). The next natural step is a statechart graphical tool (Creator plugin) that can help author SCXML files for all of those 3 implementations. Though it’s not on our roadmap, some of us (including the undersigned) were working on prototypes, and any help from the community would be appreciated!
Links
The code: http://qt.gitorious.org/qt-labs/scxml/trees/statechartz
The live demo (webkit browsers only): http://labs.trolltech.com/blogs/wp-content/uploads/2009/11/demo.html
Video capture of the live demo: http://labs.trolltech.com/blogs/wp-content/uploads/2009/11/statechartz-capture_0001.wmv
