A few months ago, we looked at how well existing performance test suites worked on mobile browsers: they generally worked poorly (if at all), and in many occasions, the limiting factor was the memory required to run these tests.

As a follow-up, we have developed a simple test (source code) that allows to determine the maximum amount of memory a browser can dispose of.

The test is interactive and with every successful pass, it prompts the user to increase the size of data put in memory until an upper limit of 100 MB JSON's . If the browser fails to fetch a given amount of data, the user can pick a lower target.

Depending on how the device is managing out-of-memory errors, it may be more or less difficult to find that upper limit, but overall, it should help get a fairly good idea of the memory limitations of a given browser/device pair.

The Mobile Test working group welcome your comments. Any comments and/or suggestions can be sent to public mailing list or on this blog.

Whilst the W3C Widget Packaging and Configuration specification goes through the W3C process, the Mobile Web Test Suites Working Group have been investing some time and effort into a Widget Testing Framework (WTF).

There is currently a debug instance running at http://wtf.webvm.net. Though soon the project will be properly setup with sources for you to poke at and an opensource license to go with it.

So how does it work? Imagine you have a "Widget runtime" (UA), some software that allows you to run W3C widgets on your computer. You aim the UA to a test, for example: http://wtf.webvm.net/view//basic/true

Step 1

Clicking install should serve you a widget with the correct mimetype application/widget.

Step 2

This is the security prompt phase. You are likely to see this if your widget is not signed.

Step 3

The widget runs and if everything goes well you should see:

Behind the scenes, an AJAX request should be fired off that reports this particular test passed to a REPORTURI which collects this data.

Step 4

In Opera, you'll be asked whether you want to keep the widget after an initial run.

Remove the widget once testing it.

Want to know more?

A more detailed WTF wiki page might help you.

The missions of the Mobile Web Test Suites Working Group is to help ensuring that Web technologies work reliably on mobile devices, and we do that by creating tests (such as the Web Compatibility Test for Mobile Browsers) that help detect bugs and inconsistencies in mobile browsers, and provide an incentive for browsers vendors to compete on fixing these problems..

But another approach we have taken since the beginning of our work has been to ensure that tests that are developed by others can also be used to that same effect. Many W3C groups develop test suites as part of the development of their specifications, and more generally, individuals from the Web community at large also create tests to illustrate a specific problem they have, a solution they're proposing, or to explore new usage of a given technology.

Unfortunately, very often these tests end up to be problematic to use on mobile devices - the browsers performances tests we reviewed back in October illustrates the type of difficulties we have encountered again and again when trying to re-use tests on mobile devices.

Over time, we have collected the most frequent problems we have found in these tests, and the results of that collection has been published as a Working Group Note this Tuesday as the Guidelines for Writing Device Independent Tests.

It's a fairly simple and short document - so if you are developing tests for Web technologies, and want to participate in making the mobile web more robust, take a look at the guidelines and try to apply them to your tests!

Should you have any feedback on the guidelines, please let us on our public mailing list public-mwts@w3.org.