Mobile Web Test Suites Working Group Blog

Categories: Announcements (11) | Opinions (1) | Testing tools (2) | Web Compatibility Test (7) | Widgets testing (3) |

What's measured improves — 19 January 2009

What's measured improves

The mobile Web needs to improve. Tests like the Web compatibility tests for Mobiles (WCT) from the Mobile Web Test Suites Working Group (mwts) are designed to measure the 'Web compatibility' of the mobile browser. The Acid2 & Acid3 tests offer similar metrics that do not specifically aim to point out Web shortcomings on mobile devices, like the WCT does.

The WCT manifesto details tests that the MWTS group openly decided were important for mobile browsers to support. Briefly for version 1.42:

  1. Useful CSS function for smaller screens
  2. Good graphics support (transparency)
  3. Compression saves money on typical mobile data plans
  4. HTTPS (HTTP over SSL) has been a problem area on mobile browsers
  5. So too have been cookies
  6. Interesting to know if XHTML is supported for legacy applications
  7. Good to have Web 2.0 AJAX applications working
  8. Scalable Graphics are of special relevance to mobile screens
  9. CSS media queries is how author should adapt content if they require
  10. Inputmode tests for a form feature to enter number easily
  11. Dynamic scalable graphics is also great to have
  12. Test for typical problem area of international URLs
  13. Testing for events
  14. Canvas helps enables games and widget displays on mobiles
  15. Contenteditable allows in line editing, which might help content generation
  16. Powerful CSS3 selectors allow flexibility for style sheets

Some tests (inputmode) might not be relevant any more that could be removed. And likewise new tests can be added. We are open to suggestions!

Notice how the test indicates with a green square if the test passes. Unlike the Acid2 test, we hope to demonstrate from a mobile browser the quality of the browser, since it might pass some tests and fail some others. The tests have been ordered relatively in order of difficulty, though it is fair to say: the more green squares the better.

Unfortunately since many mobile browser Javascript engines are fairly limited we do not currently add a percentage counter like Acid3. However please count the squares and report them to measure how well your device does.

For example my current mobile phone, the:

Mozilla/5.0 (Linux; U; Android 1.0; en-gb; dream) AppleWebKit/525.10+ (KHTML, like Gecko) Version/3.0.4 Mobile Safari/523.12.2"

T-mobile G1 scores 9/16 on version 1.42 of the test. Do ensure you have the correct UA string, version of the test alongside your score please.

There are many more mobiles that can't even reach the WCT in order measure how well it performs on the Web! So just by visiting the WCT you are on the right path to improving the Web through measurement.

Share the score, UA string and test version and together lets improve the state of the mobile Web. Your business might depend on it.

Comments, Pingbacks:

Comment from: Paul Nelson [Visitor]
The HTTPS test points to an Opera site to get the results. How does one go about debugging and identifying why we receive the results that show up on our device?

Any results for tests should be easy for people to debug and correct issues.
PermalinkPermalink 2009-01-23 @ 21:50
Comment from: Wilhelm Joys Andersen [Visitor]
Yes, as W3C did not have any suitable server to use for the HTTPS test, this test fetches an image from one of Opera's test servers:

The certificate on the server is from Thawte, one of the most popular CAs on the Web today.

If the test passes, it will simply display a green square. If it fails, the square will be red.

To identify which tests are passing or failing, see the overview here:

It should also be possible to break the composite test into individual test cases if needed. Just study the source code; the parts belonging to any one test should be clearly labeled.

PermalinkPermalink 2009-02-03 @ 14:57

Contacts: Dominique Hazael-Massieux