Testing

Current approach

We're currently planning to re-use the JavaScript Test Harness used by the HTML and WebApps Working Group.

We intend to apply the Test Methodology used for the Widgets spec for facilitate the development of our specs. We note that the methodology was developed for a format-based spec, and may not apply immediately to our API work.

Our tests will follow the W3C Test Suite licensing scheme.

Notes on testing framework used by Phonegap

Anssi presented the following points during TPAC2010Agenda:

• DAP and Open Source Testing Tools

• PhoneGap
  • A development framework for building cross-platform mobile web apps
  • All major mobile platforms supported
  • Implements some DAP APIs, releases early and often
  • Not a testing tool, uses QUnit (test suite) and Mobile Spec (unit tests) for testing

• QUnit
  • #1 popular JS test suite, used by e.g. jQuery and PhoneGap
  • A test runner for performing unit tests w/ a jQuery-like API
  • Capable of testing any generic JavaScript code, also asynchronous
  • http://docs.jquery.com/Qunit
  • http://github.com/jquery/qunit

• Mobile Spec
  • A set of QUnit-compatible unit tests for PhoneGap
  • Some DAP API unit tests available
  • Contacts API and Contacts Writer API: http://github.com/phonegap/mobile-spec/blob/master/tests/contacts.tests.js
  • System Information API: http://github.com/phonegap/mobile-spec/blob/master/tests/system.tests.js
  • http://github.com/phonegap/mobile-spec

• Why QUnit and Mobile Spec?
  • Robust, tried, tested and permissively licensed open source projects
  • Well-behaving on all major mobile platforms
  • Unit tests for some DAP APIs ready-made
  • Makes writing tests easy and fun, good fit for test-driven development
  • Re-use and contribute to existing projects

• Why Not?
  • Not optimal for functional testing (user interaction), e.g. file picker interactions
  • Potential license incompatibility w/ W3C? (PhoneGap: MIT, QUnit: MIT+GPL, Mobile Spec: MIT)
  • Not Invented Here, not aligned with other W3C testing efforts

• Misc Notes
  • Testing interfaces w/o JS binding is problematic ([NoInterfaceObject])
  • For functional testing, add Selenium to the mix: http://seleniumhq.org/