Presented by W3CReport on the Workshop on HTML.next for Games
24 September 2011, Warsaw, Poland

Abstract

The Open Media Web project organized a very successful workshop on HTML.next for Games on Saturday 24 September 2011, co-located with the onGameStart conference in Warsaw, Poland. This page reports on the outcomes of the workshop, in particular the creation of the Games Community Group in W3C, decided during the workshop.

Table of Contents

Summary

HTML5 enables the development of games that run across devices, and are both easy to deploy and easy to maintain. Several features that are not yet part of the Web platform would be directly useful for games development, though. The workshop was the occasion to engage with the games community and start listing out features of interest. Workshop participants (including people from Bocoup, Google, Mozilla, RIM, Tecnalia, Wooga) were passionate about games and Web technologies. During the workshop, more than 20 features that would enable the development of better games using regular Web technologies were reviewed, refined and classified:

This report describes the main use case for each of the features of interest and includes a short gap analysis from today's (end of September 2011) Web platform. When known, the W3C working group and links to possible draft proposals are mentioned.

To ensure that games community needs are known and properly addressed in W3C, a Games Community Group was proposed and created at the end of the workshop. This group is dedicated to tracking the implementation of Open Web Platform features directly relevant for games development, and communicating how to build games on the Open Web Platform to the general public. A community group is a discussion forum open to anyone, without fees, particularly well suited to serve as coordination point for a particular community within W3C. Are you interested in the progress of the Web platform for games development? Join the Games Community Group!

The Games Community Group now needs to refine its scope, nominate one or more chairs, document goals and findings on its Wiki page and start reaching out to the groups identified below.

To engage more people in the activities of the group, a W3C Games Community Group Summit will be organized next to the New Game Conference in San Francisco, on Thursday 3 November from 10AM to 1PM, hosted by Zynga.

W3C is also pleased to announce a new W3C online training course on Game development in HTML5, taught by Michal Budzynski, organizer of the onGameStart conference, from 31 October to 27 November 2011.

Background

HTML5 enables the development of richer and ever more interactive applications. As far as gaming is concerned, the situation evolved from a world of hacks and plug-ins to a situation where one can envision the production of a multi-player first-person-shooter game using canvas, Web Workers, Web Sockets, on-going standardization efforts on audio, real-time communications and various other Web and device APIs.

However rich this new Web platform may become, games require precise control over:

Precise control is tough to achieve, either because the right API is missing, or because the specification does not address such precise needs, or simply because the specification is not clear enough to prevent interoperability issues. The Web as a games development platform is still in its infancy. The goal of this workshop was to meet and engage in discussions with developers of the games community as early as possible, taking advantage of onGameStart, the first HTML5 Game conference, and gather inputs for HTML.next, the next version of HTML, that would enable the development of fully immersive games using regular Web technologies.

List of Participants

28 people attended the workshop, representing 14 companies. Here is the list of participants with affiliations, sorted by last name:

Workshop participants waiting for the workshop to start

  1. Garret Alfert, 5 Apps
  2. F. Javier Almeida, Tecnalia
  3. Vlado Banda, Interactive1
  4. Michal Budzynski, Virtual Design
  5. Piotr Cichosz, Britenet
  6. Alexander Conceiro, Tecnalia
  7. François Daoust, W3C
  8. Jan Filipowski, GameBoxed.com
  9. Marie-Claire Forgue, W3C
  10. Laurent Hasson, RIM
  11. Rob Hawkes, Mozilla
  12. Eduardo Ibanez, Tecnalia
  13. Krystian Jarmicki
  14. Darius Kazemi, Bocoup
  15. Sebastian Kippe, 5 Apps
  16. Petteri Koivumaki
  17. Seth Ladd, Google
  18. Sinisa Robert Loncaric, Kaliani Studio
  19. Lukasz Mankowski, SII
  20. Andrzej Mazur, Britenet
  21. Raul Otaolea, Tecnalia
  22. Maciej Partyka
  23. Rudy Pons
  24. Joceba Rodriguez, Tecnalia
  25. Boaz Sender, Bocoup
  26. Jakub Siemiatkowski, Virtual Design
  27. Kamil Trebunia, Wooga
  28. Jill-Jenn Vie, student at ENS Cachan

The Web as a Games Development Platform

Discussions at the workshop covered a number of technical topics directly relevant to Web games development and standardization. Topics were sorted according to their current standardization status:

The list of topics is not meant to be exhaustive or authoritative. It is rather meant to serve as an initial set of features that the games community may want to track and support within W3C. The order of topics within each section is irrelevant.

François Daoust edits the list of topics under discussion

François edits and projects the list of topics raised by workshop participants

New features

Features in this category are missing from the Web platform as of September 2011. They are directly relevant to games development and do not have a home in W3C yet. The group that seems most relevant is mentioned whenever possible.

Some of these features may not need to be standardized in the end based on feedback, priorities, or e.g. because they cannot be enabled securely in a Web environment.

Mouse lock

Use case
On desktop computers, a user navigates in a first person shooter (FPS) game with the keyboard to walk around and the mouse to aim and shoot.
Gap analysis
In the context of a Web browser, the mouse cursor is always on and mouse events that get fired on a DOM element stop as soon as the mouse cursor leaves the element. In particular, from the point of view of a Web application, the mouse disappears as soon as it reaches the borders of the Web browser window. To use the mouse to control movements that should not be constrained in space, there should be a way to lock the mouse cursor to a DOM element.
The need for a mouse locking mechanism extends beyond games. For instance, this feature is also useful to lock the mouse wheel to zooming in/out on a map rendered on the screen.
Extending the mechanism to keyboards might also be of interest.
Possible solution
The Mouse Lock specification draft, proposed by Vincent Scheib (Google), describes a possible solution.
Related group
The Mouse Lock proposal is under discussion within the Web Events Working Group. Current plans are to revise the charter of the group to add this specification to the list of deliverables.

Hardware feature detection

Use case
Games need to balance high quality graphics and sounds against the need to maintain smooth animations for an acceptable gameplay. This tradeoff requires adjusting settings based on the performances of the machine (enabling/disabling antialiasing, switching to higher/lower texture quality, increasing/reducing the depth of the horizon in 3D space rendering, etc.). These settings can be switched on/off manually through a settings page. However, most users do not understand performance implications of switching a particular setting on/off. More importantly, for most users, games should just work. As much as possible, settings adjustments should thus be done on an automated basis.
Gap analysis
A Web application cannot easily and automatically tell the performances of the CPU/GPU of the machine it runs on. It cannot access the hardware capabilities of the graphics card either. Such information should be exposed to Web applications so that they can take decisions on behalf of the user.
Possible solution
At a minimum, a hardware performance score should be exposed so that Web applications can take decisions on settings. The score would be representative of the CPU and GPU performances and features. Such scores already exist, e.g. 3DMark.
Exposing a score (or more) creates a privacy issue though since it eases fingerprinting: a Web site would be able to generate a unique client identifier partly based on the information exposed by the mark. The Do-not-track HTTP header currently discussed in the Tracking Protection Working Group could perhaps be extended to protect users against such possibilities. More discussion is required.
Related group
Such functionality seems to fall in scope of the Web Performances Working Group.

Keyboard Capture

Use case
In a shoot 'em up game, the user may press keys continuously to move the ship around and fire weapons non stop. During each execution of the core loop of the game, the game engine needs to track which keys are pressed to perform the right action.
Gap analysis
Key presses are event-based. Continuous key press are not reported by events, and games thus need to maintain a mapping of keys currently pressed. For efficiency and convenience, the current set of pressed keys should rather be exposed as a property of the window object so that game engines can poll the status of the keyboard.
Possible solution
No identified proposal as of today.
Related group
The HTML Working Group would provide the appropriate forum for this feature.

Joystick API

Use case
All game consoles ship with a game controller that features buttons, one or more digital sticks and/or one or more analog sticks.
In a flight simulator game, an analog stick gives users a precise control over the movements of the aircraft ailerons and elevators, thus replicating the behavior of real aircraft commands.
In a first person shooter game, the analog stick is used to control the movements of the character's head and/or weapon.
Gap analysis
The DOM Level 3 Events Specification introduces the DOM_KEY_LOCATION_JOYSTICK constant that makes it possible to detect whether buttons and digital keys got pressed on a game controller. Analog sticks positions and movements cannot be adequately covered by existing events though.
Buttons in game controllers may also include pressure sensors that need to be exposed.
Possible solution
The Joystick draft specification proposed by Scott Graham (Google) proposes to switch from an event-based mechanism to a poll-based mechanism that is more natural for games and analog measures. Game engines typically iterate over a main loop (through calls to requestAnimationFrame) to update game components based on user actions, and thus based on user input readings. An event-based mechanism attached to an analog stick would either fire more events than the game engine may handle or fewer events which would negatively affect the precision of the control.
The JoystickAPI draft specification proposed by Ted Mielczarek, Jason Orendorff, David Humphrey, Kevin Gadd (Mozilla) takes a similar approach but also exposes joystick events.
The GamePad draft specification attempts to merge both proposals.
Related group
The Gamepad API proposal is under discussion within the Web Events Working Group. Current plans are to revise the charter of the group to add this specification to the list of deliverables.

Higher-level semantics for CSS animations

Use case
Menu elements and/or user notifications in games are very often animated with effects such as bouncing, wobbling, glowing/fading, or rotating. These effects catch the eye and highly contribute to improving the user experience. Such animations are but a first step in the right direction. Since the human eye is very good at spotting variations in movements, adding some level of randomness to animations helps ensure that the player does not feel that the game is too repetitive.
The need for easy-to-achieve high-end visual effects extends well beyond games to include any Web application that wants to improve the visual user experience.
Gap analysis
Game menus and user interfaces may be achieved using regular HTML and CSS to some extent. CSS specifications such as CSS Transitions, CSS Animations, CSS 2D Transforms and CSS 3D Transforms greatly enrich the toolset of effects game developers may rely on to define rich user interfaces.
Apparently simple effects such as the ones mentioned above (e.g. wobbling or glowing) cannot easily be achieved through CSS. More complex animations require scripting. Scripting is not a good long-term solution for purely visual effects which would better be handled by the Web browser for better efficiency.
Scripting is also required to introduce any kind of randomization parameter to animations.
Possible solution
The open source Alice.js micro JavaScript library (A Lightweight Independent CSS Engine), created by Laurent Hasson and co-developed with Jim Ing (RIM), explores the use of hardware-accelerated capabilities to generate high-end visual effects. The syntax used to describe these effects remains as close as possible to CSS on purpose. The library could typically serve to explore new effects and highlight potential candidates for new CSS properties.
A randomization parameter can be added to most effects in Alice.js to achieve natural randomness.
Related group
Discussions on these additions should be brought to the CSS Working Group.

Improvements to Web Workers: « Pass by reference »

Use case
To retain a responsive UI and ensure the gameplay is not negatively impacted by delays, game engines delegate I/O intensive tasks and heavy computing to dedicated threads, while the core game engine remains available to react to user inputs. Delegated tasks may involve passing around large chunks of memory between threads, for instance, an internal representation of the virtual world depicted in the game.
Gap analysis
Web Workers allow Web application authors to spawn background workers running scripts in parallel to the main Web applications. Data that flows between Web Workers and the main thread can only be passed by copy, though. This approach is inefficient for large amounts of memory.
Possible solution
Web Workers would need to be extended to allow parameters to be passed by reference. While the feature may seem easy to add, it opens the door to race conditions and other non trivial synchronization issues.
Related group
Discussions would need to be carried out in the Web Applications Working Group that develops the Web Workers specification.

Orientation lock

Use case
While it is good practice to provide a flexible user interface that adapts itself to the size and orientation of the screen, it is not always doable or even a good idea. For instance, video playing on many smartphones requires the user to switch to landscape mode. Many games have similar needs. On mobile devices, for action games for instance, it is easy to switch the orientation without actually meaning to, leading to a broken user experience.
Gap analysis
While it is easy to detect the orientation of a screen in CSS and scripts, there is no way to constrain the user to a given mode (portrait or landscape).
Possible solution
There are no known draft proposals as of today. The basic approach would be to define something like a window.lockOrientation() function to lock the viewport orientation and window.unlockOrientation() to unlock it. The user would probably need to be made aware that the orientation is locked since that is not an expected behavior in a Web browser.
Related group
The Web Applications Working Group would probably be the right group to discuss this functionality.

Access to screen pixel density

Use case
The visual quality of the user interface is crucial in games. Whenever possible, higher-quality graphics are used to improve the visual experience of a game on screens that have a higher pixel density.
Gap analysis
CSS defines CSS pixels. Viewports are typically set to match the definition of CSS pixels. Modern smartphones ship with higher density screens (sometimes coined as retina displays). From a Web application, the screen dimensions in CSS pixels is easy to tell. The actual pixel density is not, however.
To figure out the density of pixels of the screen, the User-Agent HTTP header may be used in conjunction with a device description repository. Support for the resolution property of the CSS Media Queries specification also allows to narrow down the resolution of the screen automatically with a script. Both methods are indirect and inefficient, though.
Possible solution
There are no known draft proposals as of today. Exposing the pixel density may pose a privacy concern since it exposes another identifier that could be used for fingerprinting purpose. To some extent, this identifier is already exposed through the methods mentioned above though.
Related group
Since the goal is to expose a device feature, the Device APIs Working Group seems to be the right target for further discussions.

Surround sound support

Use case
As 5.1 speaker arrangements become common in homes, games that feature a virtual world use surround sound effects to increase the realism of the action taking place in the scene.
Gap analysis
Developments within the Audio Working Group should help create a basis for audio synthesizing using JavaScript which is not restricted to any particular speaker setting in particular. Higher-level functionality to place a sound in a surround environment may be needed.
Possible solution
The Web Audio API draft proposal by Google provides the basis mentioned above. The Audio Data API by Mozilla is more basic but compatible with other uses of media streams, e.g. to enable real-time communications. The Web Audio API could be added on top of the Audio Data API. Further higher-level functionalities might help generate surround sounds in real-time from JavaScript.
Related group
Such discussions are in scope of the Audio Working Group.

High performance timers

Use case
In platform games, accurate movements are important to ensure a good gameplay. Required accuracy typically falls below the millisecond range.
Gap analysis
Events triggered in reaction to user input feature timeStamp property, used to specify the time at which the even was created in milliseconds relative to 1970-01-01T00:00:00Z. There is no way to attach a more precise high-resolution timer to an event.
Possible solution
There are no known draft proposals as of today.
Related group
The Web Performances Working Group would probably provide the right forum for further discussions.

Extensions to CSS

Use case
Games interfaces are highly sensitive to the screen dimensions and resolution. Games often adapt sizes of the various visual game assets and adjust CSS property values to take advantage of each pixel available. Object oriented techniques also greatly help reuse, ensuring the resulting CSS remains as minimal as possible.
Gap analysis
CSS does not allow to declare constants, parameters, mixins, or simple inheritance.
Possible solution
The LESS JavaScript library extends CSS with dynamic behavior such as variables, mixins, operations, functions and nested rules. A good solution needs to retain the simplicity of CSS.
Related group
Discussions on these additions should be brought to the CSS Working Group.

On-going features

Features in this category are missing from the Web platform as of September 2011 but are being addressed by a W3C Working Group. The games community should keep an eye on progress in these fields, in particular because solutions proposed by these working groups may not address all of the requirements identified for games development.

Accurate sound triggering

Use case
In a shoot 'em up video game, enemies explode when they are hit by a weapon. The explosion sound needs to be synchronized with the animation on the screen. Multiple explosions may occur at once.
Gap analysis
The <audio> element in HTML5 was introduced to play audio. It is a good step in the right direction but the initial use cases did not include multi-channel low-latency and high-accuracy audio.
Working group
The Audio Working Group was created to address such use cases.
Draft proposals
The following proposals serve as starting points for the group:

Asset loading and smart caching solution

Use case
A racing game contains megabytes of textures and other assets to render a realistic 3D world. On the Web, downloading megabytes of data is not easily doable, e.g. if the user is on a 3G connection. To ensure that the game starts to run as fast as possible, such game assets should be stored locally and available offline.
Permissions required by the game also need to be declared in some format.
Finally, when many assets need to be retrieved, the game needs to know when it can move offline (i.e. when all the assets are locally available)
Gap analysis
The File System API is a step in the right direction. The HTML5 application cache mechanism enables offline Web applications as well. W3C widgets provide a useful packaging format. Fragmentation remains in this space.
Working group
The HTML Working Group and the Web Applications Working Group are discussing next steps. The W3C Workshop on the Future of Off-line Web Applications will discuss and identify a clear path forward for innovation in the Open Web Platform related to offline Web application invocation and use.

Real-time peer-to-peer communications

Use case
A multiplayer game requires exchanges of messages among peers, in particular to exchange players coordinates and actions.
In cooperative mode, players in a first person shooter game set up an audio communication channel to synchronize their actions.
Gap analysis
The WebSocket API (and protocol) enable bi-directional communication between a client and a server. The server needs to centralize all messages sent to players in a multiplayer game. This is suboptimal as the number of players increases. Direct peer-to-peer connections would reduce latency and improve scalability.
Working group
The Web Real-Time Communications Working Group develops the WebRTC API while the RTCWEB group in IETF discusses the underlying protocols. The solution will enable audio/video calls as well as data exchanges between peers.

Fullscreen API

Use case
Most games run fullscreen to take advantage of the space available and to ensure players are not distracted by other elements on the screen that would not match the game's color scheme and ambiance.
Gap analysis
While most Web browsers allow the user to switch to fullscreen mode (typically through the F11 key on desktop browsers), a Web application cannot automatically switch to fullscreen by itself for obvious security reasons.
Working group
The feature is already highlighted in the HTML.next input page of the HTML Working Group.

Features that require more discussion

Features in this category may or may not be seen as directly relevant features or high priority features for games. More discussions are needed to refine the scope of the desired functionality.

Screenshot API

Use case
At the end of a race, a car simulation game proposes to replay the heroic moments of the race. The user can stop the replay at anytime to take a screenshot and share it with other players or friends.
To ensure that players are not cheating e.g. using additional script libraries that could boost the performances of their character in a game, a multiplayer game may take and upload regular screenshots to analyse the user's screen on the server and detect cheaters.
Gap analysis
A Web application cannot take a screenshot of a browser viewport.
Possible group
The feature is already highlighted in the HTML.next input page of the HTML Working Group. The feature would obviously trigger privacy issues that need to be addressed.

Efficient JSON compression mechanism

Use case
A multiplayer game needs to send regular messages between players to synchronize players' views. These messages often take the form of simple JSON structures. Keeping the messages terse greatly enhances the latency required to keep the game in sync.
Gap analysis
While content may be zipped upon transfer, typical compression/decompression protocols on the Web (gzip, deflate) are not efficient for small low-latency messages. It should be possible to define more efficient compression techniques for such use cases, for instance building on the Efficient XML Interchange (EXI) Format for XML-based documents.
Better quantification of the actual need and envisioned gain would be a good idea to assess whether such an efficient compression mechanism would be useful in the context of games. This may also be a transfer protocol issue, not directly in scope for W3C.

Advanced canvas rendering capabilities

Use case
Sprite animations in a shoot 'em up game involves re-drawing parts of the underlying <canvas> element in a short amount of time.
Gap analysis
Available canvas drawing methods are not highly efficient for the animation of sprites. A few JavaScript libraries make use of the dirty rectangle rendering technique, whereby only the portions of the canvas that have changed are rendered on a new rendering pass. Native support for this technique could improve efficiency.
It is unclear at this stage what functionality needs to be added. The efficiency it would bring also needs to be quantified.

Other features

Other features were raised during the workshop and discarded, either because they are not in scope for W3C or because they simply do not seem worth standardizing.

Monetization

Selling Web applications on the Web is tough. Ads may not be particularly well suited in games and/or on mobile devices either. Workshop participants believe that new business models will emerge as Web games get developed. While micro-payments might be a topic of interest for standardization, the question of monetization is seen as largely out of scope for W3C.

Collision detection

There are roughly as many collision detection mechanisms as there are games that need to detect collisions between objects. There are no clear solution that would work in most cases and could be included in a standard.

Vector support in JavaScript

Computations in 3D worlds require manipulating vectors. Adding support for matrices operations in JavaScript would ease developments. This is a language feature though, in scope for ECMA.

WebGL APIs

Similarly, new APIs to WebGL could ease games development and improve rendering quality, such as enabling occlusion queries (to tell which objects are behind or on top of others), developing a texture compression format, or enabling WebCL to run computation on the Graphics Processing Unit (GPU). Such considerations are in scope for the Khronos Group.

Main speakers points

Several workshop participants presented their thoughts during the workshop. This section presents a summary of points raised during these talks.

François Daoust (W3C)

See Workshop on HTML-next for Gaming set of slides.

After an introduction on standardization at W3C and ways for the games community to engage in the development of the Open Web Platform, e.g. through the creation of a Community Group dedicated to games, François highlighted the list of groups in W3C that are of particular interest from a games perspective:

Other groups, not listed here, may of course be of interest for games developers depending on the game under consideration. Groups home pages link to the current status of the deliverables produced by the group as well as to the list of participants, on-going issues and other news.

Seth Ladd (Google)

Seth reviewed priority topics for games development. The topics were further discussed among participants, please check the above classification for details. Topics highlighted include:

  • Mouse Lock API
  • Keyboard capture to enable polling for keys pressed
  • Joystick API, event-based vs. poll-based to read analog inputs
  • Occlusion queries and texture compression format for WebGL
  • Web application packaging, ways to store huge number of assets locally
  • Monetization is a key topic, although out of scope for W3C. Business models will emerge
  • Audio support
  • Fullscreen API
  • Surround sound support
  • « Pass by reference » in Web Workers
  • Vector support in JavaScript

Darius Kazemi (Bocoup)

See What game devs need from HTML5 set of slides.

The number one problem for Web games development is accurate sound (re)triggering. The Web Audio API, under discussion within the W3C Web Audio Working Group, is close to perfect for games developers. Main need is to play a sound right now, synchronized with an animation. Darius cited the FMOD library as a good set of functionalities that covers 99% of games developers needs.

Other new features needed include Mouse Lock API, asset loading techniques (Fieldrunners has about 150MB of content, of which 30MB need to be loaded at a time) and feature detection for auto-configuration purpose.

There are a number of features and areas that are already good. For instance, networking issues are not restricted to Web technologies. They are more a technique problem than a technology problem. Web game developers could learn a lot from traditional game developers in that area. Web Workers are also great for games development. Finally, code protection is a false problem. Sure enough, obfuscation techniques greatly help, but there is no way to protect data in the end. Business models will evolve to cope with it.

Laurent Hasson (RIM)

Laurent exchanges with Rob during the workshop

Laurent (background, left) exchanges with Rob (foreground, right) on screen resolutions

While screen resolution is easy to extract, there is no way to tell the pixel density of a given screen, an increasing problem on mobile devices.

Another area for discussion is how to map keys used to control games on devices that do not have physical keyboard. It is impossible to tell the keyboard layout from a Web page, and there may not be any keyboard in the end. We could think of an API to register the keys used within a game so that the Web browser may propose to arrange these keys on a touch screen for instance. This would perhaps be better solved with best practices to handle keyboard events.

Laurent enquired about the status of the Web on game consoles, which is not yet aligned with the Web on desktop browsers. It is important that the Web experience moves to game consoles for the convergence to succeed.

Different new CSS properties were proposed (randomization parameters, higher-level semantics). Check above classification of topics for details.

Rob Hawkes (Mozilla)

Rob mentioned Project Paladin in Mozilla, set to create the best gaming technology available for the open Web. The project includes Mouse Lock and Joystick APIs. Another project named Electrolysis aims at separating processes that display the browser UI, Web content and plugins.

Mozilla Research is heavily looking at games from various angles, including monetization and Web application packaging. Documentation is updated all the way. Contributions are welcome!

Acknowledgements

Marie-Claire and François would like to thank Michal Budzynski, organizer of the onGameStart conference, for helping out with the organization of the workshop.

omweb logoThe research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n°248687 - Open Media Web (OMWeb)