Report on the Workshop on HTML.next for Games
24 September 2011, Warsaw, Poland

Mouse lock

Use case

On desktop computers, a user navigates in a first person shooter (FPS) game with the keyboard to walk around and the mouse to aim and shoot.

Gap analysis

In the context of a Web browser, the mouse cursor is always on and mouse events that get fired on a DOM element stop as soon as the mouse cursor leaves the element. In particular, from the point of view of a Web application, the mouse disappears as soon as it reaches the borders of the Web browser window. To use the mouse to control movements that should not be constrained in space, there should be a way to lock the mouse cursor to a DOM element.

The need for a mouse locking mechanism extends beyond games. For instance, this feature is also useful to lock the mouse wheel to zooming in/out on a map rendered on the screen.

Extending the mechanism to keyboards might also be of interest.

Possible solution

The Mouse Lock specification draft, proposed by Vincent Scheib (Google), describes a possible solution.

Related group

The Mouse Lock proposal is under discussion within the Web Events Working Group. Current plans are to revise the charter of the group to add this specification to the list of deliverables.

Hardware feature detection

Use case

Games need to balance high quality graphics and sounds against the need to maintain smooth animations for an acceptable gameplay. This tradeoff requires adjusting settings based on the performances of the machine (enabling/disabling antialiasing, switching to higher/lower texture quality, increasing/reducing the depth of the horizon in 3D space rendering, etc.). These settings can be switched on/off manually through a settings page. However, most users do not understand performance implications of switching a particular setting on/off. More importantly, for most users, games should just work. As much as possible, settings adjustments should thus be done on an automated basis.

Gap analysis

A Web application cannot easily and automatically tell the performances of the CPU/GPU of the machine it runs on. It cannot access the hardware capabilities of the graphics card either. Such information should be exposed to Web applications so that they can take decisions on behalf of the user.

Possible solution

At a minimum, a hardware performance score should be exposed so that Web applications can take decisions on settings. The score would be representative of the CPU and GPU performances and features. Such scores already exist, e.g. 3DMark.

Exposing a score (or more) creates a privacy issue though since it eases fingerprinting: a Web site would be able to generate a unique client identifier partly based on the information exposed by the mark. The Do-not-track HTTP header currently discussed in the Tracking Protection Working Group could perhaps be extended to protect users against such possibilities. More discussion is required.

Related group

Such functionality seems to fall in scope of the Web Performances Working Group.

Keyboard Capture

Use case

In a shoot 'em up game, the user may press keys continuously to move the ship around and fire weapons non stop. During each execution of the core loop of the game, the game engine needs to track which keys are pressed to perform the right action.

Gap analysis

Key presses are event-based. Continuous key press are not reported by events, and games thus need to maintain a mapping of keys currently pressed. For efficiency and convenience, the current set of pressed keys should rather be exposed as a property of the window object so that game engines can poll the status of the keyboard.

Possible solution

No identified proposal as of today.

Related group

The HTML Working Group would provide the appropriate forum for this feature.

Joystick API

Use case

All game consoles ship with a game controller that features buttons, one or more digital sticks and/or one or more analog sticks.

In a flight simulator game, an analog stick gives users a precise control over the movements of the aircraft ailerons and elevators, thus replicating the behavior of real aircraft commands.

In a first person shooter game, the analog stick is used to control the movements of the character's head and/or weapon.

Gap analysis

The DOM Level 3 Events Specification introduces the DOM_KEY_LOCATION_JOYSTICK constant that makes it possible to detect whether buttons and digital keys got pressed on a game controller. Analog sticks positions and movements cannot be adequately covered by existing events though.

Buttons in game controllers may also include pressure sensors that need to be exposed.

Possible solution

The Joystick draft specification proposed by Scott Graham (Google) proposes to switch from an event-based mechanism to a poll-based mechanism that is more natural for games and analog measures. Game engines typically iterate over a main loop (through calls to requestAnimationFrame) to update game components based on user actions, and thus based on user input readings. An event-based mechanism attached to an analog stick would either fire more events than the game engine may handle or fewer events which would negatively affect the precision of the control.

The JoystickAPI draft specification proposed by Ted Mielczarek, Jason Orendorff, David Humphrey, Kevin Gadd (Mozilla) takes a similar approach but also exposes joystick events.

The GamePad draft specification attempts to merge both proposals.

Related group

The Gamepad API proposal is under discussion within the Web Events Working Group. Current plans are to revise the charter of the group to add this specification to the list of deliverables.

Higher-level semantics for CSS animations

Use case

Menu elements and/or user notifications in games are very often animated with effects such as bouncing, wobbling, glowing/fading, or rotating. These effects catch the eye and highly contribute to improving the user experience. Such animations are but a first step in the right direction. Since the human eye is very good at spotting variations in movements, adding some level of randomness to animations helps ensure that the player does not feel that the game is too repetitive.

The need for easy-to-achieve high-end visual effects extends well beyond games to include any Web application that wants to improve the visual user experience.

Gap analysis

Game menus and user interfaces may be achieved using regular HTML and CSS to some extent. CSS specifications such as CSS Transitions, CSS Animations, CSS 2D Transforms and CSS 3D Transforms greatly enrich the toolset of effects game developers may rely on to define rich user interfaces.

Apparently simple effects such as the ones mentioned above (e.g. wobbling or glowing) cannot easily be achieved through CSS. More complex animations require scripting. Scripting is not a good long-term solution for purely visual effects which would better be handled by the Web browser for better efficiency.

Scripting is also required to introduce any kind of randomization parameter to animations.

Possible solution

The open source Alice.js micro JavaScript library (A Lightweight Independent CSS Engine), created by Laurent Hasson and co-developed with Jim Ing (RIM), explores the use of hardware-accelerated capabilities to generate high-end visual effects. The syntax used to describe these effects remains as close as possible to CSS on purpose. The library could typically serve to explore new effects and highlight potential candidates for new CSS properties.

A randomization parameter can be added to most effects in Alice.js to achieve natural randomness.

Related group

Discussions on these additions should be brought to the CSS Working Group.

Improvements to Web Workers: « Pass by reference »

Use case

To retain a responsive UI and ensure the gameplay is not negatively impacted by delays, game engines delegate I/O intensive tasks and heavy computing to dedicated threads, while the core game engine remains available to react to user inputs. Delegated tasks may involve passing around large chunks of memory between threads, for instance, an internal representation of the virtual world depicted in the game.

Gap analysis

Web Workers allow Web application authors to spawn background workers running scripts in parallel to the main Web applications. Data that flows between Web Workers and the main thread can only be passed by copy, though. This approach is inefficient for large amounts of memory.

Possible solution

Web Workers would need to be extended to allow parameters to be passed by reference. While the feature may seem easy to add, it opens the door to race conditions and other non trivial synchronization issues.

Related group

Discussions would need to be carried out in the Web Applications Working Group that develops the Web Workers specification.

Orientation lock

Use case

While it is good practice to provide a flexible user interface that adapts itself to the size and orientation of the screen, it is not always doable or even a good idea. For instance, video playing on many smartphones requires the user to switch to landscape mode. Many games have similar needs. On mobile devices, for action games for instance, it is easy to switch the orientation without actually meaning to, leading to a broken user experience.

Gap analysis

While it is easy to detect the orientation of a screen in CSS and scripts, there is no way to constrain the user to a given mode (portrait or landscape).

Possible solution

There are no known draft proposals as of today. The basic approach would be to define something like a window.lockOrientation() function to lock the viewport orientation and window.unlockOrientation() to unlock it. The user would probably need to be made aware that the orientation is locked since that is not an expected behavior in a Web browser.

Related group

The Web Applications Working Group would probably be the right group to discuss this functionality.

Access to screen pixel density

Use case

The visual quality of the user interface is crucial in games. Whenever possible, higher-quality graphics are used to improve the visual experience of a game on screens that have a higher pixel density.

Gap analysis

CSS defines CSS pixels. Viewports are typically set to match the definition of CSS pixels. Modern smartphones ship with higher density screens (sometimes coined as retina displays). From a Web application, the screen dimensions in CSS pixels is easy to tell. The actual pixel density is not, however.

To figure out the density of pixels of the screen, the User-Agent HTTP header may be used in conjunction with a device description repository. Support for the resolution property of the CSS Media Queries specification also allows to narrow down the resolution of the screen automatically with a script. Both methods are indirect and inefficient, though.

Possible solution

There are no known draft proposals as of today. Exposing the pixel density may pose a privacy concern since it exposes another identifier that could be used for fingerprinting purpose. To some extent, this identifier is already exposed through the methods mentioned above though.

Related group

Since the goal is to expose a device feature, the Device APIs Working Group seems to be the right target for further discussions.

Surround sound support

Use case

As 5.1 speaker arrangements become common in homes, games that feature a virtual world use surround sound effects to increase the realism of the action taking place in the scene.

Gap analysis

Developments within the Audio Working Group should help create a basis for audio synthesizing using JavaScript which is not restricted to any particular speaker setting in particular. Higher-level functionality to place a sound in a surround environment may be needed.

Possible solution

The Web Audio API draft proposal by Google provides the basis mentioned above. The Audio Data API by Mozilla is more basic but compatible with other uses of media streams, e.g. to enable real-time communications. The Web Audio API could be added on top of the Audio Data API. Further higher-level functionalities might help generate surround sounds in real-time from JavaScript.

Related group

Such discussions are in scope of the Audio Working Group.

High performance timers

Use case

In platform games, accurate movements are important to ensure a good gameplay. Required accuracy typically falls below the millisecond range.

Gap analysis

Events triggered in reaction to user input feature timeStamp property, used to specify the time at which the even was created in milliseconds relative to 1970-01-01T00:00:00Z. There is no way to attach a more precise high-resolution timer to an event.

Possible solution

There are no known draft proposals as of today.

Related group

The Web Performances Working Group would probably provide the right forum for further discussions.

Extensions to CSS

Use case

Games interfaces are highly sensitive to the screen dimensions and resolution. Games often adapt sizes of the various visual game assets and adjust CSS property values to take advantage of each pixel available. Object oriented techniques also greatly help reuse, ensuring the resulting CSS remains as minimal as possible.

Gap analysis

CSS does not allow to declare constants, parameters, mixins, or simple inheritance.

Possible solution

The LESS JavaScript library extends CSS with dynamic behavior such as variables, mixins, operations, functions and nested rules. A good solution needs to retain the simplicity of CSS.

Related group

Discussions on these additions should be brought to the CSS Working Group.

Accurate sound triggering

Use case

In a shoot 'em up video game, enemies explode when they are hit by a weapon. The explosion sound needs to be synchronized with the animation on the screen. Multiple explosions may occur at once.

Gap analysis

The <audio> element in HTML5 was introduced to play audio. It is a good step in the right direction but the initial use cases did not include multi-channel low-latency and high-accuracy audio.

Working group

The Audio Working Group was created to address such use cases.

Draft proposals

The following proposals serve as starting points for the group:

• Web Audio API
• Audio Data API

Asset loading and smart caching solution

Use case

A racing game contains megabytes of textures and other assets to render a realistic 3D world. On the Web, downloading megabytes of data is not easily doable, e.g. if the user is on a 3G connection. To ensure that the game starts to run as fast as possible, such game assets should be stored locally and available offline.

Permissions required by the game also need to be declared in some format.

Finally, when many assets need to be retrieved, the game needs to know when it can move offline (i.e. when all the assets are locally available)

Gap analysis

The File System API is a step in the right direction. The HTML5 application cache mechanism enables offline Web applications as well. W3C widgets provide a useful packaging format. Fragmentation remains in this space.

Working group

The HTML Working Group and the Web Applications Working Group are discussing next steps. The W3C Workshop on the Future of Off-line Web Applications will discuss and identify a clear path forward for innovation in the Open Web Platform related to offline Web application invocation and use.

Real-time peer-to-peer communications

Use case

A multiplayer game requires exchanges of messages among peers, in particular to exchange players coordinates and actions.

In cooperative mode, players in a first person shooter game set up an audio communication channel to synchronize their actions.

Gap analysis

The WebSocket API (and protocol) enable bi-directional communication between a client and a server. The server needs to centralize all messages sent to players in a multiplayer game. This is suboptimal as the number of players increases. Direct peer-to-peer connections would reduce latency and improve scalability.

Working group

The Web Real-Time Communications Working Group develops the WebRTC API while the RTCWEB group in IETF discusses the underlying protocols. The solution will enable audio/video calls as well as data exchanges between peers.

Fullscreen API

Use case

Most games run fullscreen to take advantage of the space available and to ensure players are not distracted by other elements on the screen that would not match the game's color scheme and ambiance.

Gap analysis

While most Web browsers allow the user to switch to fullscreen mode (typically through the F11 key on desktop browsers), a Web application cannot automatically switch to fullscreen by itself for obvious security reasons.

Working group

The feature is already highlighted in the HTML.next input page of the HTML Working Group.

Screenshot API

Use case

At the end of a race, a car simulation game proposes to replay the heroic moments of the race. The user can stop the replay at anytime to take a screenshot and share it with other players or friends.

To ensure that players are not cheating e.g. using additional script libraries that could boost the performances of their character in a game, a multiplayer game may take and upload regular screenshots to analyse the user's screen on the server and detect cheaters.

Gap analysis

A Web application cannot take a screenshot of a browser viewport.

Possible group

The feature is already highlighted in the HTML.next input page of the HTML Working Group. The feature would obviously trigger privacy issues that need to be addressed.

Efficient JSON compression mechanism

Use case

A multiplayer game needs to send regular messages between players to synchronize players' views. These messages often take the form of simple JSON structures. Keeping the messages terse greatly enhances the latency required to keep the game in sync.

Gap analysis

While content may be zipped upon transfer, typical compression/decompression protocols on the Web (gzip, deflate) are not efficient for small low-latency messages. It should be possible to define more efficient compression techniques for such use cases, for instance building on the Efficient XML Interchange (EXI) Format for XML-based documents.

Better quantification of the actual need and envisioned gain would be a good idea to assess whether such an efficient compression mechanism would be useful in the context of games. This may also be a transfer protocol issue, not directly in scope for W3C.

Advanced canvas rendering capabilities

Use case

Sprite animations in a shoot 'em up game involves re-drawing parts of the underlying <canvas> element in a short amount of time.

Gap analysis

Available canvas drawing methods are not highly efficient for the animation of sprites. A few JavaScript libraries make use of the dirty rectangle rendering technique, whereby only the portions of the canvas that have changed are rendered on a new rendering pass. Native support for this technique could improve efficiency.

It is unclear at this stage what functionality needs to be added. The efficiency it would bring also needs to be quantified.

Monetization

Selling Web applications on the Web is tough. Ads may not be particularly well suited in games and/or on mobile devices either. Workshop participants believe that new business models will emerge as Web games get developed. While micro-payments might be a topic of interest for standardization, the question of monetization is seen as largely out of scope for W3C.

Collision detection

There are roughly as many collision detection mechanisms as there are games that need to detect collisions between objects. There are no clear solution that would work in most cases and could be included in a standard.

Vector support in JavaScript

Computations in 3D worlds require manipulating vectors. Adding support for matrices operations in JavaScript would ease developments. This is a language feature though, in scope for ECMA.

WebGL APIs

Similarly, new APIs to WebGL could ease games development and improve rendering quality, such as enabling occlusion queries (to tell which objects are behind or on top of others), developing a texture compression format, or enabling WebCL to run computation on the Graphics Processing Unit (GPU). Such considerations are in scope for the Khronos Group.