Planet MozillaWorkshop day two

HTTP Workshop At 5pm we rounded off another fully featured day at the HTTP workshop. Here’s some of what we touched on today:

Moritz started the morning with an interesting presentation about experiments with running the exact same site and contents on h1 vs h2 over different kinds of networks, with different packet loss scenarios and with different ICWND set and more. Very interesting stuff. If he makes his presentation available at some point I’ll add a link to it.

I then got the honor to present the state of the TCP Tuning draft (which I’ve admittedly been neglecting a bit lately), the slides are here. I made it brief but I still got some feedback and in general this is a draft that people seem to agree is a good idea – keep sending me your feedback and help me improve it. I just need to pull myself together now and move it forward. I tried to be quick to leave over to…

Jana, who was back again to tell us about QUIC and the state of things in that area. His presentation apparently was a subset of slides he presented last week in the Berlin IETF. One interesting take-away for me, was that they’ve noticed that the amount of connections for which they detect UDP rate limiting on, has decreased with 2/3 during the last year!

Here’s my favorite image from his slide set. Apparently TCP/2 is not a name for QUIC that everybody appreciates! ;-)


While I think the topic of QUIC piqued the interest of most people in the room and there were a lot of questions, thoughts and ideas around the topic we still managed to get the lunch break pretty much in time and we could run off and have another lovely buffet lunch. There’s certainly no risk for us loosing weight during this event…

After lunch we got ourselves a series of Lightning talks presented for us. Seven short talks on various subjects that people had signed up to do

One of the lightning talks that stuck with me was what I would call the idea about an extended Happy Eyeballs approach that I’d like to call Even Happier Eyeballs: make the client TCP connect to all IPs in a DNS response and race them against each other and use the one that responds with a SYN-ACK first. There was interest expressed in the room to get this concept tested out for real in at least one browser.

We then fell over into the area of HTTP/3 ideas and what the people in the room think we should be working on for that. It turned out that the list of stuff we created last year at the workshop was still actually a pretty good list and while we could massage that a bit, it is still mostly the same as before.

Anne presented fetch and how browsers use HTTP. Perhaps a bit surprising that soon brought us over into the subject of trailers, how to support that and voilá, in the end we possibly even agreed that we should perhaps consider handling them somehow in browsers and even for javascript APIs… ( nah, curl/libcurl doesn’t have any particular support for trailers, but will of course get that if we’ll actually see things out there start to use it for real)

I think we deserved a few beers after this day! The final workshop day is tomorrow.

Planet MozillaUpdate from WebDriver WG meeting in July 2016

The W3C Browser Tools- and Testing Working Group met again 13-14 July 2016 in Redmond, WA to discuss the progress of the WebDriver specification. I will try to summarise the discussions, but if you’re interested in all the details the meetings have been meticulously scribed.

I wrote about the progress from our TPAC 2015 meeting previously, and we appear to have made good progress since then. The specification text is nearing completion, although it is missing a few important chapters: Some particularly obvious omissions are the complete lack of input handling, and a big, difficult void where advanced user actions are meant to be.


James has been hard at work drafting a proposal for action semantics, which we went over in great detail. I think it’s fair to say there had been conceptual agreement in the working group on what the actions were meant to accomplish, but that the details of how they were going to work were extremely scarce.

WebDriver tries to innovate on the actions as they appear in Selenium. Actions in Selenium were originally meant to provide a way to pipeline a sequence of interactions—such as pressing down a mouse button, moving the mouse, and releasing it—through a complex data structure to a single command endpoint. The idea was that this would help address some of the race conditions that are intrinsically part of the one-directional design of the protocol, and reduce latency which may be critical when interacting with a document.

Unfortunately the pipelining design to reduce the number of HTTP requests was never quite implemented in Selenium, and the API design suffered from over-specialisation of different types of input devices and actions. The specification attempts to rectify this by generalising the range of input device classes, and by associating the actions that can be performed with a certain class. This means we are moving away from a flat sequence of types, such as [{type: "mouseDown"}, {type: "mouseMove"}, {type: "mouseUp"}] to a model where each input device has its own “track”. This limits the actions you can perform with each device, which makes some conceptual sense because it would be impossible to i.e. type keys with a mouse or press a mouse button with a stylus/pen input device.

The side-effect of this design is that it allows for parallelisation of actions from one or more types of input devices. This is an important development, as it makes it possible to combine primitives for input methods such as touch: In reality, a device cannot determine whether two fingers are “associated” with the same hand. So instead of defining high-level actions such as pinch and flick, it gives you the right level of granularity to combine actions from two or more touch “finger” devices to synthesise more complex movements. We believe this is a good approach with the right level of granularity that doesn’t try to over-specify or shoehorn in primitives that might not make sense in a cross-browser automation setting.

I’m looking forward to seeing James’ work land in the specificaton text. I think probably some explanatory notes and examples are required to fully explain this concept for both implementors and users.

Input locality

A known limitation of Selenium that we are not proud of is that it does not have a good story for input with alternative keyboard layouts. We have explicitly phrased the specification in such a way that it doesn’t make it impossible to retrofit in support for multiple layouts in the future. But right now we want to finish the baseline of the specification before we try moving into this.

The current design ideas floating around are to have some way of setting a keyboard layout either through a command or a capability. This would allow / to generate key events for Shift and ? on an American layout, and Shift and 7 on Norwegian layout. The biggest reason this is hard is because we need to find the right key code conversion tables for what would happen when typing for example .

Untrusted SSL certificates

We had a big discussion on invalid, self-signed, and untrusted SSL certificates. The general agreement in the WG is that it would be good to have functionality to allow a WebDriver session to bypass the security checks associated with them, as WebDriver may be run in an environment where it is difficult or even impossible to instrument the browser/environment in such a way that they are accepted implicitly (e.g. by modifying the root store).

Different browser vendors raised questions over whether this would pass security review as implementing such a feature increases the attack surface in one of the most critical components in web browsers. A counterargument is that by the point your browser has WebDriver enabled, you probably have bigger things to worry about than the fact that untrusted certificates are implicitly accepted.

We also found that this is highly inconsistently implemented in Selenium. For the two drivers that support it, FirefoxDriver (written and maintained by Selenium) has an acceptSslCerts capability that takes a boolean to switch off security checks, and chromedriver (by Google) by contrast accepts all certificates by default. The remaining drivers have no support for it.

This leaves the working group free to decide on a new and consistent approach. One point of concern is that a boolean to disable all security checks seems like an overly coarse design. A suggested alternative is to provide a list of domains to disable the checks for, where wildcards can be expanded to cover every domain or every subdomain, so that i.e. ["*"] would be equivalent to setting acceptSslCerts to true in today’s Firefox implementation, but that ["*"] would only disable untrusted certificates on this domain.

Navigation and URLs

Because WebDriver taps into the browser’s navigation algorithm at a much later point than when a user interacts with the address bar, we decided that malformed URLs should consistently return an error. We have also changed the prose to no longer mislead users to think that navigating in effect means the same as using the address bar; the address bar is not a concept of the web platform.

There was a proposal from Mozilla to allow navigation to relative URLs, so that one could navigate to i.e. "/foo" to go to the path on the current domain, similar to how window.location = "/foo" works. This was unfortunately voted down. I feel it would be useful, even just for consistency, for the WebDriver navigation command to mirror the platform API, modulo security checks.

Desired vs. required capabilities

A big discussion during the meeting was around the continuing confusion around capabilities: Many feel they are an intermediary node concept that is best left undefined in the core specification text itself, because the specification explicitly does not define any qualities or expectations about local ends (clients bindings) or intermediary nodes (Selenium server or proxy that gives you a session).

There was however consensus around the fact that having a way to pick a browser configuration from some matrix was a good idea. The uncertainty, I think, comes largely from driver implementors who feel that once capabilities reach the driver there is very little that can be done about the sort of conflict resolution that required- and desired capabilities warrant.

For example, what does it mean to desire a profile and how do you know if the provided profile is valid? We were unable to reach any agreement on this and decided to punt the topic for our next meeting in Lisbon.

Test coverage

In order to push the specification to “Rec” (short for Recommendation) one must have at least two interoperable implemenations by two separate vendors. To determine that they are interoperable, one needs a test suite. I’ve written previously about the test harness I wrote for the Web Platform Tests that integrates WebDriver spec tests with wptrunner.

We have a few exhaustive tests for a couple of chapters, but I hope to continue this work this quarter.

Next meeting

The working group is meeting again for TPAC that this year is in Lisbon (how civilised!) in late September. I’m enormously looking forward to visiting there as I’ve never been.

We hope resolve the outstanding capabilities discussion and make final decisions on a few more minor outstanding issues then.


Planet WebKitNew <video> Policies for iOS

<video class="alignleft" loop="loop"><source src="/wp-content/uploads/forever.mp4"><source src="/wp-content/uploads/forever.webm"></video>

Since before your sun burned hot in space and before your race was born, Safari on iOS has required a user gesture to play media in a <video> or <audio> element. When Safari first supported <video> in iPhoneOS 3, media data loaded only when the user interacted with the page. But with the goal of returning more control over media playback to web developers, we relaxed this restriction in iOS 8: Safari began honoring the preload="metadata" attribute, allowing <video> and <audio> elements to load enough media data to determine that media’s size, duration, and available tracks. For Safari in iOS 10, we are further relaxing this user gesture requirement for silent <video> elements.


It turns out that people these days really like GIFs. But the GIF format turns out to be a very expensive way to encode animated images when compared to a modern video codec like H.264. We’ve found that GIFs can be up to twelve times as expensive in bandwidth and twice as expensive in energy use. It’s so expensive that many of the largest GIF providers have been moving away from GIFs and toward the <video> element. Since most of these GIFs started out their lives as video clips, were converted into animated GIFs, and were again converted back to video clips, you might say that the circle is complete.

But while this move does spare websites’ bandwidth costs as well as saving users’ batteries, it comes at a usability cost. On iOS 9, <video>s will only begin playing as a result of a user gesture. So pages which replace an <img> with a <video> will require a user gesture before displaying their animated content, and, on iPhone, the <video> will enter fullscreen when starting playback.

A note about the user gesture requirement: when we say that an action must have happened “as a result of a user gesture”, we mean that the JavaScript which resulted in the call to, for example, must have directly resulted from a handler for a touchend, click, doubleclick, or keydown event. So, button.addEventListener('click', () => {; }) would satisfy the user gesture requirement. video.addEventListener('canplaythrough', () => {; }) would not.

Similarly, web developers are doing some seriously amazing stuff by integrating <video> elements into the presentation of their pages. And these pages either don’t work at all on iOS due to the user gesture requirement, or the <video> elements completely obscure the page’s presentation by playing an animated background image in fullscreen.

WebKit’s New policies for video

Starting in iOS 10, WebKit relaxes its inline and autoplay policies to make these presentations possible, but still keeps in mind sites’ bandwidth and users’ batteries.

By default, WebKit will have the following policies:

  • <video autoplay> elements will now honor the autoplay attribute, for elements which meet the following conditions:
    • <video> elements will be allowed to autoplay without a user gesture if their source media contains no audio tracks.
    • <video muted> elements will also be allowed to autoplay without a user gesture.
    • If a <video> element gains an audio track or becomes un-muted without a user gesture, playback will pause.
    • <video autoplay> elements will only begin playing when visible on-screen such as when they are scrolled into the viewport, made visible through CSS, and inserted into the DOM.
    • <video autoplay> elements will pause if they become non-visible, such as by being scrolled out of the viewport.
  • <video> elements will now honor the play() method, for elements which meet the following conditions:
    • <video> elements will be allowed to play() without a user gesture if their source media contains no audio tracks, or if their muted property is set to true.
    • If a <video> element gains an audio track or becomes un-muted without a user gesture, playback will pause.
    • <video> elements will be allowed to play() when not visible on-screen or when out of the viewport.
    • will return a Promise, which will be rejected if any of these conditions are not met.
  • On iPhone, <video playsinline> elements will now be allowed to play inline, and will not automatically enter fullscreen mode when playback begins.
    <video> elements without playsinline attributes will continue to require fullscreen mode for playback on iPhone.
    When exiting fullscreen with a pinch gesture, <video> elements without playsinline will continue to play inline.

For clients of the WebKit framework on iOS, these policies can still be controlled through API, and clients who are using existing API to control these policies will see no change. For more fine-grained control over autoplay policies, see the new WKWebViewConfiguration property mediaTypesRequiringUserActionForPlayback. Safari on iOS 10 will use WebKit’s default policies.

A note about the playsinline attribute: this attribute has recently been added to the HTML specification, and WebKit has adopted this new attribute by unprefixing its legacy webkit-playsinline attribute. This legacy attribute has been supported since iPhoneOS 4.0, and accordance with our updated unprefixing policy, we’re pleased to have been able to unprefix webkit-playsinline. Unfortunately, this change did not make the cut-off for iOS 10 Developer Seed 2. If you would like to experiment with this new policy with iOS Developer Seed 2, the prefixed attribute will work, but we would encourage you to transition to the unprefixed attribute when support for it is available in a future seed.


So how would the humble web developer take advantage of these new policies? Suppose one had a blog post or article with many GIFs which one would prefer to serve as <video> elements instead. Here’s an example of a simple GIF replacement:

<video autoplay loop muted playsinline>
  <source src="image.mp4">
  <source src="image.webm" onerror="fallback(parentNode)">
  <img src="image.gif">
function fallback(video)
  var img = video.querySelector('img');
  if (img)
    video.parentNode.replaceChild(img, video);

On iOS 10, this provides the same user experience as using a GIF directly with graceful fallback to that GIF if none of the <video>‘s sources are supported. In fact, this code was used to show you that awesome GIF.

If your page design requires different behavior if inline playback is allowed vs. when fullscreen playback is required, use the -webkit-video-playable-inline media query to differentiate the two:

<div id="either-gif-or-video">
  <video src="image.mp4" autoplay loop muted playsinline></video>
  <img src="image.gif">
#either-gif-or-video video { display: none; }
@media (-webkit-video-playable-inline) {
    #either-gif-or-video img { display: none; }
    #either-gif-or-video video { display: initial; }

These new policies mean that more advanced uses of the <video> element are now possible, such as painting a playing <video> to a <canvas> without taking that <video> into fullscreen mode.

  var video;
  var canvas;

  function startPlayback()
    if (!video) {
      video = document.createElement('video');
      video.src = 'image.mp4';
      video.loop = true;
      video.addEventListener('playing', paintVideo);

  function paintVideo()
    if (!canvas) {
      canvas = document.createElement('canvas');
      canvas.width = video.videoWidth;
      canvas.height = video.videoHeight;
    canvas.getContext('2d').drawImage(video, 0, 0, canvas.width, canvas.height);
    if (!video.paused)
<button onclick="startPlayback()">Start Playback</button> 

The same technique can be used to render into a WebGL context. Note in this example that a user gesture–the click event–is required, as the <video> element is not in the DOM, and thus is not visible. The same would be true for a <video style="display:none"> or <video style="visibility:hidden">.

We believe that these new policies really make video a much more useful tool for designing modern, compelling websites without taxing users bandwidth or batteries. For more information, contact me at @jernoble, Jonathan Davis, Apple’s Web Technologies Evangelist, at @jonathandavis, or the WebKit team at @WebKit.

Planet Mozilla[worklog] Edition 028. appearance on Enoshima

Each time I "set up my office" (moved to a new place for the next 3 months, construction work on the main home), I'm mesmerized by how easy it is to set up a work environment. Laptop, wifi and electricity are the main things needed to start. A table and a chair are useful but non essential. And eventually an additional screen to have more working surface to be comfortable. Basically in 5 minutes we are ready to work. And that's one chance of our area of work. How long does it take before you can start working?

Working with a view on Enoshima for the next 3 months. Tune of the week: Omoide no Enoshima.

Webcompat Life

Progress this week:

Today: 2016-07-25T06:21:33.702789
296 open issues
needsinfo       5
needsdiagnosis  71
needscontact    14
contactready    34
sitewait        164

You are welcome to participate

Webcompat issues

(a selection of some of the bugs worked on this week).

  • Time to time, people are reporting usability issues which are more or less cross-browsers. They basically hinder every browsers. It's out scope for the Web Compatibility project, but hint at something interesting about browsers and users perception. Often, I wonder if browsers should do more than just supporting the legacy Web site (aka it displays), but also adjust the content to a more palatable experience. Somehow the way, Reader mode is doing on user request, a beautify button for legacy content.
  • Google Image Search and black arrow. A kind of cubist arrow for Firefox. Modern design?
  • I opened an issue on Tracking Protection and Webcompat. Adam pointed me this morning to a project on moving tracking protection to a Web extension.
  • Because we have more issues on Firefox Desktop and Firefox Android, we focus our energy there, so we need someone in the community to focus on Firefox OS issues.
  • When I test Web sites on Firefox Android, I usually do it through the remote debugging in WebIDE and instead of typing a long URI on the device itself, I usually go to the console and paste the address I want window.location = ''.
  • Starting to test a bit more in depth what appearance means in different browsers. Specifically to determine what is needed for Web compatibility and/or Web standards.
  • a WONTFIX which is a good news. Bug 1231829 - Implement -webkit-border-image quirks for compatibility. It means it has been fixed by the site owners.
  • On this Find my phone issue on Google search, the wrong order of CSS properties creates a layout issue where the XUL -moz-box was finally interpreted, but it triggered a good question from Xidorn. Should we expose the XUL display values to Web content? Add to that that some properties in the CSS never existed.
  • hangout doesn't work the same way for Chrome and Firefox. There's something happening either on the Chrome side or the servers, which creates the right path of actions. I haven't determined it yet. dev

Reading List

Follow Your Nose


  • Document how to write tests on using test fixtures.
  • ToWrite: Amazon prefetching resources with <object> for Firefox only.


Planet MozillaTenFourFox 45 is more of a thing

Since the initial liftoff of TenFourFox 45 earlier this week, much progress has been made and this blog post, ceremonially, is being typed in it. I ticked off most of the basic tests including printing, YouTube, social media will eat itself, webcam support, HTML5 audio/video, canvas animations, font support, forums, maps, Gmail, blogging and the major UI components and fixed a number of critical bugs and assertions, and now the browser is basically usable and able to function usefully. Still left to do is collecting the TenFourFox-specific strings into their own DTD for the localizers to translate (which will include the future features I intend to add during the feature parity phase) and porting our MP3 audio support forward, and then once that's working compiling some opt builds and testing the G5 JavaScript JIT pathways and the AltiVec acceleration code. After that it'll finally be time for the first beta once I'm confident enough to start dogfooding it myself. We're a little behind on the beta cycle, but I'm hoping to have 45 beta 1 ready shortly after the release of 38.10 on August 2nd (the final 38 release, barring a serious showstopper with 45), a second beta around the three week mark, and 45 final ready for general use by the next scheduled release on September 13th.

A couple folks have asked if there will still be a G3 version and I am pleased to announce the answer will very likely be yes; the JavaScript JIT in 45 does not mandate SIMD features in the host CPU, so I don't see any technical reason why not (for that matter, the debug build I'm typing this on isn't AltiVec accelerated either). Still, if you're bravely rocking a Yosemite in 2016 you might want to think about a G4 for that ZIF socket.

I've been slack on some other general interest posts such as the Power Mac security rollup and the state of the user base, but I intend to write them when 45 gets a little more stabilized since there have been some recurring requests from a few of you. Watch for those soon also.

Planet WebKitImprovements in MathML Rendering

MathML is a W3C recommendation and an ISO/IEC standard that can be used to write mathematical content on HTML5 web pages, EPUB3 e-books and many other XML-based formats. Although it has been supported by WebKit for a long time, the rendering quality of mathematical formulas was not as high as one would expect and several important MathML features were missing. However, Igalia has recently contributed to WebKit’s MathML implementation and big improvements are already available in Safari Technology Preview 9. We give an overview of these recent changes as well as screenshots of beautiful mathematical formulas rendered by the latest Safari Technology Preview. The MathML demos from which the screenshots were taken are also provided and you are warmly invited to try them yourself.

Mathematical Fonts

We continue to rely on Open Font Format features to improve the quality of the mathematical rendering. The default user agent stylesheet will try and find known mathematical fonts but you can always set the font-familyon the <math> element to pick your favorite font. In the following screenshot, the first formula is rendered with Latin Modern Math while the second one is rendered with Libertinus Math. The last formula is rendered with an obsolete version of the STIX fonts lacking many Open Font Format features. For example, one can see that integral and summation symbols are too small or that their scripts are misplaced. Scheduled for the third quarter of 2016, STIX 2 will hopefully have many issues fixed and hence become usable for WebKit’s MathML rendering.

<figure class="widescreen mattewhite">Screenshot of MathML with different fonts in Safari Technology Preview 9</figure>


WebKit now supports the href attribute which can be used to set hyperlinks on any part of a formula. This is useful to provide references (e.g. notations or theorems) as shown in the following example.

<figure class="widescreen mattewhite">Screenshot of MathML hyperlinks in Safari Technology Preview 9</figure>


Unicode contains Mathematical Alphanumeric Symbols to convey special meaning. WebKit now uses the italic characters from this Unicode block for mathematical variables and the mathvariant attribute can also be used to easily access these special characters. In the example below, you can see italic, fraktur, blackboard bold and script variables.

<figure class="widescreen mattewhite">Screenshot of MathML mathvariants in Safari Technology Preview 9</figure>


A big refactoring has been performed on the code handling stretchy operators, large symbols and radicals. As a consequence the rendering quality is now much better and many weird bugs have been fixed.

<figure class="widescreen mattewhite">Screenshot of MathML operators in Safari Technology Preview 9</figure>


Mathematical formulas can be integrated inside a paragraph of text (inline math in TeX terminology) or displayed in its own horizontally centered paragraph (display math in TeX terminology). In the latter case, the formula is in displaystyle and does not have any restrictions on vertical spacing. In the former case, the layout of the mathematical formula is modified a bit to minimize this vertical spacing and to better integrate within the surrounding text. The displaystyle property can also be set using the corresponding attribute or can change automatically in subformulas (e.g. in fractions or scripts). The screenshot below shows the layout difference according to whether the equation is in displaystyle or not. Note that the displaystyle property should also affect the font-size but WebKit does not support the scriptlevel yet.

<figure class="widescreen mattewhite">Screenshot of MathML displaystyle in Safari Technology Preview 9</figure>

OpenType MATH Parameters

Many new OpenType MATH features have been implemented following the MathML in HTML5 Implementation Note:

  • Use of the AxisHeight parameter to set the vertical position of fractions, tables and symmetric operators.
  • Use of layout constants for radicals, scripts and fractions in order to improve math spacing and positioning.
  • Use of the italic correction of large operator glyph to set the position of subscripts.

The screenshots below illustrate some of these improvements. For the first one, the use of AxisHeight allows to better align the fraction bars with the plus, minus and equal signs. For the second one, the use of layout constants for scripts as well as the italic correction of the surface integral improves the placement of its subscript.

<figure class="widescreen mattewhite">Screenshot of MathML examples with OpenType MATH parameters in Safari Technology Preview 9</figure>

Arabic Mathematics

WebKit already had support for right-to-left mathematical layout used to write Arabic mathematical notations. Although glyph-level mirroring is not supported yet, we added support for right-to-left radicals. This allows to use basic arithmetic notations as shown in the image that follows.

<figure class="widescreen mattewhite">Screenshot of Arabic Mathematics in Safari Technology Preview 9</figure>


Great changes have happened in WebKit’s MathML support recently and Igalia’s web plaform team is excited about seeing beautiful mathematical formulas in WebKit-based browsers and e-readers! If you have any comments, questions or feedback, you can contact Frédéric at or Manuel at @regocas. Detailed bug reports are also welcome to WebKit’s bug tracker. We are still refining the implementation so stay tuned for more awesome news!

Anne van KesterenFetch Standard 101

The WHATWG Fetch Standard is an essential part of the browser networking subsystem. Basically any API that involves networking (e.g., <img src>, <a href> (through navigation), XMLHttpRequest, @font-face, WebSocket) goes through Fetch. The exception is WebRTC’s RTCDataChannel and perhaps not surprisingly it has a security issue. The fetch() API is also defined in terms of Fetch and the similar naming has led to some confusion. Fetch is basically the subsystem and fetch() is one of the many APIs that exposes (part of) the capabilities of Fetch.

The basic setup is that an API prepares a request, which consists of a URL and a number of variables, feeds that to Fetch, and at some point gets a response, which consists of a body and a number of variables. Fetch takes care of content security policies, referrer policies, invoking service workers, credentials, cache modes, CORS, HSTS, port blocking, default headers (and whether they get exposed to service workers), X-Content-Type-Options: nosniff, and more. In part Fetch defines essential infrastructure such as CORS, redirect handling, port blocking, and overall terminology, and in part it serves as glue between the now numerous standards that together define the browser networking subsystem.

E.g., for redirects, Fetch defines which headers are preserved, whether a request body gets cloned and reused (it usually does), how the referrer policy gets updated, what happens with redirects to non-HTTP schemes (fail, except when navigating sometimes), but the actual connection opening and request transmission is largely left to TLS and HTTP. And as a consequence of all APIs using Fetch, redirects behave the same throughout. There are exceptions to the rule of course, but redirects are no longer a problem we need to solve on a per-API basis. And when you extrapolate this redirects example to content security policies, referrer policies, service workers, and all the other little things Fetch takes care of, it should be clear why it is essential.

(See Fetching URLs for an earlier introduction.)

W3C Team blogExploring Web platform cross-dependencies

Most of the JavaScript APIs exposed on the Web platform (both in W3C and elsewhere) rely on a formalization language called WebIDL (Web Interface Definition Language).

It provides a simple syntax to express common idioms needed when defining JavaScript APIs, and encompasses many of the specific behaviors required, expected or inherited from the 20 years legacy of Web APIs. Its common usage across APIs has facilitated more consistency across specifications (for instance in error processing), and has helped streamline the testing of browsers implementations of these APIs.

During the 2016 edition of the W3C Geek Week, my colleague François Daoust and myself have explored another way of using this formalism. We first built a crawler and scrapper of specifications (based on the excellent jsdom library) which extracts the WebIDL definitions embedded in these specifications, and the normative references that bind these specifications to specifications they build upon.

With that data collected across all the specifications we were able to identify that use WebIDL, we built a first simple analyzer of that WebIDL graph,  to identify  potential bugs in specifications, including invalid WebIDL definitions, duplication of names in interfaces, references to undefined interface names, missing normative references.

We also built a simple explorer of that WebIDL data that allows to determine which interfaces are defined where, and re-used by what other specifications.

This combination of tools has already allowed us to identify and file a number of specification bugs, and we hope to continue developing them and exploring the resulting data to inform future APIs development and provide opportunities for even more consistency across the Web Platform.

Planet WebKitFrédéric Wang: MathML Improvements in WebKit

In a previous blog post, I explained the work made by Igalia’s web platform team to refactor WebKit’s MathML layout classes. I stated that although some rendering improvements were a nice side effect, the main goal of the first phase was really to clean the code up so that it is easier for developers to work on MathML in the future. Indeed this really made things easier to review: Quite unexpectedly to me, the second phase only took 4 days to be upstreamed… Kudos to Brent Fulgham for having reviewed so many patches in such a short period of time!

In this blog post, I am going to give an overview of the improvements made during these two first phases taking changeset r203109 as a reference. The changes will be available in WebKitGTK+ 2.14 in September and are likely to be included this month in the next Safari Technology Preview. It definitely remains more work to do such as the third phase or other rendering improvements, but I believe we have already made a big step forward!

Mathematical Fonts

Two years ago, basic support for operator stretching via the OpenType MATH table was added to WebKit. During the refactoring, we improved that support and also made use of more parameters to improve the math layout (see section about OpenType MATH parameters below). While Latin Modern Math will be used in most screenshots, the following one shows that you can use any math fonts. By default WebKit will try and use one of these fonts but if none are available or if you force a non-math font-family then the rendering quality may not be good.

The following screenshot gives the rendering for various fonts. For the last one we used the value sans-serif to illustrate issues with non-math fonts (displaystyle integral too small, mathvariant italic glyphs taken from another font, missing italic correction etc).

Screenshots of a formula with various fonts Integral obtained by contour integration
WebKitGTK+ r203109 with various font-family values.
From top to bottom: TeX Gyre Schola, Latin Modern, DejaVu Math, STIX, and sans-serif.

The href Attribute

This new feature is obvious: You can now create a hyperlink for any part of a mathematical formula! Here is a screenshot of the MathML Torture Test 21 with additional links, as displayed in WebKit r203109. Click the image to load the test case in your browser and test it.

Screenshot of MathML Torture test 21 with hyperlinks

The mathvariant Attribute

Unicode contains Mathematical Alphanumeric Symbols to convey special meaning such as double-struck or specific Arabic styles. Characters for these symbols are generally provided by math fonts. In MathML, mathematical variables are automatically rendered using the italic characters from this Unicode block. One can also access these characters via the mathvariant attribute and that attribute is actually used by many LaTeX-to-MathML converters.

In the following screenshot, you can see that the letters f, x and y are now drawn with this special mathematical italic glyphs and that WebKit uses the conventional fraktur style for the Lie algebra g. Note that the prime is still too small because WebKit does not make use of the ssty feature yet.

Screenshot of Wikipedia article on Lie Algebra Homomorphism of Lie algebra
Top: Safari 9.1.1. Bottom: Safari r203109.

Operators and Spacing

As said in my previous blog post, the rendering of large and stretchy operators have been rewritten a lot and as a consequence the rendering has improved. Also, I mentioned that the width of operators may depend on their height. This may cause accumulated approximations during the computation of preferred widths. The old flexbox-based implementation incorrectly forced layout during preferred computation to avoid that but a quick workaround for that security concern caused the approximate preferred widths to be used for the logical widths. With our new implementation, the logical width is now correctly calculated. Finally, we added partial support for the mpadded element which is often used to tweak spacing in mathematical formulas.

The screenshot below illustrates the fix for a serious regression with large operator (summation symbol) as well as improvements in the rendering of stretchy operators (horizontal braces). Note that the formula has a hack with a zero-width mpadded element which used to cause improper spacing (large gap between the group of a’s and the group of b’s).

Screenshot from Mozilla's MathML torture test using Safari 9.1.1 or the the current refactoring Tests 21 and 22 from the MathML torture test
Left: Safari 9.1.1. Right: Safari r203109.

The following screenshot shows how incorrect width computations used to cause excessive spacing after the stretchy radical and slash symbols:

Screenshot of Wikipedia article on Trigonometric Functions Sine of 1 degree
Top: Safari 9.1.1. Bottom: Safari r203109.

The displaystyle Property

Mathematical formulas can be integrated inside a paragraph of text (inline math in TeX terminology) or displayed in its own horizontally centered paragraph (display math in TeX terminology). In the latter case, the formula is in displaystyle and does not have any restrictions on vertical spacing. In the former case, the layout of the mathematical formula is modified a bit to optimize this vertical spacing and to better integrate within the surrounding text. The displaystyle property can also be set using the corresponding attribute or can change automatically in subformulas (e.g. in fractions).

In the following screenshot the fix for the large operator regression is obvious but you can also notice that the summation is now slightly different for the definition of a Bézier curve (top) and for the one of a rational Bézier curve (bottom). For example, to save some vertical space in the fractions, the sigma symbol is drawn smaller and the scripts attached to it are moved on its right. However, the script size could still be improved when we implement the scriptlevel property.

Screenshot of Wikipedia article on Bézier Curves Definitions of Bézier Curve
Left: Safari 9.1.1. Right: Safari r203109.

OpenType MATH Parameters

Many new OpenType MATH features have been implemented following the MathML in HTML5 Implementation Note:

  • Use of the AxisHeight parameter to set vertical position of fractions, tables and symmetric operators.
  • Use of layout constants for radicals (some of them were already used), scripts and fractions. This improves math spacing and positioning and allows to adjust them according to value of the displaystyle property discussed in the previous section.
  • Use of the italic correction of large operator glyph to set the position of subscripts.

The screenshots below illustrate some of these improvements. For the first one, the use of AxisHeight allows to better align the fraction bar with the plus sign. For the second one, the use of layout constants for scripts as well as the italic correction of the integral improves the placement of limits.

Screenshot of Wikipedia article on the Continued Fractions Generalized Continued Fraction
Top: Safari 9.1.1. Bottom: Safari r203109.
Screenshot of Wikipedia article on the Gamma Function Definition of the Gamma Function
Top: Safari 9.1.1. Bottom: Safari r203109.

Right-to-left radicals

One of the advantage of the old flexbox-based implementation is that right-to-left layout was available for free. This support has of course been preserved in the new implementation. We also added a simple workaround to mirror radicals using a scale transform as shown in the screenshot below. However, general glyph-level mirroring is still missing.

Screenshot of Wikipedia article on Lie Algebra Test 13 from the MathML torture test (Machrek Style)
Top: Safari 9.1.1. Bottom: Safari r203109.


Igalia’s web platform team has been able to follow the MathML in HTML5 Implementation Note in order to significantly improve the rendering of mathematical expressions in WebKit. More work remains to do but we will definitely appreciate any feedback that can help improving native MathML support in web engines. We are also excited to continue work and collaboration at the next Web Engines Hackfest!

Planet MozillaFirefox 47 Support Release Report

Firefox 47 Release and Firefox iOS 4.0

In London, Mozlondon, we had a session on creating a SUMO Release Report a few weeks after major updates to Mozilla products. This post will be the first to include testimonials from users and submissions from users in the community to make it unique to SUMO. With the intention to highlight all of the work that the community comes to accomplish together, the user testimonials, feedback, copious issues found, brought to attention and solved, knowledge base articles created, and collaborated on, as well as article translations to so many languages and organized social media this report shows how much we need your help. Core Community Members and new ones are equally as important.  We have highlighted the issues that were and are actively being tracked down to improve Firefox and other Mozilla products.

We have lots of ways to contribute, from Support to Social to PR, the ways you can help shape our communications program and tell the world about Mozilla are endless. For more information: []

Customer Kudos to the SUMO community

1104 users said “thank you” out of the 7300 answers during this time.

christ1 - 1127702 jscher2000 - 1127482 linuxmodder - 1126874 corel-1127667

We cannot include all of the thank yous that were received, however these are many of the community members that also received thank yous from Firefox users. Shout outs to Fred, cor-el, Seburo, philipp, Matt, Zenos, Scribe, jscher2000, James, Wayne Mary, Chris Ilias, Christ1, the-edmeister, Tonnes, Toad-Hall. They all received direct thank yous from users and their solved issues.

Feedback and community highlights

Knowledge Base

Article Voted “helpful” (English/US only) Global views Constructive User Feedback

(June 7 – June 30)

Allow Firefox to load multiple tabs in the background 71-76% 5340 “Why do you think it is a good idea to confuse existing users with taking away the options they once had? Why change the options that they choose to set? I am mildly upset”
Pages appear tiny when I print or view them in Print Preview 51-62% 3281 “Still having an issue.”
“My print_paper_height and _width settings appear in millimeters even though the paper being used is set to 8.5 x 11 inches. Margin settings still appear in inches”
“Prints half size in width. Followed instructions exactly”
“page goes from very small when printing to very large font when using email”
“actually my log in page is about the size of a dollar bill…..I cant see it because it is so small I can go to Internet Explorer and I have NO PROBLEM but firefox another story”
Firefox support has ended for OS X 10.6, 10.7 and 10.8 57-83% 3222
Watch DRM content on Firefox 66-70% 209049 “never had a problem watching videos on amazon prime till you people came up with this explanation that to non tech people is just jibberish”
“all of a sudden not work to stream toytube or netfilx”
“Has no mention of whether Linux will have Widevine support in the future. This seems odd given that Google Chrome already has that support built-in.”

(June 7 – June 30)

Turn off web fonts in Firefox for Android 100% 340 none
What’s new in Firefox for Android 8 none
Firefox Marketplace Apps Stop Working on Firefox 47+ for Android 71-95% 26612 none
What’s new in Firefox for iOS (version 4.0) 60-87% 1,618,561 None
Add Firefox to the Today view on your iOS device 75-85% 18,071 None
Certificate warnings in Firefox for iOS 72-76% 2,751 none

**No articles were linked from major publications (via Google Analytics.) but if you see any in your region, please mention them.


Article Top 10 locale coverage Top 20 locale coverage
Desktop (June 7 – June 30)
Allow Firefox to load multiple tabs in the background 100% 66.6%
Pages appear tiny when I print or view them in Print Preview 100% 66.6%
Firefox support has ended for OS X 10.6, 10.7 and 10.8 40% 23.8%
Watch DRM content on Firefox 100% 80%
Android (June 7 – June 30)
Turn off web fonts in Firefox for Android 100% 66.6%
What’s new in Firefox for Android 60% 33%
Firefox Marketplace Apps Stop Working on Firefox 47+ for Android 100% 57%

Support Forum Threads

One of the major impacted issues during the first three weeks of the release was an increase in reports to fake updates and malware from those updates. Many of them were reported and still be investigated.

Not solved top viewed threads – GA

In this spreadsheet:

Bugs Created from Forum threads – SUMO Community

Bugs mentioned

Social Support Highlights

Brought to you by Sprinklr

Total contributors in program

In this time we had a total of 14 of you login and participate in the Firefox 47 release.

New users added in period of the report

Welcome Magno, Daniella, Luis, and TheoC to the team you were very active these past three weeks and thank you for supporting Mozilla Open Source users on Facebook and Twitter on the Sprinklr tool.

Top 5 Contributors

User Number of Replies
Andrew Truong 56
Noah Y 24
Jhonatas Rodrigues Machado 12
Alex_Mayorga 6
Magno Reis 4

Number of Replies: 111

Trending issues in Sprinklr

Outbound top engagement:

Each Facebook outbound post reached one person for support, the two major engagements overlapped with Code Emoji and the plane,

Top Twitter Posts

This version we removed the tag summary and are currently working on items that translate to more specific categories. Not working will be removed and more will be added. However taking a deep dive in the top categories for outbound messages to Mozilla Open Source product users this what we found.

  • “Not working” is associated with these suggested troubleshooting steps and categories: antivirus was mentioned 3 times, malware  was mentioned once, trying a Firefox Refresh 1, hanging 1, Trying Safe mode 1, and Kaspersky 1, changing a Firefox setting was mentioned once, and a more complicated website issue was asked to troubleshoot in the forums. This can be concluded to be some of the most common troubleshooting steps for Firefox for Desktop described here:
  • “Crashes” is associated with outbound troubleshooting that  say check out how to ttp:// or go to the support forum 50/50 of the time.
  • “Video” tags were the third top category and basic troubleshooting were to clear the cache, asks if the website is the issue and suggests html5.

Planet WebKitFrédéric Wang: MathML Refactoring in WebKit


If you follow WebKit developments, you are certainly aware that Igalia has been working on WebKit’s MathML implementation for some time. More recently, effort has been made to write a clean implementation addressing issues reported by WebKit reviewers in the past. After joining Igalia in March, I have been in charge of getting this work reviewed and merged into WebKit’s development branch. In the past four months, we have been successful in upstreaming the first phase of the refactoring and the work accomplished is described in this blog post.

Screenshot from Mozilla's MathML torture test using Safari 9.1.1 or the the current refactoring
MathML torture tests with the Latin Modern Math font
Left: Safari 9.1.1. Right: Current MathML branch.

Note that the focus was on code refactoring so the improvement may not be obvious for non-developers. Nevertheless many issues have already been fixed as a consequence of that work: math italic, position of scripts, stretchy and large operators, rendering update and more. More importantly, this preliminary step opens the way for beautiful math rendering based on TeX rules and the OpenType MATH table. Rendering improvements and implementation of new features have already started in the next phase of the refactoring, so stay tuned…

Design Issues

As explained in a previous report, the main design issues in the flexbox-based implementation released in 2013 can essentially be summarized in three points:

  1. WebKit’s code to stretch operator was not efficient at all and was limited to some basic fences buildable via Unicode characters.
  2. WebKit’s MathML code violated many layout invariants, making the code unreliable.
  3. WebKit’s MathML code relied heavily on the C++ renderer classes for flexboxes and had to manage too many anonymous renderers.

For the first point, the performance issue had been fixed by Igalia developers right after the initial feedback from WebKit developers and we improved that again during our refactoring. Partial support for the OpenType MATH table was added during my crowdfunding project and allowed to stretch more operators with the help of math fonts. For the second point, the main issue was also fixed right after the initial feedback. However one could still have some doubts regarding the layout steps, given the complexity implied by the third point. That last issue was still problematic so far and addressing it was the main achievement of our refactoring.

Technically, the dependence on flexbox is unnecessary and the implementation actually only used a limited set of flexbox features. Thus executing the whole flexbox code was overkill. It can also be a burden for people working on other places of the layout code. For example Javi Fernández has worked on improving the box alignments in the past and he had hard time fixing the MathML code impacted by his changes. This is probably the cause of the bad position of the summation symbol that can be seen in the screenshot above.

From the layout perspective, most of the rendering logic was implemented in the flexbox classes and the MathML “renderer” classes were really just managing the creation and update of anonymous nodes and style. Although this sounds good code reuse it actually made impossible to understand how and when layout steps happen or to add more advanced features. The new implementation replaced this manipulation of the render tree with simple arithmetic calculations on box metrics which is safer and more reliable. Also, complex renderers such as RenderMathMLScripts or RenderMathMLRoot actually achieve better rendering quality with less code!

As an example of the complexity, RenderMathMLUnderOver can behave as a RenderMathMLScripts in some situation so we really want the former class to reuse the latter class. As we will see below the old implementation of the two renderers were quite different: RenderMathMLUnderOver only relied on setting column direction in the user agent stylesheet while RenderMathMLScripts created a complex render tree structures with anonymous style. Hence it seemed difficult to share the two cases or to handle DOM changes causing to move from one case to the other one. With our new implementation, this is simply reduced to simple C++ inheritance.

When I started to work on WebKit some years ago, I made the mistake of continuing with the existing approach. The implementation of multiscripts or automatic italic mathvariant added more anonymous objects and made the situation even worse. After the end of my crowdfunding project, Alex Castro did more cleanup and tried to implement important features such as displaystyle but he also soon realized that it was too hard to work with the current code base…

Layout Refactoring

In order to solve the issues discussed in the previous section, Javi and Alex worked on a new MathML branch where the first step was to remove the inheritance on the flexbox layout classes. During the Web Engines Hackfest 2015, I collaborated with the Igalia’s web platform team team to continue the work on this branch. In a second step, we rewrote many MathML renderer functions so that they stop creating anonymous nodes or style. We obtained very encouraging results: The implementation looked much simpler and much more understandable!

Alex announced the initial plan on the webkit-dev mailing list. He started opening bugs and attaching patches to merge the first step. However, that step still required many of the flexbox logic and so made code hard to understand for reviewers. Hence when I joined Igalia four months ago Alex asked me to try and see how to reorganize patches so that the two initial steps can be submitted in one go. This corresponds to the first phase mentioned in the introduction. As indicated on the wiki page, the layout refactoring consisted in rewriting the following member functions of each renderer class:

  • computePreferredLogicalWidths: calculate preferred widths, based on the preferred widths of child renderers.
  • layoutBlock: set final position and size of child renderers.
  • firstLineBaseLine: calculate the ascent of the renderer.
  • paint (optional): perform special painting such as fraction bars.

Refactored renderers no longer rely on any flexbox code nor anonymous renderers and the functions mentioned above essentially perform arithmetic computations. By reading the code, one can be sure that we follow standard layout rules and that we do not perform unnecessary reflow. Moreover, the rules specific to math rendering are only located in the MathML renderers and can be better understood. Details for each class are provided in the next subsections. After all the layout functions were rewritten and the code managing the render tree structure removed, we were able to make the RenderMathMLBlock class inherit from RenderBlock instead of RenderFlexibleBox. Many of the bugs could then be immediately closed or otherwise fixed with small follow-up patches.


RenderMathMLSpace is a simple class inserting blank boxes for adjusting spacing of math formulas. Obviously, we do not need any of the complexity of flexbox so it was straightforward to write the layout functions.

3x Large space between 3 and x.


RenderMathMLRow performs rendering of a row of math items. Since WebKit does not support linebreaking in MathML at the moment, this is just putting child boxes on a same baseline. One specificity is that some operators can be stretched vertically and so their width may depend on their height.

{ 2x x 3 Row containing a stretched brace, a fraction and a scripted element.

Again, flexbox features are useless here. With the old code, it was not clear whether we were violating the CSS invariant with preferred and logical widths and which kind of relayout or render tree changes would happen when doing the stretch call. By properly implementing the layout functions previously mentioned all of this became much more trustable.


RenderMathMLFraction draws a fraction with numerator and denominator.

x+1y+2 Simple fraction.

This used to be implemented using a column direction for the fraction element. Numerator and denominator were wrapped into anonymous nodes with additional style to leave space for the fraction bar and to adjust the horizontal alignments.

RenderMathMLFraction (flexbox with column direction)
  RenderMathMLBlock (anonymous flexbox)
    RenderMathMLRow (numerator)
  RenderMathMLBlock (anonymous flexbox)
    RenderMathMLRow (denominator)

It was relatively easy to implement this without any anonymous nodes and again the use of flexbox did not sound justified. For example, to calculate the preferred width we just take the maximum of the preferred widths of the numerator and denominator. For the layout, the calculation of the logical width is similar and we calculate the horizontal coordinates of numerator and denominator so that their centers are aligned. Vertical metrics are similarly calculated from the vertical metrics of the numerator and denominator. During that step, we also fixed some bugs with the linethickness attribute and added support for some OpenType MATH table constants.

Scripts above and below

RenderMathMLUnderOver is used to attach some scripts above and below a base. Each child can itself be a horizontal stretchy operator.

base→ Base with stretchy arrow over it.

This was implemented in the user agent stylesheet by using flexboxes with column direction for the corresponding MathML elements and the C++ class had additional rules to fire the stretching. So the problems and solutions for this class were essentially a mixed of the cases of RenderMathMLFraction and RenderMathMLRow we just discussed.

Subscripts and Superscripts

RenderMathMLScripts is used for a base with some arbitrary number of scripts. All the scripts can have different positions (pre, post, sub, super) and metrics (width, ascent and descent). We must avoid collisions and take care of horizontal and vertical alignements.

base a b c d e f Base with pre and post scripts.

The old code used a complex render tree with additional style to achieve the best possible result. However, the quality was still bad as you can see for the script attached to the integral in the screenshot above. Managing the render tree was a nightmare: Just to give the idea, additional anonymous node and style were used to allow horizontal and vertical adjustments (similar to RenderMathMLFraction above) and prescripts had negative order property so that they were positioned before the base.

    Base Wrapper (anonymous flexbox)
      RenderMathMLRow (base)
    SubSupPair Wrapper (anonymous flexbox with column direction)
      RenderMathMLRow (post-subscript)
      RenderMathMLRow (subperscript)
    SubSupPair Wrapper (anonymous flexbox with column direction)
      RenderMathMLRow (post-subscript)
      RenderMathMLRow (post-superscript)
    ... (more postscripts)
    RenderMathMLBlock (prescripts separator)
    SubSupPair Wrapper (anonymous flexbox with column direction and order -1)
      RenderMathMLRow (pre-subscript)
      RenderMathMLRow (pre-subperscript)
    SubSupPair Wrapper (anonymous flexbox with column direction and order -1)
      RenderMathMLRow (pre-subscript)
      RenderMathMLRow (pre-superscript)
    ... (more prescripts)

Rules from TeX and the OpenType MATH table are relatively complex and we decided to implement them directly in the new refactoring as otherwise it was impossible to get decent quality. The code is still complex but we now have clear rules, we only perform simple calculations and the render tree structure matches the DOM tree.

“Enclosing” Notations

RenderMathMLMenclose is a row of math items with some additional notations. Gurpreet Kaur implemented this element two years ago but she followed the same approch, combining anonymous nodes and style for some simple notations and special painting for others.

x+1 circle and strike notations

During the refactoring, the code has been completely rewritten so that RenderMathMLMenclose is now essentially a derived class of RenderMathMLRow with the measuring and painting functions adjusted to take into account the additional notations. During that refactoring, we also removed support for unused radical notation, which was implemented using an anonymous RenderMathMLSquareRoot (see Radicals section below).

Helper Classes for Operators

The RenderMathMLOperator class is used for math operators. It was quite complex class and we decided to extract from it two features that are unrelated to layout:

The remaining code was indeed the real layout part but the mess with anonymous node and style was only removed later (see Text Classes below). Although it seems we just needed to move the code out of RenderMathMLOperator into those new classes, the case of MathOperator was particularly difficult. We had to split the effort into several small steps to make review possible and also fixed many issues due to the entanglement and confusion of these three different features of the RenderMathMLOperator class… The work done for MathOperator actually improved the rendering of stretchy operators as you can see for the horizontal braces in the screenshot above.


RenderMathMLRoot is used for square root or arbitrary N-th root. Many of the TeX and OpenType MATH table rules were already used by the old implementation with anonymous nodes and style. However, there were bugs difficult to fix related to zooming, child removal or style change due to the management of the anonymous RenderMathMLOperator to draw the radical sign.

x+1 + x+1 3 square and cube roots

The old implementation actually had two classes for the square and general cases (RenderMathMLSquareRoot and RenderMathMLRoot). The usual technique with various anonymous wrappers and style was used. After the refactoring, we were able to merge everything in a single RenderMathMLRoot class. Because the square root behaves as an mrow, we also made that class derive from RenderMathMLRow to reuse as much code as possible. Here is are how the render trees used to look like:

  RenderMathMLBlock (anonymous used for metric adjustements)
    RenderMathMLRadicalOperator (anonymous used for the radical symbol)
  RenderMathMLRootWrapper (anonymous used for the children)
    RenderMathMLRow (child 1)
    RenderMathMLRow (child 2)
    RenderMathMLRow (child N)

  RenderMathMLRootWrapper (anonymous for the index)
  RenderMathMLBlock (anonymous used for metric adjustements)
     RenderMathMLRadicalOperator (anonymous used for the radical symbol)
  RenderMathMLRootWrapper (anonymous for the base)

Again, we rewrote the implementation using only simple box positioning. The difficult part was to get rid of the anonymous RenderMathMLRadicalOperator to draw the radical symbol. This class was derived from RenderMathMLOperator and extended it with some fallback drawing when math fonts were not available. After having extracted stretchy operator shaping from RenderMathMLOperator it became possible to use the MathOperator helper class to draw the radical symbol. We implemented the fallback for missing math fonts the same as Gecko: Use a scale transform to stretch the base glyph for U+221A SQUARE ROOT. As a bonus, we used such transform to implement glyph mirroring, as required to draw right-to-left radicals in some Arabic mathematical notations.

Text Classes

These classes are containers for math text such as variables or operators. There is a generic RenderMathMLToken class and a derived class RenderMathMLOperator adding features specific to operators such as spacing, dictionary property, stretching… Anonymous wrappers and style were used to implement automatic italic mathvariant or operator spacing. The RenderText child of RenderMathMLOperator was (re)built as an anonymous text node so that is was possible to convert U+002D HYPHEN-MINUS into U+2212 MINUS SIGN or to provide some text for anonymous operators created by RenderMathMLFenced (see Unchanged Classes section).

RenderMathMLToken (e.g. mi element)
  RenderMathMLBlock (anonymous flexbox used to apply CSS italic)
    RenderBlock (anonymous created elsewhere to honor CSS rules)
        text run "x"

RenderMathMLOperator (mo element)
   RenderMathMLBlock (anonymous flexbox used for spacing)
    RenderBlock (anonymous created elsewhere to honor CSS rules)
      RenderText (anonymous destroyed and built again)
         text run "−"

We did a big refactoring to remove all the anonymous nodes created by the MathML renderer classes. Just like for MathOperator, we had to be careful and submit various small pieces as the text rendering was quite sensible to code change.

The simplified operator spacing that was supported by WebKit was easy to implement with the new approach. To do automatic italic mathvariant, we modified the paint function to use Mathematical Alphanumeric Symbols instead of CSS italic as you can notice for the variables displayed in the screenshot above. Hence we could remove the RenderMathMLBlock anonymous wrapper.

The use of an anonymous node for the text prevented it to appear in the dumped render tree of layout tests and also required some hacks in the accessibility code to expose that text. In order to address the cases of the minus sign and of mfenced operators, we decided to use our new MathOperator class again. Indeed MathOperator is actually also able to draw unstretched operators made of a single character and this works for the minus sign and for mfenced operators used in practice.

Unchanged Classes

Two classes have not been modified but such modifications were not needed to remove the dependency on RenderFlexibleBox:

  • RenderMathMLFenced is used for an mrow-like element that is defined in the MathML specification as strictly equivalent to constructions with rows and operators. It is implemented as a derived class of RenderMathMLRow and creates anonymous RenderMathMLOperators. This is the only remaining class that modifies the render tree structure. Note that prominent MathML websites and generators do not use the mfenced element, so it is not a big concern.

  • RenderMathMLTable is used for table layout. It is just derived from RenderTable, not RenderFlexibleBox. We did not change anything for now but we considered creating our own implementation in order to make our code independent from HTML table, to support MathML-specific table features and to make it better integrated with the rest of the MathML code.


Even if our main focus was on rendering, the changes we made also had impact on the MathML accessibility code. Indeed, the accessibility tree is generated from the MathML renderer classes: Since we changed the latter during the refactoring, we also had to adjust the accessibility code. Fortunately, we are lucky to have Joanmarie Diggs in our team and she was able to provide some help here.

First, the accessibility code exposes the linethickness of fractions to implement Apple’s AXMathLineThickness attribute. In practice, this is really necessary to know whether the linethickness is null or not (e.g. binomial coefficient VS the Legendre symbol). Apple’s unit test seemed to expose the ratio between the actual thickness and the default thickness but the accessibility code really just reads the actual thickness calculated by RenderMathMLFraction. Our fix and improvement for linethickness made the Apple’s unit test fail so we had to adjust RenderMathMLFraction to expose the value expected by that test.

In general, the accessibility code does not care about anonymous nodes created for layout purpose and there was some code to avoid exposing them in the accessibility tree. So removing all the anonymous during the layout refactoring was actually a good and safe thing to do. There were some helper functions to implement Apple’s AXMathRootRadicand and AXMathRootIndex attributes that had to be adjusted, though. These functions used to do some work to skip the anonymous wrappers and we were actually able to simplify them.

There was also some specific code for the RenderMathMLOperators and their anonymous RenderText that were necessary to expose the text content. Actually, there was an old bug in the accessibility code and the anonymous RenderMathMLOperators created by mfenced were not correctly exposed. The unit test we had for mfenced operators was only checking the text content but it was still passing and so the regression had never been detected before. After the layout refactoring we removed the anonymous RenderText of mfenced operators and so broke that test… We thus spent some time to fix the RenderMathMLOperator code. Essentially, we removed all the old hacks and only left a specific handling for mfenced operators. We also used this opportunity to improve and extend our MathML accessibility tests.

Finally, the MathML accessibility code was directly implemented into a generic AccessibilityRenderObject class. There was some functions to access math nodes and properties but also specific cases scattered all over the code (anonymous boxes, mfenced operators, math roles etc). In order to facilitate future work and maintenance we decided to move all the MathML code into a new AccessibilityMathMLElement class. Hence the work implied by the layout refactoring actually encouraged us to improve the organization and testing of our accessibility code!


In the past four months, Igalia’s web platform team has successfully upstreamed the refactoring of WebKit’s MathML renderer classes and we are now very confident about the quality of the layout code. In addition to the people mentioned above I would personally like to thank everybody who helped with this work. More specifically, I am very grateful to other people from Igalia (Martin Robinson, Sergio Villar and Manuel Rego) or Apple (Brent Fulgham and Darin Adler) who have spent some time to review patches. As a nice side effect of this work, mathematical formulas look better and the accessibility code has been improved. More is happenning in the next two phases. We are looking forward to continuing implementation of Web standards and collaboration with browser vendors at the next Web Engines Hackfest!

Jeremy KeithBackdoor Service Workers

When I was moderating that panel at the Progressive Web App dev Summit, I brought up this point about twenty minutes in:

Alex, in your talk yesterday you were showing the AMP demo there with the Washington Post. You click through and there’s the Washington Post AMP thing, and it was able to install the Service Worker with that custom element. But I was looking at the URL bar …and that wasn’t the Washington Post. It was on the CDN from AMP. So I talked to Paul Backaus from the AMP team, and he explained that it’s an iframe, and using an iframe you can install a Service Worker from somewhere else.

Alex and Emily explained that, duh, that’s the way iframes work. It makes sense when you think about it—an iframe is pretty much the same as any other browser window. Still, it feels like it might violate the principle of least surprise.

Let’s say you followed my tongue-in-cheek advice to build a progressive web app store. Your homepage might have the latest 10 or 20 progressive web apps. You could also include 10 or 20 iframes so that those sites are “pre-installed” for the person viewing your page.

Enough theory. Here’s a practical example…

Suppose you’ve never visited the website for my book, (if you have visited it, and you want to play along with this experiment, go to your browser settings and delete anything stored by that domain).

You happen to visit my website There’s a little blurb buried down on the home page that says “Read my book” with a link through to I’ve added this markup after the link:

<iframe src="" style="width: 0; height: 0; border: 0">

That hidden iframe pulls in an empty page with a script element:

<!DOCTYPE html>
<html lang="en">
<meta charset="utf-8">
<title>HTML5 For Web Designers</title>
if ('serviceWorker' in navigator) {

That registers the Service Worker on my book’s site which then proceeds to install all the assets it needs to render the entire site offline.

There you have it. Without ever visiting the domain, the site has been pre-loaded onto your device because you visited the domain

A few caveats:

  1. I had to relax the Content Security Policy for to allow the iframe to be embedded on

    Header always set Access-Control-Allow-Origin: ""
  2. If your browser’s settings has “Block third-party cookies and site data” selected in the preferences, the iframe-invoked Service Worker won’t install:

    Uncaught (in promise) DOMException: Failed to register a ServiceWorker: The user denied permission to use Service Worker.

The example I’ve put together here is relatively harmless. But it’s possible to imagine more extreme scenarios. Imagine there’s a publishing company that has 50 websites for 50 different publications. Each one of them could have an empty page waiting to be embedded via iframe from the other 49 sites. You only need to visit one page on one of those 50 sites to have 50 Service Workers spun up and caching assets in the background.

There’s the potential here for a tragedy of the commons. I hope we’ll be sensible about how we use this power.

Just don’t tell the advertising industry about this.

Jeremy KeithUnlabelled search fields

Adam Silver is writing a book on forms—you may be familiar with his previous book on maintainable CSS. In a recent article (that for some reason isn’t on his blog), he looks at markup patterns for search forms and advocates that we should always use a label. I agree. But for some reason, we keep getting handed designs that show unlabelled search forms. And no, a placeholder is not a label.

I had a discussion with Mark about this the other day. The form he was marking up didn’t have a label, but it did have a button with some text that would work as a label:

<input type="search" placeholder="…">
<button type="submit">

He was wondering if there was a way of using the button’s text as the label. I think there is. Using aria-labelledby like this, the button’s text should be read out before the input field:

<input aria-labelledby="searchtext" type="search" placeholder="…">
<button type="submit" id="searchtext">

Notice that I say “think” and “should.” It’s one thing to figure out a theoretical solution, but only testing will show whether it actually works.

The W3C’s WAI tutorial on labelling content gives an example that uses aria-label instead:

<input type="text" name="search" aria-label="Search">
<button type="submit">Search</button>

It seems a bit of a shame to me that the label text is duplicated in the button and in the aria-label attribute (and being squirrelled away in an attribute, it runs the risk of metacrap rot). But they know what they’re talking about so there may well be very good reasons to prefer duplicating the value with aria-label rather than pointing to the value with aria-labelledby.

I thought it would be interesting to see how other sites are approaching this pattern—unlabelled search forms are all too common. All the markup examples here have been simplified a bit, removing class attributes and the like…

The BBC’s search form does actually have a label:

<label for="orb-search-q">
Search the BBC
<input id="orb-search-q" placeholder="Search" type="text">
<button>Search the BBC</button>

But that label is then hidden using CSS:

position: absolute;
height: 1px;
width: 1px;
overflow: hidden;
clip: rect(1px, 1px, 1px, 1px);

That CSS—as pioneered by Snook—ensures that the label is visually hidden but remains accessible to assistive technology. Using something like display: none would hide the label for everyone.

Medium wraps the input (and icon) in a label and then gives the label a title attribute. Like aria-label, a title attribute should be read out by screen readers, but it has the added advantage of also being visible as a tooltip on hover:

<label title="Search Medium">
  <span class="svgIcon"><svg></svg></span>
  <input type="search">

This is also what Google does on what must be the most visited search form on the web. But the W3C’s WAI tutorial warns against using the title attribute like this:

This approach is generally less reliable and not recommended because some screen readers and assistive technologies do not interpret the title attribute as a replacement for the label element, possibly because the title attribute is often used to provide non-essential information.

Twitter follows the BBC’s pattern of having a label but visually hiding it. They also have some descriptive text for the icon, and that text gets visually hidden too:

<label class="visuallyhidden" for="search-query">Search query</label>
<input id="search-query" placeholder="Search Twitter" type="text">
<span class="search-icon>
  <button type="submit" class="Icon" tabindex="-1">
    <span class="visuallyhidden">Search Twitter</span>

Here’s their CSS for hiding those bits of text—it’s very similar to the BBC’s:

.visuallyhidden {
  border: 0;
  clip: rect(0 0 0 0);
  height: 1px;
  margin: -1px;
  overflow: hidden;
  padding: 0;
  position: absolute;
  width: 1px;

That’s exactly the CSS recommended in the W3C’s WAI tutorial.

Flickr have gone with the aria-label pattern as recommended in that W3C WAI tutorial:

<input placeholder="Photos, people, or groups" aria-label="Search" type="text">
<input type="submit" value="Search">

Interestingly, neither Twitter or Flickr are using type="search" on the input elements. I’m guessing this is probably because of frustrations with trying to undo the default styles that some browsers apply to input type="search" fields. Seems a shame though.

Instagram also doesn’t use type="search" and makes no attempt to expose any kind of accessible label:

<input type="text" placeholder="Search">
<span class="coreSpriteSearchIcon"></span>

Same with Tumblr:

<input tabindex="1" type="text" name="q" id="search_query" placeholder="Search Tumblr" autocomplete="off" required="required">

…although the search form itself does have role="search" applied to it. Perhaps that helps to mitigate the lack of a clear label?

After that whistle-stop tour of a few of the web’s unlabelled search forms, it looks like the options are:

  • a visually-hidden label element,
  • an aria-label attribute,
  • a title attribute, or
  • associate some text using aria-labelledby.

But that last one needs some testing.

Update: Emil did some testing. Looks like all screen-reader/browser combinations will read the associated text.

Planet WebKitImproving Color on the Web

The past few years have seen a dramatic improvement in display technology. First it was the upgrade to higher-resolution screens, starting with mobile devices and then desktops and laptops. Web developers had to understand high-DPI and know how to implement page designs that used this extra resolution. The next revolutionary improvement in displays is happening now: better color reproduction. Here I’ll explain what that means, and how you, the Web developer, can detect such displays and provide a better experience for your users.

Let’s call the typical computer monitor, the type that you’ve been using for more than a decade, an sRGB display. Apple’s recent hardware, including the late-2015 Retina iMac and early-2016 iPad Pro, are able to show more colors than an sRGB display. Such displays are called wide-gamut (I’ll explain what sRGB and gamut are below).

Why is this useful? A wide-gamut system will often provide a more accurate representation of the original color. For example, my colleague @hober has some extremely funky shoes.

<figure class="widescreen mattewhite">hober's bright orange shoes</figure>

Unfortunately, what you’re seeing above doesn’t really convey just how funky the shoes are! The problem is the shoe fabric uses colors that are unable to be represented by an sRGB display. The camera that took the photograph (Sony a6300) has a sensor that is able to capture more accurate color information, and that data are included in the original file, but the display can’t show it. Here’s a version of the photo where every pixel that used a color outside the range of a typical display has been replaced with a light blue:

<figure class="widescreen mattewhite">hober's bright orange shoes, but with all the out-of-gamut pixels replaced by blue</figure>

As you can see, the colors on the shoe fabric and much of the grass are outside the range of the sRGB display. In fact, fewer than half of the pixels in the photograph are accurately represented. As a Web developer, you should be concerned by this. Suppose you’re an online store selling these shoes. Your customers are not going to know exactly what color they’ve ordered, and might be surprised when their purchase arrives.

This problem is lessened with a wide-gamut display. If you have one of the devices mentioned above, or a similar device, here is a version of the photograph that will show you more colors:

<figure class="widescreen mattewhite">hober's bright orange shoes, this time with a color profile embedded</figure>

For those of you on a wide-gamut display, you’ll see the shoes are a brighter orange, and there is slightly more variation in the green grass. Unfortunately, if you don’t have a wide-gamut display, you’re probably seeing something very close to the first image above. In this case, the best I can offer is that tinted image that highlights which parts of the image you’re missing out on.

Anyway, this is great news! Wide-gamut displays are more vibrant, and provide a more accurate depiction of reality. Obviously you’re going to want to make sure you serve your users with imagery that makes use of this technology.

Here’s another example, this time with a generated image. To users on an sRGB display there is a uniform red square below. However, it’s a bit of a trick. There are actually two different shades of red in that image, one of which is only distinct on wide-gamut displays. On such a display you’ll see a faint WebKit logo inside the red square.

<figure class="widescreen mattewhite">A red square with a faint WebKit logo</figure>

Sometimes the difference between a normal and wide-gamut image is subtle. Sometimes it is more dramatic.


Take a look at this page of examples that lets you swap between different versions of images to compare, as well as shows you where in the image there are pixels outside of the range of an sRGB display. There’s also a slightly more interactive version that shows you the different images side by side.


Here’s a quick explainer for terms you often hear when discussing color.

Color Space: A color space is an environment in which you can define and compare colors. There are a few types of color spaces, each using a different set of parameters to describe the colors. For example, a greyscale color space only has one parameter which controls the level of brightness going from black to white. You’re probably familiar with RGB-type color spaces that use red, green, and blue as parameters representing the colors of the lights in a display that add together to make a final color. Print workflows often use CMYK-type color spaces, where cyan, magenta, yellow, and black are the parameters representing the colors of the inks.

Color Profile: In 1993, a group of vendors formed the ICC to define a standard that described color spaces. A color profile is data that defines what the color space of a device is, and can be used to convert between different spaces. The common ones are given names, such as sRGB (or more formally, IEC 61966-2-1). My use of sRGB above now makes a bit more sense: such a display is able to show colors corresponding to the sRGB color space. Color profile data can be written to a file, or embedded directly into an image, which allows a computer to understand what the color values in the image actually mean.

Gamut: A gamut is the range of colors a device can process, or that a color space can define. In the case of a computer display, a gamut is all the colors it can accurately show. Visualizing a gamut is a bit complicated, but it is slightly similar to the color pickers you see in design software. Imagine the range of all possible colors applied to a surface, with the primary colors at different extremes. As you move towards the red extreme, the color gets more red. Moving towards blue, more blue, etc. A gamut is the area on this imaginary surface that represents as far as the device can go in all directions. You might see a gamut represented with a diagram such as this, where the colored shape is showing what a typical human eye can see, and the white triangle shows the range of the sRGB color space (as you notice, it’s a lot smaller than what the eye could detect).

<figure class="widescreen mattewhite aligncenter">Diagram showing the sRGB gamut

Image source: Wikipedia </figure>

These diagrams can be a bit misleading because they are showing you colors that you can obviously see, and then telling you that a gamut doesn’t include those colors. However, they do give you a nice way to compare the size of different gamuts. Note also that here you’re seeing a two-dimensional representation of a color gamut surface, when in reality it is a three- or four-dimensional space (this is all pretty complicated – we’re just trying to give a simple introduction).

Wide-gamut: This is an informal term that the industry is using to describe devices or color spaces that have a gamut larger than sRGB, which is the gamut that nearly all computer displays have used for the last decade or so. Wider gamut displays have been around for a while, but were mostly limited to professional use. Now they are becoming available to regular consumers, which means that there are more colors available. Sometimes wide-gamut is also referred to as extended color. The modern Apple displays support the Display P3 color space, which is about 25% wider than sRGB.

Color Depth: Computers can use varying levels of precision, or depth, to represent a color. This is not the same as gamut, which describes a range of colors. Rather it is the number of distinct colors within the gamut that can be defined. Web developers are probably familiar with the CSS rgb() syntax, which takes 0-255 values for red, green and blue. That is a depth of 8 bits per channel, for a total of 16,777,216 different colors. If you add in alpha/opacity, you can store the color in 32 bits. If you use a depth of 8 bits per channel, you can only ever represent that same number of colors, no matter what color space you’re using – it would just be a different set of colors. If you chose 16 bits per channel, you would have a deeper space, and could represent more colors within the gamut. A good example of this is when you draw a gradient between similar colors: you can see banding, where the computer and display don’t have the depth to show a smooth range of colors between the end points.

Here is an example which shows how a lack of color depth produces banding, even though all the colors between the end points are within the gamut (it’s an artificial example to exaggerate the effect).

A smaller color depth might demonstrate distinct jumps between similar colors.

<figure class="widescreen mattewhite">A light red to dark red gradient with distinct bands</figure>

With a larger color depth, the jumps are much less noticeable.

<figure class="widescreen mattewhite">A light red to dark red gradient without distinct bands</figure>

With that introduction, let’s get into the details of color on the Web and recent improvements in WebKit to help you develop content that is more aware of color. We also introduce some features we’ve proposed to the W3C that will allow you to take even more advantage of this new display technology.

Colors on the Web

The Web has often struggled to handle colors correctly. I’m sure there are some readers out there who painfully remember Web-safe colors! While we’ve moved on from that, we still have limitations, such as HTML and CSS having been defined to work only in the sRGB color space. Just like the example of hober‘s shoes above, this means there are many colors that your CSS, images, and canvas are unable to represent. This is a problem if you’re trying to show your family the spring flowers blooming in your garden, or shopping for a bright red sports car to help with your mid-life crisis.

So when we showed the photograph of the shoes above, an sRGB display squished all the colors outside the sRGB gamut into colors that it could show. But a display with a wider gamut such as Display P3 didn’t need to squish into sRGB. Here’s a visual representation showing the difference between sRGB and the Display P3 color space.

<figure class="widescreen mattewhite">sRGB P3 gamut comparison</figure>

The colored triangle is the sRGB space. The white triangle shows the coverage of the Display P3 space, significantly bigger than sRGB, particularly in the more saturated reds, yellows, purples and greens. The black outline shows the ability of the typical human eye.

Remember the red square with the faint WebKit logo? That was generated by creating an image in the Display P3 color space, filling it with 100% red, rgb(255, 0, 0), and then painting the logo in a slightly different red, rgb(241, 0, 0). On an sRGB display, you can’t see the logo, because all the red values above 241 in Display P3 are beyond the highest red in sRGB, so the 241 red and the 255 red end up as the same color.

Note: I saw a bit of confusion about this on Twitter, so let’s try an alternate explanation. Basically, all the fully red values between 241/255 and 255/255 in the Display P3 color space are indistinguishable when shown in sRGB. This doesn’t mean that a 241 red from Display P3 is the same as a 255 red in sRGB – unfortunately, it’s not quite that simple and I don’t want to get too detailed in this introductory article. For those who are interested, on macOS there is an app called Color Sync Utility that allows you to convert between color spaces in different ways, as well as comparing gamuts.

So now you understand why you should be aware of color, and that you should use this technology to give users a better experience. That’s the background — let’s talk about what this means for WebKit.

Color-matching Images

We mentioned above that the Web is defined to use sRGB. WebKit/Safari on Mac has operated in this color space for a number of years now, meaning you’d get consistent colors across different displays (at the time of writing, most other browser engines just operate in what’s called the device color space, which means they don’t process the color values before passing them on to the display hardware).

WebKit color-matches all images on both iOS and macOS. This means that if the image has a color profile, we will make sure the colors in the image are accurately represented on the display, whether it is normal or wide gamut. This is useful since many digital cameras don’t use sRGB in their raw format, so simply interpreting the red, green and blue values as such is unlikely to produce the correct color. Typically, you won’t have to do anything to get this color-matching. Nearly all image processing software allows you to tag an image with a color profile, and many do it by default.

With the examples linked above, you’re seeing this color-matching in action on Safari from OS X 10.11.3 and above, as well as iOS 9.3 and above (Retina devices). All I had to do was make sure that the images had been tagged with the appropriate color profile.

If an image doesn’t have a tagged profile, WebKit assumes that it is sRGB. This makes it easy for your generated artwork, such as border and background images, to match what you have defined in CSS. That is, a CSS value of rgb(255, 0, 0) would match the corresponding full sRGB red value.

Note: This was another point that provoked some comment, with some suggesting an untagged image should not be assumed to be sRGB. The reason we do this is explained above: it’s to make sure that colors in the image match the CSS colors in the page.

That’s ok for the display technology from the last decade. But now that we have displays that support a wider gamut, you’ll want more control over how your content is shown.

Detecting the Display

I’ve explained above why you should prefer to serve wide-gamut images to users on a wide-gamut display. If you serve wide-gamut images to users not on a wide-gamut display, WebKit will color-match the images and show them in the sRGB space. However, this conversion into sRGB can be done a few ways and isn’t guaranteed to happen identically on other browsers or platforms. As a discerning Web developer or designer you will want to convert your images offline to better control what the end user will see. Also, embedding the color profile adds a little to the file size, so you might not want to send that extra data if it isn’t necessary.

The best solution is to serve a wide-gamut image when the user is on a wide-gamut display, and an sRGB image when the user is not on a wide-gamut display. This is just another case of responsive images, and is exactly what the element and media queries were designed to handle.

WebKit now supports the (new to CSS Color Level 4) color-gamut media query. Here’s how you use it:

  <source media="(color-gamut: p3)" srcset="photo-wide.jpg">
  <img src="photo-srgb.jpg">

You can also use this query inside a style sheet:

.main {
  background-image: url("photo-srgb.jpg");

@media (color-gamut: p3) {
  .main {
    background-image: url("photo-wide.jpg");

Or script:

if (window.matchMedia("(color-gamut: p3)").matches) {
  // Do especially colorful stuff here.

The color-gamut query accepts p3 and rec2020 as values, which are intentionally vague terms to specify the range of colors supported by the system, including the browser engine and the display hardware. By default, since nearly all displays support sRGB, there is no need to test for that functionality. But a typical modern wide-gamut display can support the range of colors included in the DCI P3 space, or near about, and would pass the media query. For example, the Display P3 space I mentioned above is a variant of DCI P3. The rec2020 value indicates the system has a display that supports an even wider color gamut, one approximately defined by the Rec. 2020 space (at the moment, it’s quite rare to come across hardware on the Web that actually supports Rec. 2020, so you probably don’t need to worry about that yet).

Since media queries have a graceful fallback, you can start using color-gamut right away to give wide-gamut users nicer colors while still accomodating users who don’t yet have compatible browsers or hardware.

Future Directions

Serving and rendering wide-gamut images is relatively straightforward, but what about if you want to combine it with other page elements like a background color, or paint the image into a canvas element? These are some of the challenges the standards bodies are grappling with, and I’d like to talk about where they’re going.

Wide-gamut colors in CSS

I showed above that rgb(241, 0, 0) in Display P3 is rgb(255, 0, 0) in sRGB. What can you do if you want a color defined in CSS to match something inside a wide-gamut image? We can’t yet specify those colors in CSS.

This is what members of the WebKit project have proposed for CSS. The current idea is to add a new function called color() that can take a color profile as well as the parameters defining the color.

/* NOTE: Proposed syntax. Not yet implemented. */
strong {
  color: color(p3 1.0 0 0); /* 100% red in the P3 color space */

In practice, you’d probably use this with an @supports rule.

strong {
  color: rgb(255, 0, 0); /* 100% red in the sRGB color space */

@supports (color: color(p3 0 0 0)) {
  strong {
    color: color(p3 1.0 0 0); /* 100% red in the P3 color space */
Note: I originally had a bad typo, showing the syntax as color(p3, 255, 0, 0). This is one of the annoyances of the existing rgb() function. The new color() function will take floating point numbers as parameters, not 8-bit integers.

CSS will define some known color profile names, so you can easily find the color values you want. There is still some discussion about allowing authors to link to external profiles, or maybe point at an image that has an embedded profile.

In addition, CSS might decide to allow you to specify values outside 0-255 (or 0-100%) in the existing rgb() function. For example, rgb(102.34%, -0.1%, 4%) would be a color that has more red, less green and a tiny touch of blue. The problem here is that it might be confusing to understand these values (what does it mean to be a negative green?).

Another suggestion is to be able to define a color space for the entire document, which would mean that all the regular CSS color values would be interpreted in that space. External images with embedded profiles would still be color matched.

These discussions are happening in the W3C CSS Working Group at the moment. Your opinions are valuable – the group needs more input from Web developers. If you’re interested in participating, look for emails with subjects starting “[css-color]” on the www-style email list.

WebKit hopes to implement these features as soon as we’re confident they are stable.

Wide-gamut colors in HTML

While CSS handles most of the presentation of an HTML document, there is still one important area which is outside its scope: the canvas element. Both 2D and WebGL canvases assume they operate within the sRGB color space. This means that even on wide-gamut displays, you won’t be able to create a canvas that exercises the full range of color.

The proposed solution is to add an optional flag to the getContext function, specifying the color space the canvas should be color matched to. For example:

// NOTE: Proposed syntax. Not yet implemented.
canvas.getContext("2d", { colorSpace: "p3" });

There is something else to consider, which is how to create a canvas that has a greater color depth. For example, in WebGL you can use half-float textures that give you 16 bits of precision per color channel. However, even if you use these deeper textures inside WebGL, you’re limited to 8 bits of precision when you paint the WebGL into the document.

There needs to be a way for the developer to specify the depth of the color buffer for a canvas element.

This gets more complicated when combined with the getImageData/putImageData functions (or the readPixels equivalent in WebGL). With today’s 8 bit per channel buffers, there is no loss of precision going into and out of a canvas. Also, the transfer can be efficient, both in performance and memory, because the canvas and program side data are the same type. But when you have variable color depths, this might not be possible. For example, WebGL’s half float buffer doesn’t have an equivalent type in JavaScript, which means there will either have to be some transformation as the data is read or written, as well as use more memory to store the data, or you’ll have to work with a raw ArrayBuffer and do awkward math with bit masks.

These discussions are currently happening on the WhatWG github and soon at the W3C. Again, we’d love for you to be part of the conversation.


Wide-gamut displays have arrived and are the future of computing devices. As your audience using these beautiful displays grows, you’ll want to take advantage of the stunning range of colors on offer and provide users with a more engaging Web experience.

WebKit is very excited to provide improved color features to developers through color matching and color gamut detection, available today in the Safari Technology Preview, as well as the macOS Sierra and iOS 10 betas. We’re also keen to start implementing more advanced color features, such as specifying wide-gamut colors in CSS, tagging canvas elements with profiles and allowing greater color depth.

If you have any comments, questions, delicious cookies or spam I can be found at or @grorgwork. You can also contact Jonathan Davis at, @jonathandavis, or send general comments to the @webkit Twitter account.

W3C Team blogPerspectives on security research, consensus and W3C Process

Linux Weekly News published a recent story called “Encrypted Media Extensions and exit conditions”, Cory Doctorow followed by publishing “W3C DRM working group chairman vetoes work on protecting security researchers and competition”. While the former is a more accurate account of the status, we feel obligated to offer corrections and clarifications to the latter, and to share a different perspective on security research protection, consensus at W3C, W3C’s mission and the W3C Process, as well as the proposed Technology and Policy Interest Group.

There have been a number articles and blog posts about the W3C EME work but we’ve not been able to offer counterpoints to every public post, as we’re focusing on shepherding and promoting the work of 40 Working Groups and 14 Interest Groups –all working on technologies important to the Web such as: HTML5, Web Security, Web Accessibility, Web Payments, Web of Things, Automotive, etc.

TAG statement on the Web’s security model

In his recent article, Cory wrote:

For a year or so, I’ve been working with the EFF to get the World Wide Web Consortium to take steps to protect security researchers and new market-entrants who run up against the DRM standard they’re incorporating into HTML5, the next version of the key web standard.

First, the W3C is concerned about risks for security researchers. In November 2015 the W3C Technical Architecture Group (TAG), a special group within the W3C, chartered under the W3C Process with stewardship of the Web architecture, made a statement (after discussions with Cory on this topic) about the importance of security research. The TAG statement was:

The Web has been built through iteration and collaboration, and enjoys strong security because so many people are able to continually test and review its designs and implementations. As the Web gains interfaces to new device capabilities, we rely even more on broad participation, testing, and audit to keep users safe and the web’s security model intact. Therefore, W3C policy should assure that such broad testing and audit continues to be possible, as it is necessary to keep both design and implementation quality high.

W3C TAG statements have policy weight. The TAG is co-Chaired by the inventor of the Web and Director of W3C, Tim Berners-Lee. It has elected representatives from W3C members such as Google, Mozilla, Microsoft and others.

This TAG statement was reiterated in an EME Factsheet, published before the W3C Advisory Committee meeting in March 2016 as well as in the W3C blog post in April 2016 published when the EME work was allowed to continue.

Second, EME is not a DRM standard. W3C does not make DRM. The specification does not define a content protection or Digital Rights Management system. Rather, EME defines a common API that may be used to discover, select and interact with such systems as well as with simpler content encryption systems. We appreciate that to those who are opposed to DRM, any system which “touches” upon DRM is to be avoided, but the distinction is important. DRM is on the Web and has been for many years. We ask pragmatically what we can do for the good of the Web to both make sure a system which uses protected content insulates users as much as possible, and ensure that the work is done in an open, transparent and accessible way.

A several-month TF to assess EFF’s proposed covenant

Cory further wrote, about the covenant:

As a compromise that lets the W3C continue the work without risking future web users and companies, we’ve proposed that the W3C members involved should agree on a mutually acceptable binding promise not to use the DMCA and laws like it to shut down these legitimate activities — they could still use it in cases of copyright infringement, just not to shut down activity that’s otherwise legal.

The W3C took the EFF covenant proposal extremely seriously. Made as part of EFF’s formal objection to the Working Group’s charter extension, the W3C leadership took extraordinary effort to resolve the objection and evaluate the EFF proposed covenant by convening a several month task force. Hundreds of emails were exchanged between W3C Members and presentations were made to the W3C Advisory Committee at the March 2016 Advisory Committee meeting.

While there was some support for the idea of the proposal, the large majority of W3C Members did not wish to accept the covenant as written (the version they voted on was different from the version the EFF made public), nor a slightly different version proposed by another member.

Member confidentiality vs. transparent W3C Process

Cory continued:

The LWN writeup is an excellent summary of the events so far, but parts of the story can’t be told because they took place in “member-confidential” discussions at the W3C. I’ve tried to make EFF’s contributions to this discussion as public as possible in order to bring some transparency to the process, but alas the rest of the discussion is not visible to the public.

W3C works in a uniquely transparent way. Specifications are largely developed in public and most groups have public minutes and mailings lists. However, Member confidentiality is a very valuable part of the W3C process. That business and technical discussions can happen in confidence between members is invaluable to foster broader discussion, trust and the opportunity to be frank. The proceedings of the HTML Media Extensions work are public however, discussions amongst Advisory Committee members are confidential.

In his post, Nathan Willis quoted a June 6 blog post by EFF’s Cory Doctorow, and continued:

Enough W3C members endorsed the proposed change that the charter could not be renewed. After 90 days’ worth of discussion, the working group had made significant progress, but had not reached consensus. The W3C executive ended this process and renewed the working group’s charter until September.

Similar wording is found in an April EFF blog post, attributing the renewal to “the executive of the W3C.” In both instances, the phrasing may suggest that there was considerable internal debate in the lead-up to the meeting and that the final call was made by W3C leadership. But, it seems, the ultimate decision-making mechanism (such as who at W3C made the final decision and on what date) is confidential; when reached for comment, Doctorow said he could not disclose the process.

Though the Member discussions are confidential, the process itself is not.

In the W3C process, charters for Working Groups go to the Advisory Committee for review at different stages of completion. That happened in this case. The EFF made an objection. By process, when there are formal objections the W3C then tries to resolve the issue.

As part of the process, when there is no consensus, the W3C generally allows existing groups to continue their work as described in the charter. When there is a “tie-break” needed, it is the role of the Director, Tim Berners-Lee, to assess consensus and decide on the outcome of formal objections. It was only after the overwhelming majority of participants rejected the EFF proposal for a covenant attached to the EME work that Tim Berners-Lee and the W3C management felt that the EFF proposal could not proceed and the work would be allowed to continue.

Next steps within the HTML Media Extensions Working Group

Cory also wrote:

The group’s charter is up for renewal in September, and many W3C members have agreed to file formal objections to its renewal unless some protection is in place. I’ll be making an announcement shortly about those members and suggesting some paths for resolving the deadlock.

The group is not up for charter renewal in September but rather, its specifications are progressing on the time-line to “Recommendation“. A Candidate Recommendation transition will soon have to be approved, and then the spec will require interoperability testing, and Advisory Committee approval before it reaches REC. One criteria for Recommendation is that the ideas in the technical report are appropriate for widespread deployment and EME is already deployed in almost all browsers.

To a lesser extent, we wish to clarify that veto is not part of the role of Working Group chairs; indeed Cory wrote:

Linux Weekly News reports on the latest turn of events: I proposed that the group take up the discussion before moving to recommendation, and the chairman of the working group, Microsoft’s Paul Cotton, refused to consider it, writing, “Discussing such a proposed covenant is NOT in the scope of the current HTML Media Extensions WG charter.”

As Chair of the HTML Media Extensions Working Group, Paul Cotton’s primary role is to facilitate consensus-building among Group members for issues related to the specification. A W3C Chair leads the work of the group but does not decide for the group; work proceeds with consensus. The covenant proposal had been under wide review with many lengthy discussions for several months on the W3C Advisory Committee mailing lists. Paul did not dismiss W3C-wide discussion of the topic, but correctly noted it was not a topic in line with the chartered work of the group.


In the April 2016 announcement that the EME work would continue, the W3C reiterated the importance of security research and acknowledged the need for high level technical policy discussions at W3C – not just for the covenant. A few weeks prior, during the March 2016 Advisory Committee meeting the W3C announced a proposal to form a Technology and Policy Interest Group.

The W3C has, for more than 20 years, focused on technology standards for the Web. However, recognizing that as the Web gets more complex and its technology is increasingly woven into our lives, we must consider technical aspects of policy as well. The proposed Technology and Policy Interest Group, if started, will explore, discuss and clarify aspects of policy that may affect the mission of W3C to lead the Web to its full potential. This group has been in preparation before the EME covenant was presented, and will be address broader issues than anti-circumvention. It is designed as a forum for W3C Members to try to reach consensus on the descriptions of varying views on policy issues, such deep linking or pervasive monitoring.

While we tried to find common ground among our membership on the covenant issue, we have not succeeded yet. We hope that EFF and others will continue to try. We recognize and support the importance of security research, and the impact of policy on innovation, competition and the future of the Web. Again, for more ample information on EME and frequently asked questions, please see the EME Factsheet, published in March 2016.

Jeremy KeithCarolyn

At An Event Apart in Boston, I had the pleasure of meeting Hannah Birch from Pro Publica. It turns out that she was a copy editor in a previous life. I began gushing about the pleasure of working with a great editor.

I’ve been lucky enough to work with some of the best. Working with Mandy on HTML5 For Web Designers was wonderful. One of these days I hope to work with Owen Gregory.

When I think back on happy memories of working with world-class editors, I always a remember a Skype call about an article I was writing for The Manual. I talked with my editor for hours about the finer points of wordsmithery, completely losing track of time. It was a real joy. That editor was Carolyn Wood.

Carolyn is going through a bad time right now. A really bad time. A combination of awful medical problems combined with a Kafkaesque labyrinth of health insurance have combined to create a perfect shitstorm. I feel angry, sad, and helpless. At least I can do something about that last part. And you can too.

If you’d like to help, Karen has set up a page for contributing to help Carolyn. If you could throw a few bucks in there, I would appreciate it very much. Thank you.

Planet MozillaMozilla Awards $385,000 to Open Source Projects as part of MOSS “Mission Partners” Program


For many years people with visual impairments and the legally blind have paid a steep price to access the Web on Windows-based computers. The market-leading software for screen readers costs well over $1,000. The high price is a considerable obstacle to keeping the Web open and accessible to all. The NVDA Project has developed an open source screen reader that is free to download and to use, and which works well with Firefox. NVDA aligns with one of the Mozilla Manifesto’s principles: “The Internet is a global public resource that must remain open and accessible.”

That’s why, at Mozilla, we have elected to give the project $15,000 in the inaugural round of our Mozilla Open Source Support (MOSS) “Mission Partners” awards. The award will help NVDA stay compatible with the Firefox browser and support a long-term relationship between our two organizations. NVDA is just one of eight grantees in a wide range of key disciplines and technology areas that we have chosen to support as part of the MOSS Mission Partners track. This track financially supports open source software projects doing work that meaningfully advances Mozilla’s mission and priorities.

Giving Money for Open Source Accessibility, Privacy, Security and More

Aside from accessibility, security and privacy are common themes in this set of awards. We are supporting several secure communications tools, a web server which only works in secure mode, and a distributed, client-side, privacy-respecting search engine. The set is rounded out with awards to support the growing Rust ecosystem and promote open source options for the building of compelling games on the Web. (Yes, games. We consider games to be a key art-form in this modern era, which is why we are investing in the future of Web games with WebAssembly and Open Web Games.)

MOSS is a continuing program. The Mission Partners track has a budget for 2016 of around US$1.25 million. The first set of awards listed below total US$385,000 and we look forward to supporting more projects in the coming months. Applications remain open both for Mission Partners and for the Foundational Technology track (for projects creating software that Mozilla already uses or deploys) on an ongoing basis.

We are greatly helped in evaluating applications and making awards by the MOSS Committee. Many thanks again to them.

And The Winners Are….

The first eight awardees are:

Tor: $152,500. Tor is a system for using a distributed network to communicate anonymously and without being tracked. This award will be used to significantly enhance the Tor network’s metrics infrastructure so that the performance and stability of the network can be monitored and improvements made as appropriate.

Tails: $77,000. Tails is a secure-by-default live operating system that aims at preserving the user’s privacy and anonymity. This award will be used to implement reproducible builds, making it possible for third parties to independently verify that a Tails ISO image was built from the corresponding Tails source code.


Caddy: $50,000. Caddy is an HTTP/2 web server that uses HTTPS automatically and by default via Let’s Encrypt. This award will be used to add a REST API, web UI, and new documentation, all of which make it easier to deploy more services with TLS.

Mio: $30,000. Mio is an asynchronous I/O library written in Rust. This award will be used to make ergonomic improvements to the API and thereby make it easier to build high performance applications with Mio in Rust.


DNSSEC/DANE Chain Stapling: $25,000. This project is standardizing and implementing a new TLS extension for transport of a serialized DNSSEC record set, to reduce the latency associated with DANE and DNSSEC validation. This award will be used to complete the standard in the IETF and build both a client-side and a server-side implementation.


Godot Engine: $20,000. Godot is a high-performance multi-platform game engine which can deploy to HTML5. This award will be used to add support for Web Sockets, WebAssembly and WebGL 2.0.


PeARS: $15,500. PeARS (Peer-to-peer Agent for Reciprocated Search) is a lightweight, distributed web search engine which runs in an individual’s browser and indexes the pages they visit in a privacy-respecting way. This award will permit face-to-face collaboration among the remote team and bring the software to beta status.


NVDA: $15,000. NonVisual Desktop Access (NVDA) is a free, open source screen reader for Microsoft Windows. This award will be used to make sure NVDA and Firefox continue to work well together as Firefox moves to a multi-process architecture.

This is only the beginning. Stay tuned for more award announcements as we allocate funds. Open Source is a movement that is only growing, both in numbers and in importance. Operating in the open makes for better security, better accessibility, better policy, better code and, ultimately, a better world. So if you know any projects whose work furthers the Mozilla Mission, send them our way and encourage them to apply.

Planet WebKitNext Steps for Legacy Plug-ins

The web platform is capable of amazing things. Thanks to the ongoing hard work of standards bodies, browser vendors, and web developers, web standards are feature-rich and continuously improving. The WebKit project in particular emphasizes security, performance, and battery life when evaluating and implementing web standards. These standards now include most of the functionality needed to support rich media and interactive experiences that used to require legacy plug-ins like Adobe Flash. When Safari 10 ships this fall, by default, Safari will behave as though common legacy plug-ins on users’ Macs are not installed.

On websites that offer both Flash and HTML5 implementations of content, Safari users will now always experience the modern HTML5 implementation, delivering improved performance and battery life. This policy and its benefits apply equally to all websites; Safari has no built-in list of exceptions. If a website really does require a legacy plug-in, users can explicitly activate it on that website.

If you’re a web developer, you should be aware of how this change will affect your users’ experiences if parts of your websites rely on legacy plug-ins. The rest of this post explains the implementation of this policy and touches on ways to reduce a website’s dependence on legacy plug-ins.

How This Works

By default, Safari no longer tells websites that common plug-ins are installed. It does this by not including information about Flash, Java, Silverlight, and QuickTime in navigator.plugins and navigator.mimeTypes. This convinces websites with both plug-in and HTML5-based media implementations to use their HTML5 implementation.

Of these plug-ins, the most widely-used is Flash. Most websites that detect that Flash isn’t available, but don’t have an HTML5 fallback, display a “Flash isn’t installed” message with a link to download Flash from Adobe. If a user clicks on one of those links, Safari will inform them that the plug-in is already installed and offer to activate it just one time or every time the website is visited. The default option is to activate it only once. We have similar handling for the other common plug-ins.

When a website directly embeds a visible plug-in object, Safari instead presents a placeholder element with a “Click to use” button. When that’s clicked, Safari offers the user the options of activating the plug-in just one time or every time the user visits that website. Here too, the default option is to activate the plug-in only once.

<figure class="widescreen mattewhite" style="padding-top: 0; padding-bottom: 0;">Sheet to turn a plug-in on in Safari 10</figure>

Safari 10 also includes a menu command to reload a page with installed plug-ins activated; it’s in Safari’s View menu and the contextual menu for the Smart Search Field’s reload button. All of the settings controlling what plug-ins are visible to web pages and which ones are automatically activated can be found in Safari’s Security preferences.

Whenever a user enables a plug-in on a website, it’ll remain enabled as long as the user regularly visits the website and and website still uses the plug-in. More specifically, Safari expires a user’s request to activate a plug-in on a particular website after it hasn’t seen that plug-in used on that site for a little over a month.

Recommendations for Web Developers

Before Safari 10 is released this fall, we encourage you to test how these changes impact your websites. You can do that by installing a beta of macOS Sierra. There will be betas of Safari 10 for OS X Yosemite and OS X El Capitan later this summer.

To avoid making your users have to explicitly activate a plug-in on your website, you should try to implement features using technologies built into the web platform. You can use HTML5 <audio>, <video>, the Audio Context API, and Media Source Extensions to implement robust, secure, customized media players. New in Safari 10, text can be cut or copied to the clipboard using execCommand, which was previously only possible using a plug-in. A host of CSS features, including animations, backdrop filters, and font feature settings can add some visual polish to a site. And WebGL is great for creating interactive 2D or 3D content, like games.

If you serve a different version of your website to mobile browsers, it may already implement its media playback features using web standards. As browsers continue to transition away from legacy plug-ins, you can preserve the rest of your users’ experiences by serving those same implementations to all visitors of your site.

If you can’t replace a plug-in-based system in the short term, you may want to teach your users how to enable that plug-in for your website in Safari. In an enterprise setting, system administrators can deploy managed policies to enable a plug-in on specific websites, if necessary.

Help Us Help You

If you find that you can’t implement parts of your websites without using legacy plug-ins, you can help yourself and other developers by telling us about it. In general, any time the web platform falls short of your needs, we want to know about it. Your feedback has and will continue to shape the priorities of the WebKit project and the Safari team. To send that type of feedback, please write email to or tweet at Jonathan Davis.

And if you have questions about Safari’s policies for using Flash or other plug-ins, feel free to reach me on Twitter at @rmondello.

Steve Faulkner et alUsing the HTML lang attribute

What do an American actor, a British sitcom character and an HTML attribute have in common? If you’ve ever watched Mary Poppins and winced at Dick Van Dyke’s attempt at an English accent, or found yourself laughing at Delboy Trotter trying to speak French in Only Fools and Horses, you may well guess the answer.

The HTML lang attribute is used to identify the language of text content on the web. This information helps search engines return language specific results, and it is also used by screen readers that switch language profiles to provide the correct accent and pronunciation.

To set the primary language for a document, you use the lang attribute on the <html> element:

<html lang="en">

The lang attribute takes an ISO language code as its value. Typically this is a two letter code such as “en” for English, but it can also be an extended code such as “en-gb” for British English.

The lang attribute must also be used to identify chunks of text in a language that is different from the document’s primary language. For example:

<html lang="en">
<p>This page is written in English.</p>
<p lang="fr">Sauf pour ce qui est écrit en mauvais français.</p>

The lang attribute is forgotten surprisingly often, perhaps because it makes no apparent difference unless you use a screen reader or you are a search engine. If you’re in any doubt at all what a difference it makes though, listen to this screen reader demo!


Updated: .  Michael(tm) Smith <>