Planet MozillaDisappointed in @W3C for Recommending Longdesc

W3C has advanced the longdesc attribute to a Recommendation, overruling objections from browser makers.

Not a single browser vendor supported advancing this specification to recommendation.

Apple formally objected when it was a Candidate Recommendation and provided lengthy research and documentation (better than anyone has before or since) on why longdesc is bad technology (in practice has not and does not solve the problems it claims to).

Mozilla formally objected when it was a Proposed Recommendation, agreeing with Apple’s research and reasoning.

Both formal objections were overruled.

For all the detailed reasons noted in Apple’s formal objection, I also recommend avoid using longdesc, and instead:

  • Always provide good alt (text alternative) attributes for images, that read well inline if and when the image does not load. Or if there’s no semantic loss without the image, use an empty alt="".
  • For particularly rich or complex images, either provide longer descriptions of images in normal visible markup, or linked from a image caption or other visible affordance. See accessibility expert James Craig’s excellent Longdesc alternatives in HTML5 resource for even more and better techniques.

Perhaps the real tragedy is that many years have been wasted on a broken technology that could have been spent on actually improving accessibility of open web technologies. Not to mention the harrassment that’s occurred in the name of longdesc.

Sometimes web standards go wrong. This is one of those times.

IEBlogA break from the past: the birth of Microsoft's new web rendering engine

As we announced last month, Project Spartan will be the new browser across all Windows 10 devices, from phones to tablets, PCs and beyond. You’ll hear about the new browser’s features in the coming months but in this post, we want to tell you more about what motivated us to build a new rendering engine focused on interoperability with other modern browsers ― all in the name of making the Web “just work” for our customers. This new rendering engine was designed with Project Spartan in mind, but will also be available in Internet Explorer on Windows 10 for enterprises and other customers who require legacy extensibility support.

<iframe frameborder="0" height="315" src="https://www.youtube.com/embed/iY6Z42YFq-4" width="560"></iframe>

Modernizing IE while not breaking the Web

Internet Explorer has been the Microsoft Web browser for 20 years and has evolved to work with ever-changing Web sites over that time. Since 2009, we’ve been systematically bringing major investments to the browser to modernize each of the browser subsystems: from fast new JavaScript and layout engines, GPU-accelerated rendering and 3D graphics, and multi-threaded touch input to improved F12 developer tools and backwards compatibility with Enterprise Mode.

Big changes in 5 years of IE
Big changes in 5 years: Fish IE (GPU rendering), Chakra fastest on SunSpider (JavaScript performance), Touch Effects (multi-touch input), new F12 developer tools, Assassin’s Creed Pirates (WebGL)

As we’ve been rolling out these significant changes with major versions of IE, we also did our best to abide by the mantra of “don’t break the Web.” This is an ambitious goal given that the Web consists of over 44 billion Web sites, so we prioritized compatibility testing on the Web’s top 9000 Web sites globally which accounts for roughly 88% of Web traffic. This represented the sweet spot of where the “head of the Web” becomes the “long tail” – allowing us to focus our testing resources on the most impactful sites. Prior to release, we would ensure a compatibility pass rate greater than previous releases of IE and our competitors on those top 9000 sites.

The long tail matters

And yet, even as we released new and improved versions of IE, we heard complaints about some sites being broken in IE – from family members, co-workers in other parts of Microsoft, or online discussions. As we examined these sites, they were often not on our top 9000 list. They were issues like not being able to complete a reservation online for the barbershop down the street, or not being able to log in to see the schedule for the local kids soccer league.

In Windows 10 planning, we set out to tackle this apparent discrepancy – how could our real-world compatibility be declining when our compatibility testing data shows that we are improving?

Rethinking our assumptions

As we dug in, we uncovered a series of issues that led us to realize that we needed to significantly rethink our approach to compatibility with the Web at large:

  • Legacy vs. modern. While we were pushing ahead with new HTML5 features, we were also expected to preserve compatibility with old versions of IE, particularly for enterprise Web apps. Limited compatibility was provided through document compatibility modes within Trident, however compatibility could not be guaranteed and it provided consistent obstacles towards fixing long-standing IE-specific behaviors. Furthermore, fixing long standing interoperability bugs with other modern browsers could actually break sites who have coded to the IE-specific behavior.
  • CV list. Our compatibility pass rates were dependent on the presence of the compatibility view list. This allowed us to “fix” broken sites by forcing them into old document modes which emulated legacy IE behaviors. However, this approach requires testing and maintenance, and doesn’t scale well beyond the top sites.
  • X-UA-Compatible. Some sites forced an older document mode using the “x-ua-compatible” header. However, rather than using it as a temporary stopgap, they would rely upon that to keep that version of the site working in future versions of IE while they developed an evergreen code path of their site for other modern browsers.
  • Standards focus. Our focus on building new HTML5 features was to comply with Web standards, which in turn should lead to interoperability among browsers. However, interpretations of the standards document could easily vary, leading to real-world interoperability gaps and ultimately more bug fixing for Web developers and more broken sites for customers.

Project Spartan engineers at work

A break from the past

Faced with these issues, we were inspired to have “courage in the face of reality”. We needed a plan to make it easy for Web developers to build compatible sites regardless of which browser they develop first for. We needed a plan which ensured that our customers have a good experience regardless of whether they browse the head or tail of the Web. We needed a plan which gave enterprise customers a highly backward compatible browser regardless of how quickly we pushed forward with modern HTML5 features.

In order to really address these challenges, we realized that we couldn’t just incrementally improve on our previous approach, we needed a break from the past – all without losing the major investments that we had been making since 2009.

A pragmatic approach

The break meant bringing up a new Web rendering engine, free from 20 years of Internet Explorer legacy, which has real-world interoperability with other modern browsers as its primary focus – and thus our rallying cry for Windows 10 became “the Web just works.” This pragmatic viewpoint, which was initially proven out by our work in Windows Phone 8.1 Update, meant that Web standards would continue to be important but should function in the background to drive real-world interoperability between browsers.

This interoperability-focused approach brought the obvious question of adopting an existing open-source rendering engine such as WebKit. While there were some advantages, upon further investigation it was not the right path forward for two important reasons. First, the Web is built on the principle of multiple independent, yet interoperable implementations of Web standards and we felt it was important to counter movement towards a monoculture on the Web. Second, given the engineering effort required, we found that we could deliver an interoperability focused engine to customers significantly faster if we started from our own engine (especially if unshackled from legacy compatibility concerns), rather than building up a new browser around an open-source engine. We will continue to look at open source and shared source models where it makes sense and look forward to sharing more details in coming posts.

A new Web rendering engine is born

As detailed in Jacob Rossi’s article for Smashing Magazine, the new engine began as a fork of MSHTML.dll but has since diverged very quickly. By making this split, we were able to keep the major subsystem investments made over the last several years, while allowing us to remove document modes and other legacy IE behaviors from the new engine. On the other hand, our legacy engine (MSHTML.dll) can remain largely unchanged (outside of security and other high priority fixes) to help guarantee legacy compatibility for our enterprise customers. We also built up capabilities to switch between the legacy and new rendering engines seamlessly.

A clean break also necessitates a new user-agent string to ensure that no IE-specific code was being sent. This built upon a long browser history of using whatever tokens are necessary to get the desired content from servers. Although this meant a lower compatibility rate initially, it was also useful to reveal interoperability issues that needed to be fixed!

Finally, since legacy IE code is no longer being sent, we are able to drastically reduce our reliance on the CV list. This means that our top site compatibility rate will match our long tail compatibility much more closely.

Fixing patterns instead of sites

However, a new engine was not enough – we also needed to significantly revamp how we find, track and fix issues on the long tail of the Web. To do so, we do daily analysis on trillions of URLs crawled in conjunction with Bing to detect patterns that exist in the head of the Web and the tail of the Web. By fixing these patterns, sites just end up working. This data is augmented by thousands of daily feedback reports from users via the “smiley face” icon.

In addition, we revised our internal engineering processes to prioritize real-world interoperability issues uncovered by our data analysis. With these processes in place, we set about fixing over 3000 interoperability bugs and adding over 40 new Web standards (to date) to make sure we deliver on our goals.

We don’t see this interoperability effort having an end date – we’ll be continuously checking the data and rolling out improvements to the new rendering engine. For users that upgrade to Windows 10, the engine will be evergreen, meaning that it will be kept current with Windows 10 as a service.

Project Spartan engineers at work

A community effort

Our mission to create a Web that “just works” won’t be successful without your help. That’s why the Project Spartan journey has included a number of ways for you to get involved. Here’s a quick rundown of ways you can help:

  • Join the Windows Insider Program to get the latest Windows 10 previews and test your Web sites on our new rendering engine by enabling experimental Web features in the about:flags page. If you don’t have a device to install it on, try out RemoteIE which will stream our latest browser from the Azure cloud to Windows, iOS or Android devices.
  • If you see any site issues, “Send a Frown” using the smiley icon so we can track it. If you have a more detailed description of the issue, you can open a bug on Connect.
  • Check out our Web platform roadmap at status.modern.ie. If anything is missing, submit a feature request or vote for an existing request at our UserVoice site.
  • Send us a message on Twitter @IEDevChat, or join a monthly #AskIE tweetchat (next one on 2/26 @ 12PM PST). For help on more detailed issues, ask a question on our StackOverflow page.

We believe the break from IE’s past we’ve made to create a new rendering engine will help make the browsing experience for our customers better, and make building Web sites that just work across browsers easier for Web developers. We’ll be sharing more details on specific aspects of the approach outlined above in the coming weeks and look forward to your continued participation in this journey with us.

Charles Morris, Program Manager Lead, Project Spartan

Planet MozillaThis Week In Servo 25

This week, we merged 60 pull requests.

Selector matching has been extracted into an independent library! You can see it here.

We now have automation set up for our Gonk (Firefox OS) port, and gate on it.

Notable additions

Screenshots

Parallel painting, visualized:

New contributors

Meeting

Minutes

  • Rust in Gecko: We’re slowly figuring out what we need to do (and have a tracking bug). Some more candidates for a Rust component in Gecko are an MP4 demultiplexer, and a replacement of safe browsing code.
  • Selector matching is now in a new library
  • ./mach build-gonk works. At the moment it needs a build of B2G, which is huge (and requires a device), but we’re working on packaging up the necessary bits which are relatively small.

IEBlogPointer Events W3C Recommendation, Interoperable Touch, and Removing the Dreaded 300ms Tap Delay

Today, the W3C published Pointer Events as a final Recommendation standard. This journey began a little over 2 years ago when we first submitted our proposal to the W3C for a common event model across pointing input devices. Since then, the spec has evolved and improved with the collaboration of Mozilla, Google, Opera, jQuery, IBM, Microsoft Open Technologies, and many others. Improvements have included better extensibility for new device types, additional configuration values for scrolling and zooming behavior, event constructors for synthesizing custom pointer events, and more.

We updated and unprefixed our implementation in IE11. With the addition of DOM event constructors in the new rendering engine in the Windows 10 Tech Preview, our implementation is now 100% compliant with the W3C test suite. We’re also excited to see Firefox builds with 100% pass rate now with work underway to ship this to users soon. Additionally, jQuery, Dojo, and other open source contributors are now maintaining a faithful polyfill called “PEP” and intend to use this in jQuery UI, jQuery Mobile, and Dojo. With the polyfill, it’s possible to code to pointer events across browsers today.

We also continue to see great interest from Web developers lobbying for other browsers to implement (Pointer Events has more stars on Chrome’s issue tracker than 99.6% of all other tickets, open or closed). So we’re hopeful that other browsers will join us in advancing interoperability here.

Beyond Pointer Events, we’re also getting ready to update our new rendering engine with additional improvements for touch interoperability. Here’s a few of the changes you can anticipate and what you can do to take advantage of them.

Improving Touch Performance by Removing the 300ms Tap Delay

When modern browsers first came to mobile devices, there was an interesting problem. How do you make sites designed for a big desktop monitor work on a tiny screen? As a result, mobile browsers scaled Web pages in a (complicated) way to make it fit better. The double-tap gesture was then introduced to quickly zoom in and out of relevant content.

This solution provided a reasonable experience for sites that aren’t optimized for mobile. However, it comes with a tradeoff. From the perspective of a gesture recognizer, any single tap could be followed by another making it a double-tap instead. So before committing to the action of a tap (e.g. navigating a link or clicking a button), the browser must pause for a moment to see if another tap is right behind it. This introduces a noticeable delay (300ms) when you’re just tapping to activate something. This problem has been well documented and many fixes or workarounds have been proposed, including the popular FastClick library and others.

Starting in IE10 (and now in Chrome, coming soon to Firefox), sites could disable double-tap zoom with the CSS touch-action property from Pointer Events:

.disable300msDelay {
touch-action: manipulation;
}

Additionally in IE11, if you disable zoom altogether in your viewport rule (e.g. user-scalable: no) then this also disables the 300ms delay.

Now in the new engine, setting a viewport rule with a width less than or equal to the device-width (a strong signal that you’re not a desktop site we need to optimize for small screens) will also disable the 300ms delay. This results in significant performance wins for mobile sites.

Touch Interop Improvements

We recognize that not all browsers support Pointer Events, and so for interoperability we added Touch Events to Windows Phone 8.1 Update. In a future update to Windows 10, we’ll be improving the support in a few ways based on site compatibility and defacto behavior. We’ll also provide toggles in about:flags to enable/disable these changes to aid developers with testing.

Remove MSPointer events – we deprecated the experimental vendor-prefixed version of pointer events, MSPointer, in IE11. Now that unprefixed pointer events have been shipping for a while, we’re ready to remove MSPointer from the platform. Sites should update to use the standardized unprefixed events.

Fire touchmove during scroll – in Windows Phone 8.1, IE11 would fire a touchcancel event whenever a pan gesture was detected and scrolling begins (ending the event stream for that touch). Chrome recently changed to closer match Safari and we’re joining them. Now, touchmove events will continue to fire (asynchronously) during a scroll.

Simplify mouse events for touch – all modern touch browsers synthesize mouse events for touch in order to be more compatible with sites designed only with mouse in mind. IE previously used a more advanced algorithm for generating these events. However, oddly enough, many sites that use touch events also use mouse events and expect those to follow the webkit model. In this new model, only a tap gesture will produce mouse events (and the events produced are a predefined sequence that all fire at once: mousemove, mousedown, mouseup, click).

Support Touch Events for more devices – we’ve talked before about the compatibility issues resulting from supporting touch events on devices with a mouse. We’ve been working on outreach to sites to be more compatible with browsers supporting touch events on these types of devices. As a step towards touch events on all devices (regardless of input capabilities), we’re rolling out behavior where touch events are enabled on any device (desktop, tablet, or phone) that has a touch screen, which is how Chrome works today. In about:flags, you’ll be able to override this for testing and enable/disable touch events on any device.

There's a lot of positive changes here and we’d love your help in testing them. Look for these changes and more in updates to the Windows 10 Technical Preview and RemoteIE soon. We look forward love to hearing your feedback via the Internet Explorer Platform Suggestion Box on UserVoice, @IEDevChat on Twitter, and in the comments below.

Jacob Rossi, Senior Program Manager, Project Spartan

 

Update (2/25/2015) - Google has closed public comments on their Pointer Events issue tracker (however, you can still star the issue).  We've reached out to Google for their preferred forum for web developers to provide feedback and we'll update this post when that becomes available.

Update 2 (2/25/2015) - Google has reopened comments 

Planet MozillaUnreal Engine 4.7 Binary Release Includes HTML5 Export

With the launch of Unreal Engine 4.7, Epic Games has added the ability to export to HTML5 as a regular part of their Windows binary engine releases. One of the world’s most advanced game engines is one step closer to delivering truly amazing content right in your browser. With this addition, developers will be able to create content in Unreal Engine 4.7, immediately compile it to the Web, and launch it in their browser of choice with the click of a button.

InstructionEpic Games’ work with WebGL and HTML5 has been part of the Unreal Engine 4 source code distribution for many months and has been maturing over the past year. While still a pre-release version, the HTML5 output is robust enough for developers to use it with their content and give feedback on the technology. Mozilla is excited to support Epic Games in their continuing effort to bring this amazing engine to the Web.

FF

Screenshot of output from UE4

In the leadup to GDC, Mozilla will be publishing a series of articles about different aspects of the technology that makes it possible to bring native engines to the Web. Mozilla will also be showcasing several next generation Web technologies at our booth including WebVR demos built in Unreal Engine 4. Output from the engine will also be used to showcase Firefox Developer Tools and demonstrate how they can be leveraged with this type of content.

Mozilla will be taking part in Epic’s HTML5 export presentation which will be broadcast live on Twitch at 2pm PT Thursday, March 5th, and viewed at www.twitch.tv/unrealengine.

For more information on this news from Epic Games, visit their blog.

Come take a look at where the Web is heading at the Firefox Booth (South Hall Booth #2110) or learn more about Unreal Engine 4.7 at Epic Games’ Booth (South Hall Booth #1024).

Anne van KesterenSame-origin policy

The same-origin policy, sometimes referred to as SOP, is the foundation of the web platform’s somewhat flawed security model. Without a browser, https://untrusted.example/ (Untrusted) can access any number of servers through curl. It cannot however access any servers located behind a firewall. With a browser, Untrusted can fetch resources from servers accessible to the user visiting Untrusted. Therefore, when a browser is involved the reach of Untrusted is what Untrusted can reach through curl plus with what the user can reach through curl. SOP prevents Untrusted from accessing the contents of resources on https://intranet.local/ (Intranet).

SOP also protects the contents of resources that depend on HTTP cookies and/or authentication (credentials). Most request contexts, such as img and script elements, include credentials in fetches by default. Thus if the user has stored credentials for https://credentialed.example/ (Credentialed), they will be included in outgoing fetches from Untrusted. (See ambient authority for why this might lead to problems.) Being able to access the contents of resources of Credentialed would be as problematic as accessing those of Intranet.

Because of SOP XMLHttpRequest has historically had a same-origin restriction. Reading the contents of resources of Credentialed and Intranet would be problematic. However, this also excludes access to notable non-Credentialed non-Intranet servers, such as https://example.com/ (Example). The problem with Example is that it cannot be distinguished from Intranet (private IPv4 address ranges are not reliably used). We invented CORS so that Untrusted can access the contents of resources on Example (and even on Credentialed and Intranet) as long as the resource opts in. To better understand CORS we first need to look at the historical request contexts.

At some point the same-origin policy did not exist and various requests could be made across origins, including credentials, leading to some leakage of Credentialed and Intranet data. Because of the network effects of the web these holes could not be fixed and are now enshrined and part of the security model:

  • CSS can be fetched across origins as long as the MIME type is text/css. document.styleSheets is not available unless CORS is used, but since CSS rules are applied and getComputedStyle() exists, a leak exists.
  • Images can be fetched across origins and will leak the width and height of any resources that can be decoded as an image. With scripting timing attacks can be used to determine whether there is a server on the other side. When a cross-origin image is painted upon a canvas context, that context is tainted and certain methods that expose pixel data will cease to operate. CORS can be used on the image to keep the context fully working. Audio and video have similar leakage. (No MIME type checks enforced on the fetched resource.)
  • JavaScript fetched across origins will execute. And it gets worse. JavaScript fetched from Credentialed by Untrusted will execute with the powers of Untrusted, creating that ambient authority problem mentioned earlier. When CORS is not used SyntaxError exceptions thrown will not expose any data. That is good because MIME type checks are not enforced for JavaScript either, but also comments and dead code blocks might get exposed that would otherwise not be.
  • Forms can be submitted across origins and have a couple of additional vectors. Forms can use the POST method. Forms can also include request bodies of the types text/plain, application/x-www-form-encoded, and multipart/form-data. The response however is not exposed.

Now as should be clear from above what CORS enables is allowing reading the contents of resources across origins (e.g. from Untrusted to Credentialed). To a far greater extent than the enshrined information leaks that already exist. CORS also enables the full power of XMLHttpRequest across origins. However, that would be far more than what has been traditionally allowed through images and forms (e.g. custom methods and headers). Therefore requests that use the full power require a CORS preflight request. The CORS preflight request confirms that the URL understands CORS. The final response still has to include the relevant CORS headers.

Hopefully this makes it clear that the same-origin policy solves a real problem. And that we need CORS due to Credentialed, Intranet, the more powerful requests it allows, and the inability to distinguish Example from either Credentialed or Intranet. This is also the reason we do not have a TCP API. It would be great to have a solution that would remove the need for CORS and allow a TCP API, but if you think you have one and it involves asking the user, think again. And if you want to expand the information leaks (e.g. allowing document.styleSheets without CORS), reconsider. All the information leaks we have are enshrined bugs from the time JavaScript became a thing.

(Please understand that this is introductory material. It is simplified somewhat for brevity.)

Planet MozillaNueva interfaz para Firefox en tabletas

Nuevamente es martes de actualizaciones para Firefox y tengo el placer de compartir con ustedes las novedades que encontrarán en la nueva versión del panda rojo tanto para escritorios como dispositivos móviles. En este lanzamiento sobresalen la nueva interfaz para las tabletas y la introducción de los primeros cambios hacia una arquitectura multiprocesos.

En aras de mejorar la experiencia de usuario, el nuevo Firefox adquiere un buen balance entre simplicidad y poder al ofrecer una barra de pestañas horizontal y un panel de pestañas a pantalla completa. Diseñada para uso horizontal y vertical, la nueva interfaz permite aprovechar todo el tamaño de pantalla que ofrecen las tabletas disponibles en el mercado.

firefox-tablets

La introducción de los primeros cambios hacia una arquitectura multiprocesos, hará que Firefox maneje las pestañas, complementos y plugins fuera del proceso principal. Electrolysis (e10s) es el nombre del proyecto que busca añadir esta característica al núcleo del navegador y una vez terminado, proveerá mayor seguridad, estabilidad y, si algo falla, Firefox no se cerrará inesperadamente.

El moderno diseño para las búsquedas ahora debuta en otras localizaciones del navegador más traducido del mundo. Al visitar una página web que sugiera adicionar un motor de búsqueda a Firefox, se mostrará un ícono + en la barra de búsqueda.

sugerencias-busquedas-firefox36

Los sitios “fijados” en la página nueva pestaña se podrán sincronizar a través de las cuentas Firefox en todos tus dispositivos. Se ha dado soporte completo a la versión 2 del protocolo HTTP, el cual habilita una web más rápida, escalable y responsiva. También se ha localizado el Panda Rojo en uzbeco gracias a la Comunidad de Uzbequistán.

Para Android Lollipop se ha añadido el idioma Maithili (mai).

Otras novedades

Si deseas conocer más, puedes leer las notas de lanzamiento (en inglés).

Puedes obtener esta versión desde nuestra zona de Descargas en español e inglés para Linux, Mac, Windows y Android. Recuerda que para navegar a través de servidores proxy debes modificar la preferencia network.negotiate-auth.allow-insecure-ntlm-v1 a true desde about:config.

W3C Team blogW3C, Mobile World Congress 2015 and you

The past year has seen been one of significant milestones in W3C with the highlight being that HTML5 became a Recommendation in late October of 2014. We have seen the Open Web Platform continuing to have an impact on a diverse set of Industries. Within W3C this is manifest by the continued growth of our Digital Publishing Interest Group and the launch of both our Web Payments Interest Group and Automotive Working Group. When you look at the hottest thing going on across Industries you have to recognize the Internet of Things as a movement that has gained an incredible amount of traction. This work within W3C is happening in our Web of Things Interest Group. When you look at the Exhibitors and Attendees as well as the Sessions being held during Mobile World Congress 2015 there is a huge overlap in these conversations. In fact, many of the GSMA Members that also Members of W3C are active participants in one or more of these groups – shouldn’t you be?

W3C will be at Mobile World Congress 2015 but we will not have a booth this year. We will be represented by Dominique Hazael-Massieux, our Mobile Guru, and J. Alan Bird, our Global Business Development Leader. They will be on-site at Fira Gran Via in Barcelona, Spain from the afternoon of Monday, 02 March 2015 through Thursday, 05 March 2015. If you are in any of the industries that are being touched by the Open Web Platform we’d love to have a conversation with you. If you’re not in one of the Industries mentioned above but are curious about W3C, we’d also love to have a conversation with you. The best way to schedule that is to send an e-mail to abird@w3.org.

We look forward to seeing you in Barcelona and working together to help the Web reach its Full Potential!

Bruce LawsonReading List

Planet MozillaThis Week In Servo 24

This week, we merged 37 pull requests.

Servo, Rust, HTML5ever, and Hyper are participating in this year’s Google Summer of Code! You can find the project ideas here.

Notable additions

New contributors

Planet MozillaProgressive Enhancement is not about JavaScript availability.

I have been telling people for years that in order to create great web experiences and keep your sanity as a developer you should embrace Progressive Enhancement.

escalators are great. when there is an issue, they become stairs and still work

A lot of people do the same, others question the principle and advocate for graceful degradation and yet others don’t want anything to do with people who don’t have the newest browsers, fast connections and great hardware to run them on.

People have been constantly questioning the value of progressive enhancement. That’s good. Lately there have been some excellent thought pieces on this, for example Doug Avery’s “Progressive Enhancement Benefits“.

One thing that keeps cropping up as a knee-jerk reaction to the proposal of progressive enhancement is boiling it down to whether JavaScript is available or not. And that is not progressive enhancement. It is a part of it, but it forgets the basics.

Progressive enhancement is about building robust products and being paranoid about availability. It is about asking “if” a lot. That starts even before you think about your interface.

Having your data in a very portable format and having an API to retrieve parts of it instead of the whole set is a great idea. This allows you to build various interfaces for different use cases and form factors. It also allows you to offer this API to other people so they can come up with solutions and ideas you never thought of yourself. Or you might not, as offering an API means a lot of work and you might disappoint people as Michael Mahemoff debated eloquently the other day.

In any case, this is one level of progressive enhancement. The data is there and it is in a logical structure. That’s a given, that works. Now you can go and enhance the experience and build something on top of it.

This here is a great example: you might read this post here seeing my HTML, CSS and JavaScript interface. Or you might read it in an RSS reader. Or as a cached version in some content aggregator. Fine – you are welcome. I don’t make assumptions as to what your favourite way of reading this is. I enhanced progressively as I publish full articles in my feed. I could publish only a teaser and make you go to my blog and look at my ads instead. I chose to make it easy for you as I want you to read this.

Progressive enhancement in its basic form means not making assumptions but start with the most basic thing and check every step on the way if we are still OK to proceed. This means you never leave anything broken behind. Every step on the way results in something usable – not necessarily enjoyable or following a current “must have” format, but usable. It is checking the depth of a lake befor jumping in head-first.

Markup progressive enhancement

Web technologies and standards have this concept at their very core. Take for example the img element in HTML:

<img src="threelayers.png" 
     alt="Three layers of separation 
          - HTML(structure), 
          CSS(presentation) 
          and JavaScript(behaviour)">

By adding an alt attribute with a sensible description you now know what this image is supposed to tell you. If it can be loaded and displayed, then you get a beautiful experience. If not, browsers display this text. People who can’t see the image at all also get this text explanation. Search engines got something to index. Everybody wins.

<h1>My experience in camp</h1>

This is a heading. It is read out as that to assistive technology, and screen readers for example allow users to jump from heading to heading without having to listen to the text in between. By applying CSS, we can turn this into an image, we can rotate it, we can colour it. If the CSS can not be loaded, we still get a heading and the browser renders the text larger and bold as it has a default style sheet associated with it.

Visual progressive enhancement

CSS has the same model. If the CSS parser encounters something it doesn’t understand, it skips to the next instruction. It doesn’t get stuck, it just moves on. That way progressive enhancement in CSS itself has been around for ages, too:

.fancybutton {
  color: #fff;
  background: #333;
  background: linear-gradient(
    to bottom, #ccc 0%,#333 47%,
    #000 100%
  );
}

If the browser doesn’t understand linear gradients, then the button is white on dark grey text. Sadly, what you are more likely to see in the wild is this:

.fancybutton {
  color: #fff;
  background:  -webkit-gradient(
    linear, 
    left top, left bottom, 
    color-stop(0%,#ccc), 
    color-stop(47%,#333), 
    color-stop(100%,#000));
}

Which, if the browser doesn’t understand webkit gradient, results in a white button with white text. Only because the developer was too lazy to first define a background colour the browser could fall back on. Instead, this code assumes that the user has a webkit browser. This is not progressive enhancement. This is breaking the web. So much so, that other browsers had to consider supporting webkit specific CSS, thus bloating browsers and making the web less standardised as a browser-prefixed, experimental feature becomes a necessity.

Progressive enhancement in redirection and interactivity

<a href="http://here.com/catalog">
See catalog
</a>

This is a link pointing to a data endpoint. It is keyboard accessible, I can click, tap or touch it, I can right-click and choose “save as”, I can bookmark it, I can drag it into an email and send it to a friend. When I touch it with my mouse, the cursor changes indicating that this is an interactive element.

That’s a lot of great stuff I get for free and I know it works. If there is an issue with the endpoint, the browser tells me. It shows me when the resource takes too long to load, it shows me an error when it can’t be found. I can try again.

I can even use JavaScript, apply a handler on that link, choose to override the default behaviour with preventDefault() and show the result in the current page without reloading it.

Should my JavaScript fail to execute for all the reasons that can happen to it (slow connection, blocked resources on a firewall level, adblockers, flaky connection), no biggie: the link stays clickable and the browser redirects to the page. In your backend you probably want to check for a special header you send when you request the content with JS instead of as a redirect from the browser and return different views accordingly.

<a href="javascript:void()">
Catalog</a> or 
<a href="#">Catalog</a> or 
<a onclick="catalog()">Catalog</a>

This offers none of that working model. When JS doesn’t work, you got nothing. You still have a link that looks enticing, the cursor changed, you promised the user something. And you failed at delivering it. It is your fault, your mistake. And a simple one to avoid.

XHTML had to die, HTML5 took its place

When XHTML was the cool thing, the big outcry was that it breaks the web. XHTML meant we delivered HTML as XML. This meant that any HTML syntax error – an unclosed tag, an unencoded ampersand, a non-closed quote meant the end user got an error message instead of the thing they came for. Even worse, they got some cryptic error message instead.

HTML5 parsers are forgiving. Errors happen silently and the browser tries to fix them for you. This was considered necessary to stop the web from breaking. It was considered bad form to punish our users for our mistakes.

If you don’t progressively enhance your solutions, you do the same thing. Any small error will result in an interface that is stuck. It is up to you to include error handling, timeout handling, user interaction like right-click -> open in new tab and many other things.

This is what progressive enhancement protects us and our users from. Instead of creating a solution and hoping things work out, we create solutions that have a safety-belt. Things can not break horribly, because we planned for them.

Why don’t we do that? Because it is more work in the first place. However, this is just intelligent design. You measure twice, and cut once. You plan for a door to be wide enough for a wheelchair and a person. You have a set of stairs to reach the next floor when the lift is broken. Or – even better – you have an escalator, that, when broken, just becomes a set of stairs.

Of course I want us to build beautiful, interactive and exciting experiences. However, a lot of the criticism of progressive enhancement doesn’t take into consideration that nothing stops you from doing that. You just have to think more about the journey to reach the final product. And that means more work for the developer. But it is very important work, and every time I did this, I ended up with a smaller, more robust and more beautiful end product.

By applying progressive enhancement to our product plan we deliver a lot of different products along the way. Each working for a different environment, and yet each being the same code base. Each working for a certain environment without us having to specifically test for it. All by turning our assumptions into an if statement. In the long run, you save that way, as you do not have to maintain various products for different environments.

We continuously sacrifice robustness of our products for developer convenience. We’re not the ones using our products. It doesn’t make sense to save time and effort for us when the final product fails to deliver because of a single error.

Steve Faulkner et alTPG at CSUN 2015

The 30th Annual International Technology and Persons with Disabilities conference (otherwise known as CSUN), gets underway on Monday 2nd March 2015. Several of the TPG team will be there, and here’s where you’ll find us if you’d like to say hello or hangout for a bit.

Monday 2nd March

Beyond Code and Compliance: Integrating Accessibility Across The Development Life Cycle

When:
9am PST (full day pre-conference workshop).
Where:
Hillcrest AB.
Who:
Billy Gregory, Hans Hillen, Henny Swan, Karl Groves, Léonie Watson, Mike Paciello, Shane Paciello and Steve Faulkner.

TPG will explore ways for integrating accessibility into software development. Whether you work in a large or small organization, follow an agile or waterfall process, are experienced or just starting out, this workshop will guide you toward reliably developing accessible and usable products.

Tuesday 3rd March

Implementing ARIA and HTML5 into Modern Web Applications

When:
1.30pm PST (half day pre-conference workshop).
Where:
Hillcrest AB.
Who:
Hans Hillen and Steve Faulkner.

In this afternoon session we will discuss how ARIA and HTML5 can be utilized to create modern, accessible web applications. The session complements the morning’s “Introduction to ARIA and HTML5″ session, by continuing with more advanced topics and hands-on examples. We recommend attending both half sessions as a full day workshop.

Wednesday 4th March

Web Components: Background, opportunities and challenges

When:
1.20pm PST.
Where:
Cortez Hill C.
Who:
Karl Groves with Alice Boxhall (Google).

Web Components are a potential paradigm shift for the way we build websites. We will explain the technologies involved and discuss the accessibility challenges faced.

Do we need to change the web accessibility game plan (Redux)?

When:
1.20pm PST.
Where:
Gaslamp AB.
Who:

 Léonie Watson with John Foliot (JP Morgan Chase), Jared Smith (WebAIM), Glenda Sims (Deque) and Jennison Asuncion (Linked In).

Revisiting the 2011 CSUN session of the same name, we look back 4-years to evaluate predictions, successes and failures, and a re-setting of the state-of-our-state.

Screen readers, browsers and HTML: The current state of play

When:
1.20pm PST.
Where:
Hillcrest AB.
Who:
Steve Faulkner and Charlie Pike, with Sarita Sharan (CA Technologies).

Introducing a research project into screen reader and browser interaction with HTML elements, including a review of findings using slides and live demonstrations.

Thursday 5th March

Accessible graphics with SVG

When:
9am PST.
Where:
Seaport B (IBM suite).
Who:
Léonie Watson, with Rich Schwertdfeger (IBM), Fred Esch (IBM), Doug Schepers (W3C), Jason White (ETS), Markku Häkkinen (ETS) and Charles McCathie Nevile (Yandex).

A look at the future of accessible graphics on the web and the possibility space of SVG content.

CEO Roundtable: The Future of Web Accessibility

When:
1.20pm PST.
Where:
Solana Beach AB.
Who:
Mike Paciello with David Wu (AISquared) and Tim Springer (SSB-Bart).

A panel discussion of accessibility industry leaders, addressing needs and challenges faced by website owners who try to provide accessibility and inclusion on their websites.

Moving the Digital Accessibility Needle: Updates & Perspectives from US DOJ & Access Board

When:
2.20pm PST.
Where:
Mission Beach AB.
Who:
Mike Paciello with Rebecca Bond (DoJ) and Gretchen Jacobs (Access Board).

Two of the nation’s leading agencies – the US Access Board and the Department of Justice will explain important updates to Section 508 and the ADA, as well as share their organizational perspective concerning accessibility to ICT.

30th CSUN Anniversary Celebration

When:
7pm PST.
Where:
Seaport Ballroom D/E.
Who:
Billy Gregory, Charlie Pike, Deb Rapsis, Graeme Coleman, Hans Hillen, Henny Swan, Karl Groves, Léonie Watson, Mike Paciello, Shane Paciello and Steve Faulkner.

Please join us for an exciting evening of entertainment to celebrate the 30th anniversary of the CSUN Conference. Geri Jewell, one of our past Keynote Speakers, will serve as the program’s emcee and performances by musician and humorist, Mark Goffeney and comedian Chris Fonseca are sure to make this a night to remember. The celebration is sponsored by IBM and The Paciello Group and will continue with a reception sponsored by Amazon, following the program.

Friday 6th March

WAI-ARIA: Common pitfalls and solutions with the Viking and the Lumberjack

When:
9am PST.
Where:
Cortez Hill B.
Who:
Billy Gregory and Karl Groves.

With great power comes great responsibility. Learn how to avoid WAI-ARIA anti-patterns with Karl “The Viking” Groves and Billy “The Lumberjack” Gregory.

What’s in a name? Accessible name computation

When:
10am PST.
Where:
Cortez Hill B.
Who:
Billy Gregory and Karl Groves.

Assistive technologies convey names for UI controls according to an algorithm. This session discusses how this algorithm works, and ways developers get it wrong.

The secret life of an accessible media player

When:
10am PST.
Where:
Hillcrest CD.
Who:
Henny Swan.

A journey around the web looking at what makes both an accessible and usable multimedia player.

Interaction notifier: Making RIA interaction accessible to everyone

When:
1.20pm PST.
Where:
Gaslamp CD.
Who:
Hans Hillen and Léonie Watson.

Users are not always aware of interaction models in Rich Internet Applications. To address this, “Interaction Notifier” adds discoverable contextual documentation to rich web content.

Managing remediation of accessibility web defects in a large enterprise

When:
1.20pm PST.
Where:
Gaslamp AB.
Who:
Karl Groves with Daniel Frank (Wells Fargo Bank).

The authors describe the practical implementation of a weighting and scoring methodology for web property accessibility defects at a large business enterprise.

Lessons learned: Dynamic content updates and ARIA live regions

When:
2.20pm PST.
Where:
Gaslamp CD.
Who:
Hans Hillen with Jennifer Gauvreau (CGI) and Elizabeth Whitmer (CGI).

TPG/CGI will share techniques for dealing with dynamic content updates and discuss challenges with inconsistent ARIA Live Region support by browser and AT products.

Implementing Accessibility In a Widely Distributed Web Based Visualization and Analysis Platform

When:
TBD.
Where:
TBD.
Who:
Mike Paciello and Graeme Coleman, with Dr. Georges Grinstein (U-Mass Lowell), Franck Kamayou (U-Mass Lowell).

We present a solution based on previous research that allows a system to do automatic analysis of a line chart visualization to extract and then present it’s intended message for blind and low vision consumers. Previous advancements in this area, an implemented prototype of the proposed solution and a description of the platform in which it was built are presented, as well as a discussion of the implications of this research and future work.

Saturday 7th March

SS12 Code for a Cause Finals – Project Possibility

When:
9am.
Where:
Mission Beach C.
Who:
Mike Paciello with students from CSUN, USC, UCLA.

This exciting event will host the innovative open source projects the top teams from CSUN, UCLA and USC have created. A continental breakfast will be served following the presentations and judging, prior to the announcement of
the First Place Team. We encourage you to mark your calendars for this important occasion to support the student teams and the time and work they have invested.

Planet MozillaFlipboard and the “mobile web” dream…

According to the luminaries of the web design world, the “mobile web” doesn’t exist. Stephen Hay said it, Smashing Magazine said so and Jeremy Keith amongst many others.

Except, it does. Not as a technical reality, but as a dream of marketing and overly excited managers who believe in magical powers of their engineers. And it is nothing new. We’ve been there before, and we probably will get there again. Every few years a new form factor or technology comes out that promises a web that is catering perfectly to a very niche need whilst being open and available to all. Or – in other words – water that is almost, but not totally wet.

Flipboard’s magical journey at 60 fps

Last week Flipboard engineering released the Kraken with their “60 fps on the mobile web” post, explaining how they managed to give the new “web version” of Flipboard a buttery smooth 60 frames per second on mobile by forfeiting the DOM and rendering the whole thing in Canvas. The general outrage was that it was read as a call to give up on the DOM and use react.js and Canvas instead.

Kid needs light-up shoes to run faster - we need canvas to have a performing web app

Not surprisingly, this sparked a lot of discussion and annoyance toward them. But let’s step back a bit: first of all, it is a good thing when engineering teams publish their findings and tell us why they made some decisions. It can be a great learning resource and is a courageous thing to do. In a world where people spend most of their time dismembering technical posts on hacker news putting yourself out there means you also need to deal with a lot of polarised feedback. You need to learn fast how to deal with trolls and people using these systems to grandstand. So, thanks are in order – thank you, Flipboard. The post is detailed, it explains what they tried to do and how they achieved it.

The usefulness of such a resource starts getting questionable when hype kicks in and people take it as gospel. Flipboard did not claim that what they do is the perfect way to achieve happiness on mobile. Instead they are very open about their intentions:

In the pursuit of 60fps we sometimes resort to extreme measures. Flipboard for mobile web is a case study in pushing the browser to its limits. While this approach may not be suitable for all applications, for us it’s enabled a level of interaction and performance that rivals native apps.

Right now, mobile is still hot. HTML5 is still hot. A company pushing the boundaries of both is hot. That’s why it is good value for a tech company to write articles like that. And I am 100% sure the Flipboard engineers have the heart in the right place and rightfully are proud of what they have achieved.

Solving the right problem with the wrong approach

That said, the title is terrible as it promises a simple answer to a huge problem. Or, on reflection, not a problem, but a massive misunderstanding.

What this post told me most of all is that it isn’t solving an issue of the web. It is about simulating a native interface for a mobile touch device at 60 fps using web technologies. It’s aim is to run smoothly on the currently hot device that the most affluent people use – let’s say the iPhone 6. They are not beating around the bush about this at all:

Now we’re coming full circle and bringing Flipboard to the web. Much of what we do at Flipboard has value independent of what device it’s consumed on: curating the best stories from all the topics, sources, and people that you care about most. Bringing our service to the web was always a logical extension.

There’s a massive flaw in this: it tries to bring value independent of what device it is consumed on and then it applies assumptions and concepts that are only beneficial to a mobile, touch-enabled device.

This is a porting exercise. This is bringing something to the web. It is not creating a web product. That would be reaching as many people as possible, and making your content available independent of hardware, ability and environment. You can not “go to web”. The web is not another form factor when your starting point is a very restricted environment. That’s how we got text-only versions of Flash movies and “accessible versions” of DHTML monstrosities. Using web technology means you start with the assumption that you have no control over the way your content is consumed. And that’s as far removed as it can from what Flipboard said:

Flipboard launched during the dawn of the smartphone and tablet as a mobile-first experience, allowing us to rethink content layout principles from the web for a more elegant user experience on a variety of touchscreen form factors.

And this is why calling this a web experience is ill-advised from the get-go. As Faruk Ates put it:

…what Flipboard unleashed onto the world a “Web” version is akin to calling a collection of tires, AA batteries and spare car parts a Tesla.

This is not about the technical implementation. The flaw is in the idea of turning a native app that was specifially designed for a certain form factor into a “web version”. The journey explained is full of assumptions.

The first one is the big 60. All our web technology efforts need to match the 60 fps threshold or they are useless, right? Well, looking at my Nexus 5 device with Android Lollipop and anything I could do to be state-of-the-art I can safely say that not a single app works consistently smoothly at 60 fps or doesn’t stutter from time to time. So, I guess, I have no apps.

Of course, we need goals to aspire to and some things to measure against. But defining something like 60fps as a baseline is pretty naive considering the fragmentation of hardware and the amount of abstraction layers an HTML5 app has to go through. Maybe considering less greedy interaction models might be a way forward?

Which brings me to the second assumption: endless scrolling is what the web is about. And from there, the article tries to make that work. True, a lot of solutions have endless scrolling. Also true, many a time that is more annoying than a pagination would have been.

What Flipboard did here is to build a native-feel solution in an environment that has no native paradigms. There are no form-factor standards on the web. They change constantly. The parallax scrolling page today is the multi-level DHTML drop-down of tomorrow.

Stretching the canvas

It is a daring and seemingly clever solution to go for canvas to reach good performance in an endless scrolling interface. But that’s all it is – a currently clever solution with a very short shelf-life. As Rachel Nabors put it:

Re: recreating the DOM in canvas and accessibility: we will look back on this as a hack. Browsers will catch up, will render the DOM faster.

When working on Firefox OS and low-end devices, the main issue just a few months ago was that canvas was too slow and people used the DOM instead and complained that it is the wrong API for games. The push was towards canvas to become hardware accelerated or – even better – mobile browsers to support WebGL. Now we have this and many a DOM solution using all kind of 3D transform tricks in CSS to force hardware accelerated layers look silly. Much like this might look silly very soon.

The honest approach to this would be to give the article the title “How we created endless scrolling at 60 fps on high-end mobile devices by replacing the DOM with Canvas”. But that’s not as quotable and it doesn’t make you sound like a trailblazer solving a “real and very common” problem.

Access denied

Flipboard already admits that in terms of accessibility, this is a horrible solution. Screenreader availability aside, my warning bells went off when I saw elements with not only fixed positions on the screen but also fixed dimensions. And when you use a tool that supports text but means you need to calculate line breaks, I’d get a queasy feeling in my stomach. There goes resizing the font. There goes copying and pasting. There goes translating a text. All things the web is great at. And it is nothing new.

Every single time we replaced text with graphics to gain more control we had to re-invent these basic features of a text-based web. Remember image replacement techniques of headlines before we had web fonts? Remember how Flash had to bend over backwards to seed the browser history and to make text highlight-able? Remember how it was impossible on mobile devices to copy and paste at all?

When reading the section of the post about accessibility I get flashbacks to horrible times:

This area needs further exploration. Using fallback content (the canvas DOM sub-tree) should allow screen readers such as VoiceOver to interact with the content. We’ve seen mixed results with the devices we’ve tested. Additionally there is a standard for focus management that is not supported by browsers yet.

One approach that was raised by Bespin in 2009 is to keep a parallel DOM in sync with the elements rendered in canvas. We are continuing to investigate the right approach to accessibility.

Wait a second. Re-read this, please, and tell me if I am crazy when I think this sounds like a lot of duct-tape around a leaking faucet.

We have a perfectly accessible way of achieving all of this: we call it HTML and we have the DOM and the accessibility API. That way we’d have a perfectly working web site that gets found, indexed, is accessible to a lot of people. We can then convert into a fast performing canvas solution on mobile devices that are touch-enabled. We can test for that – web technology is good at that. Think about it: a data source that comes with lots of accessibility benefits and already has an API to read it in part? Sign me up.

Instead we consider adding a lot of HTML content inside the canvas element as a fallback and hope browsers and assistive technology will do something with that.

Content inside canvas becomes interesting when canvas isn’t supported. Today, that’s an almost non-existent use case. Browsers support canvas. It is a blank, stateless placeholder in the page that we paint on and wipe. It is an etch-a-sketch in the page. It isn’t even vector based – it is a dumb collection of pixels. That’s why canvas is fast – the Flipboard article does a good job explaining that.

Canvas is to HTML5 what DIV and SPAN are to HTML4 - it has no semantic meaning. It is a placeholder for generated, visual content. Not text, not interactive elements. When canvas came to be, the first comments were that it is the new applet. And that is still quite a sensible way of describing it from a markup/semantic meaning angle. It is as much a black box in the page as Flash was to the rest of the markup.

Replace the DOM, but not its virtues

The problem with Flipboard’s solution to the mobile performance issue is that it wants to use canvas because of its stateless and high-performance nature and get all the benefits of HTML and the DOM at the same time. This is asking too much.

We could, however, get quite a long way by creating our canvas content from HTML that is in the document. And that is what it means to build a web product – you build a sensible HTML solution first, and then use CSS and JavaScript to enhance it. That way you don’t need to worry about covering the basics after you fine-tuned the final product. That approach will always result in disappointing fallbacks that are hardly ever maintained.

The “mobile web” thing

The whole goal of this exercise was to match native performance by using web technology and applying ideas of native environments. And this is a thing we should be aware of. The mobile web.

The issue is that according to this post and many others of the same ilk, the mobile web exists. The mobile web is not a technical thing – it is a misconception and one that is hard to match. Companies nowadays start with a native experience. This is where the short-term gain is. This is where the short-term high user numbers lie. This is the beautiful technology – the shiny devices, the backing from OS vendors and the exciting interaction models that come with the OS. It feels good, it feels that there is something to work with.

Then these people hear about the benefits of web technologies: cross-platform support, lower cost of engineering as you don’t need to pay experts of red-hot new technologies, re-use across different form factors and so on. A lot of times this happens when the honeymoon period of the native solution is over and there is new growth to be unearthed.

Then, they want both. Hence, “the mobile web”.

We have two issues to battle with this: a technical problem and an image problem.

Let’s talk technical first: matching the performance of a native app with a web app is a game the web can currently not win. At least not if all you count as performance is its fps speed and how well it integrates with the OS. The reason is that the cards are stacked against us. Unless the browser becomes the most important thing in a mobile OS, we will always have a hacky experience.

To the OS vendor, the browser is a necessary thing to offer – another app much like the calculator or the phone app. From a marketing and monetary point of view, having people build exclusively for your platform with your tools and in your languages is a win-win. You have a captive audience and you can show their numbers in huge figures during your keynotes to your stakeholders. Making the browser amazing means you have to share.

Add to this the ridiculous browser fragmentation and the amount of zombie browsers on older systems that will never get an update again and you will find that matching the newest and coolest of the native offering is really hard.

The image problem is that the web seems stale. Our market thrives on hype and short term gains. All our messaging about the web is praising its longevity. What you do now means you will benefit in years to come, even if it means some extra work. Your products will be easy to maintain for years to come. Future technology will already support what you do now and it will be easy to extend if you use progressive enhancement. This is our messaging, and it is great messaging.

Except when all you care about is the next month’s user and download numbers and how many people posted selfies and sent stickers to each other that you sold them. Then this message is just noise and sounds like arrogant “in my days” speeches.

If we love the web and want it to work out, we need to pick our battles. And the real battle lies with operating systems not having a browser we can rely on. And with lack of support of very basic features we need to give an app experience across browsers and platforms. And with a lack of standards that really get to fruition and are implemented instead of being replaced by a library that is open source, but never leaves beta.

The mobile web is real – and it isn’t. It is a dream of people who don’t get the nature of the web trying to harvest its benefits. And right now, these people get all the VC money and get the tech press excited as they promise something new. The hype as it is now is not sustainable. It killed these people once in the dotcom boom and will again – very soon.

We can stomp our foot about that and get angry and we can repeat tiring old messages. Or we can bide our time and fix what is broken and keeps us from performing the way we want to. Let those who need the quick short-term win pretend they found the solution. Our time can be used differently. The amount of devices, form factors and user needs is not decreasing any time soon. The web can cover all of them. Let’s make that happen once the fever dream of the one true app form factor is over. Much like there is not one mode of transport to cover all needs, there is no such thing as a perfect app or interaction model.

Steve Faulkner et alHTML5 canvas sub DOM

All modern browser implement the HTML5 specification’s canvas element fallback concept. This means that the content inside a canvas elements start and end tags is exposed to assistive technology users even when the browser supports canvas. It also means that any interactive elements, inside a canvas elements start and end tags such as links or form controls will be included in a documents default tab order even when the browser supports canvas.

Canvas has a navigable sub DOM

The HTML5 specification states:

When a canvas element represents embedded content, the user can still focus descendants of the canvas element (in the fallback content). When an element is focused, it is the target of keyboard interaction events (even though the element itself is not visible). This allows authors to make an interactive canvas keyboard-accessible: authors should have a one-to-one mapping of interactive regions to focusable elements in the fallback content. (Focus has no effect on mouse interaction events.)

Which translates as meaning the content inside a canvas element:

If interactive, should be included in the tab order and operable with the keyboard. All content is accessible to assistive technology such as screen readers. So in the canvas example, the text alternative should be announced by screen readers and the links are navigable and operable using the keyboard.

If you are using any modern browser with the keyboard to navigate interactive elements, you can tab to the links in the canvas example. If you are using a screen reader, you can also access the text alternative (in this case) content inside the canvas element.

Canvas example with a text alternative and interactive elements (links) inside the canvas element.

Canvas example HTML code:

<canvas id="example" width="260" height="200">
<h2>Shapes</h2>
<p>A rectangle with a black border. In the background is a pink
circle. Partially overlaying the
<a href="http://en.wikipedia.org/wiki/Circle"
onfocus="drawCircle();" onblur="drawPicture();">circle</a>.
Partially overlaying the circle is a green
<a href="http://en.wikipedia.org/wiki/Square" onfocus="drawSquare();"
onblur="drawPicture();">square</a> and a purple
<a href="http://en.wikipedia.org/wiki/Triangle"
onfocus="drawTriangle();" onblur="drawPicture();">triangle</a>,
both of which are semi-opaque, so the full circle can be seen
underneath.</p>
</canvas>

NVDA announces Canvas sub DOM content (from example page):

“Shapes heading  level 2
A rectangle with a black border. In the background is a pink circle. Partially overlaying the visited  link circle. Partially overlaying the circle is a green visited  link square and a purple visited  link triangle, both of which are semi-opaque, so the full circle can be seen underneath.”

Other examples:

Video demo’s:

Some Potential Issues

  • Where content is currently included in canvas, it is almost exclusively of the “get a better browser that supports canvas” variety, so screen reader users will encounter this when using any modern browser.
  • Authors will most likely continue to provide useless information (for screen reader users) as the general definition of “fallback content” in the HTML5 specification is at odds with how canvas content is presented to keyboard and AT users in browsers that support the canvas element and at odds with how the term is used in reference to other embedded content elements where the fallback content is ONLY available to users if the prmary content is not supported.
  • Keyboard focus is lost if interactive elements are included inside the canvas, because while elements inside the canvas element are focusable by default, there is no corresponding default indication of focus to identify where current focus is. This means that developers must provide pseudo focus rings that are displayed when an interactive element in the canvas sub DOM receives focus, an example of this is provided triggered by inline (for demo purposes only) onfocus/onblur event handlers. There is a proposal for dedicated canvas API methods for the display of focus rings, but these have not yet been agreed upon or implemented.

Developers Be(a)ware

In Firefox and IE and Chrome and Opera and Safari HTML content inside the canvas element is available at all times to assistive technology users and keyboard users can operate links and controls and they are included in the default tab order. So please provide alternatives to on screen canvas content and interactivity in the canvas sub DOM. DO NOT include a dumb message telling the user that they need to get a browser that supports canvas!

HTML5 definition of ‘fallback content

Some embedded content elements can have fallback content: content that is to be used when the external resource cannot be used (e.g. because it is of an unsupported format). The element definitions state what the fallback is, if any.

This general definition of fallback content is problematic, because in the case of canvas and how its use pertains to assistive technology and keyboard users. In modern browsers the <canvas> element is supported, so the content in the sub DOM is not fallback. The example I provided could  have had hit testing on the shapes drawn on the canvas so that a click event could be passed to the links in the sub DOM. The use of the term ‘fallback’ for the contents of the sub DOM helps to mislead developers about sub DOM use. An example of erroneous information is provided in the video: HTML5 Canvas Fallback Content Tutorial

Steve Faulkner et alFlipboard – React Canvas Accessibility

Flipboard announced yesterday that they are using HTML5 canvas to improve performance on mobile devices. This has accessibility ramifications.

flipboard accessibility

Testing the Flipboard site on a mobile device such as an iPhone, quickly reveals that the content is not available to screen reader users. This is because the content is painted onto a canvas element using React Canvas. The words, structure and UI are all visual, there is nothing but pixels.

Reading the React Canvas documentation. It states:

Accessibility

This area needs further exploration. Using fallback content (the canvas DOM sub-tree) should allow screen readers such as VoiceOver to interact with the content. We’ve seen mixed results with the iOS devices we’ve tested.

I put together a very simple example of what Flipboard could do to make the content trapped in the canvas pixels available to other users:

screenshot of Flipboard story about John Stewart resigning from the Daily Show, as rendered using HTML5 canvas.

Structured text and interactive content within the canvas sub DOM

<canvas id="myCanvas" width="300" height="533">
<header>
<p>From the daily edition.</p>
<h2>
<a id="theLink" 
href="http://www.buzzfeed.com/jarettwieselman/why-well-miss-jon-stewart-on-the-daily-show" 
target="_blank"> 19 Reasons We’ll Miss Jon Stewart On “The Daily Show”</a>
</h2>
</header>
<p>buzzfeed - Jarett Wieselman</p>
<p>We’ll need a million Moments of Zen to emotionally endure this loss. Nearly 16 years 
after he took over from Craig Kilborn as host, Jon Stewart announced on Tuesday that 
he would be leaving The Daily Show. But it’s hard to imagine a TV future without 
him for many reasons… Like, the fact that, …
</p>
</canvas>

Notes:

  • Tested with VoiceOver on iPhone 4, the structured HTML content is available and can be interacted with.
  • The test file only includes one event handler on the canvas that activates the link in the canvas sub DOM (to keep it simple for demo purposes and also because its the limit of my canvas scripting abilities.)
  • It is presumed that since all the data that is written to the canvas is available it should not be onerous to include it in the canvas sub DOM and this could occur after the canvas has been painted, so as not to effect the performance benefits.

Planet MozillaCSS Parser and Tokenizer API

The first time I mentioned the need to reach the internal CSS parser of a rendering engine from JavaScript was 17 years ago during the CSS WG meeting I hosted at Électricité de France in 1998, during a corridor discussion. All the people around said it was out of question. Then I put it back on the table in 2002, with little success again. Today, I was actioned by the CSS WG and the TAG in the Houdini Task Force to start editing a new document (with Tab Atkins, Shane Stevens and Brian Kardell) called CSS Parser and Tokenizer API. It does not say it's going to be accepted or anything, but at least it went beyond the 5 minutes chat and rebuttal. I am extremely happy and proud :-) If we succeed making it happen, this is going to be a game-changer for a lot of web applications!!!

W3C Team blogW3C Updates General Document License

W3C announced today an update to liberalize its general document license. The updated license —applied today to all documents the W3C has published under its general document license— permits the creation of derivative works not for use as technical specifications, and the excerpting of Code Components under the W3C Software License.

When writing Recommendations, we want to encourage contribution toward and implementation of standards. We also want to encourage consistent implementation of standards and limit the likelihood of confusion or non-interoperability from divergent versions of a single specification. The updated license works to balance these concerns. Accordingly, this update facilitates the re-use of code, including in packages licensed under the GNU GPL. It also grants clear permissions to enable those documenting software, writing guides and tutorials, and implementing specifications to use excerpts of W3C documents as authoritative source material. The copyright license does not permit the modification of W3C documents to create competing technical specifications.

This license update stems from numerous discussions in the Patents & Standards Interest Group, from experimentation in other groups such as the Second Screen Working Group and the HTML Working Group, and discussion of re-licensing of unfinished specifications. Recognizing that this change may not satisfy all users or use cases, we will continue to seek consensus on options that meet even more needs.

Planet MozillaAdvanced ARIA Tip #2: Accessible modal dialogs

One question that came up more and more in recent months is how to create an accessible modal dialog with WAI-ARIA. So without further ado, here’s my take on the subject!

An example

To lay the ground work for some terms we’ll be using, let’s look at an example. A dialog is basically a sub window of an application that asks a question, requires some input, or allows to set options etc. Desktop applications have dialogs of all shapes and sizes, but there are some unique properties that distinguish them from other windows. The primary one is that it usually cannot be moved out of the main application window. Another is that usually, when it is open, the application around it becomes inert, and you cannot do anything with it as long as the dialog is open. The dialog is what we call modal. Yes, there are also non-modal dialogs, but they are then more like different panels in the main application window rather than a dialog in the classic sense of the word.

Let’s look at a simple example: Open your favorite word processor, type in something or paste your lorem ipsum, and then just close the window. You’ll get a dialog that consists of the following:

  • A title that contains the name of the application or some phrase like “Save the document”.
  • Text that states that you haven’t saved the document and what you want to do now.
  • A series of buttons like “Save”, “Don’t Save”, “Cancel” etc.

Screen readers will speak the title, the text, and the currently focused button automatically when you do the above steps with it running. This is important, so keep that in mind. It automatically reads the title and the text.

In the following example, we’ll call the title the “label”, and the text the “description”.

The simple Confirmation dialog

In HTML5 markup, this dialog could look something like this.

<div id="myDialog" role="dialog" aria-labelledby="myTitle" aria-describedby="myDesc">
  <div id="myTitle">Save "untitled" document?</div>
  <div id="myDesc">You have made changes to "untitled.txt" that have not been saved. What do you want to do?</div>
  <button id="saveMe" type="button">Save changes</button>
  <button id="discardMe" type="button">Discard changes</button>
  <button id="neverMind" type="button">Cancel</button>
</div>

Let’s go through this step by step:

  1. The first line declares a container that encompasses the whole dialog. Since HTML5 still doesn’t have a proper dialog element that is supported everywhere and gives us what we want, we will not use that element due to possible unpredictability of browser implementations. Instead, we’ll solely rely on WAI-ARIA to give us the semantics we need. The role is “dialog”.
  2. The aria-labelledby references the ID of our title. This is important since that will give the accessible object for the dialog the proper title, or label, or accessible name, that is spoken first by the screen reader when focus moves into the dialog.
  3. The aria-describedby associates the main dialog text with the dialog as well, which will turn it into the accessible description of the dialog object upon later translation. AccDescription is additional information that enhances the name and is spoken after name, role, and states. This concept goes back to the very old Windows days and is supported by all modern screen readers. If screen readers find an accDescription for a dialog accessible, they’ll speak it. Linux and OS X have adopted this concept later.
  4. Below the actual divs that contain the title and description texts, there are three buttons. These are regular button elements with a type of “button” so they are considered valid even outside of a form element.

From this, there are some rules to be deduced:

  1. If you use the role “dialog”, you must use aria-labelledby to point to the element containing the visible dialog title. The aria-labelledby attribute must be on the same HTML element as the role “dialog” attribute.
  2. If your dialog has one main descriptive text, you must use aria-describedby to point to its element, also on the same element that has role “dialog”. You can also point to multiple text elements by separating their IDs with spaces.

One note about Safari on OS X: In my testing, I found that the dialog text would not immediately be read. Safari sometimes has the tendency to map aria-describedby in a funny way that makes the description end up as help text, spoken after a several second delay, or retrievable via Ctrl+Option+Shift+H. Apple should really change this to something that gets picked up as dialog text. The fact that VoiceOver reads dialogs correctly in, for example, TextEdit shows that it has principal provisions for that, it just needs to be mapped correctly from Safari.

The heavy-lifting

OK, that was the easy part. For the following, you must remember that WAI-ARIA merely provides semantic information. It tells the screen reader what the elements represent. It does not automatically introduce certain types of behavior. You have to use JavaScript to do that. The following sections describe what you need to do, and why.

Initial focus

When you open your dialog, for example by setting it from display: none; to display: block;, or by inserting its markup into the DOM, you have to give it keyboard focus. Focus should be on the first keyboard focusable element. In our example above, keyboard focus would go to the “Save changes” button. Opening the dialog will not cause the browser to set focus in there automatically, so you’ll have to do it manually in JS via the .focus() method. Setting focus will cause the screen reader to recognize the change of context and detect that focus is now inside a dialog, and will do its magic to read the title and description of the dialog automatically.

Trapping keyboard focus inside the dialog

Since your dialog is most likely modal, you will probably have something in place that ignores mouse clicks outside of it. You will have to do something similar for keyboard focus. The common way to move around controls is via the tab key. So you’ll have to make sure your keyboard focus never leaves the dialog. A few rules for this:

  1. Create a natural tab order by arranging elements in the DOM in a logical order. That will cause the tab order to flow naturally without you having to do much. If you have to use the tabIndex attribute for anything, give it a value of 0 so the element fits into the natural tab order.
  2. On the very last element of your dialog, add a special keyPress handler that traps the forward tab key to redirect it to the first element of the dialog. Otherwise, tab would jump outside the dialog into the surrounding document or to the browser UI, and we don’t want that.
  3. Likewise, add a keyPress handler to the very first focusable element inside the dialog that traps the shift+tab key combination so the backwards tab order also doesn’t leave the dialog.

Provide an escape route

As is customary in dialogs, there is usually a way to escape out of it without making changes. Often, this does the same as whatever the Cancel button does. You’ll have to provide the same functionality to your dialog. So, make sure pressing Escape will close your dialog, and usually discard any changes.

Restoring focus after closing the dialog

Of course, Escape isn’t the only way to close your dialog. You would also provide a button to accept changes and execute on whatever functionality you desire. In any case, after you close the dialog, you must!!!, you absolutely must!!! then set focus back to the element that opened the dialog, or any other useful element from which the user of your app will most likely continue working. Never ever let keyboard focus go into Nowhere Land after the user dismisses a dialog you opened. In the worst case, it lands on the document or somewhere else far away from where the user’s context was before they opened the dialog. There is nothing more frustrating than to have to spend 10 or more seconds trying to find one’s last place again from where one left off.

Other dialog cases

Of course, your dialogs will most likely be more complex than what I showed above. Here are some more hints on what to do in certain situations.

A dialog with only a single button

Sometimes, you’ll create a dialog just to let users know that something completed successfully. Just follow the rules laid out above with focus on the single Close or OK button, and make sure the title and description are fully wired up via ARIA attributes as shown.

A dialog without descriptive text

If you don’t have general descriptive text, well then that’s that. Just make sure your form inputs are properly labeled as is customary for HTML forms, and if you group things together, use fieldset and legend elements to provide context. Also, of course, you can always use aria-describedby on individual inputs or other focusable elements to provide additional contextual information that you visually put outside the main label for that element or such. It all comes down to listening to your dialog and seeing if you can still understand what you’re asking of the user.

Multi-tabbed dialogs

Of course these are possible in HTML as well! You’ll probably not have general descriptive text, but yes, you can have a tablist/tabs/tabpanels construct inside a dialog just fine. You may then have aria-labelledby on the tabpanel pointed to its tab, and aria-describedby pointing to descriptive text within that panel that may give some context. Screen readers will pick that up when you move into the panel as well. Initial focus will then go to either the first tab in the tab list with the tab panel already opened, or the selected tab with its associated tab panel opened, if the dialog had been opened before and you want to preserve that when the user comes back. Your choice.

What about role “alertdialog”?

Hmm… Uhh… Don’t use it. Seriously, I’ve been working with ARIA for over seven years now, and I still haven’t figured out why role “alertdialog” was ever invented. It has all the properties of a “dialog”, and yet inherits the aria-live attribute with a value of “assertive”. That will cause its label, or title, to be spoken by a screen reader immediately upon its appearance, interupting everything else.

But a correctly marked up dialog will already speak all relevant information upon initial focus, and there should not be a reason not to give a dialog keyboard focus when it opens, so having a dialog yell at the screen reader user would only make sense if one would not initially set focus to it. And that is, from a usability standpoint, generally not a good thing to do in my opinion. I know opinions may differ on this, but since you’re on my blog and I can give you my advice, that is: Don’t use role “alertdialog”.

Other examples

Other fellow accessibility evangelists have created dialog examples that work similarly, but also have a few differences to what I discussed above. Lisa Seeman’s dialog example, for example, sets focus to the initial heading that contains the dialog title, and doesn’t associate the text above the fake login form to the dialog description. Also, in my testing, while shift+tab was trapped, tab was not, so I could move outside the dialog into the browser UI.

HeydonWork’s example shows the technique of using role “document” inside the dialog to make screen readers switch back to their reading mode. Cleverly used, this technique can help to work in application and browse mode automatically wherever appropriate.

And the most complete and comprehensive implementation I’ve seen, and which worked in a lot of combinations I tested, is this example by Greg Kraus called Incredible Accessible Modal Window. Thanks, Sina, for the pointer, as I didn’t even know about this one when I wrote the initial version of this post.

Conclusion

Creating a modal dialog that is fully accessible requires some work, but it is not rocket science or even magic. The important things to be aware of have been highlighted throughout this article, like associating both a label and descriptive text which screen readers will then pick up. It is also important to always control where focus goes inside the dialog, and when it closes.

I hope that this introduction to the topic of modal dialogs on the web helps you create more accessible experiences in the future! And of course, your questions and comments are always welcome!

happy hacking!

Planet MozillaJfokus2015 – My keynote, HTML5 talk, screencasts, slides and the resources I mentioned

The last two days I spent at the waterfront conference center in Stockholm, Sweden attending Jfokus2015, Sweden’s biggest developer conference with 1,700 participants. To make this even scarier, I gave the opening keynote one day after joining a new company and a technical talk about HTML5 at a conference mostly frequented by Java developers.

jfokus closing keynote and audience

Jfokus is a big, seven track event. Many of the other talks covered frameworks, out-of-the-box solutions or big topics like smart homes, self-aware drones and other IOT matters. The organisation was very smooth and as it is the case with all Nordic events, the catering excellent.

The keynote: you don’t need another hero app, you need one that survives the Thunderdome

I was very humbled to hear that I’ve been chosen to give the opening keynote and I put a talk together that covers the current state of the app space and what developers can do to stay relevant in a world of declining install numbers and OS functionality takeovers.

There’s a screencast of the keynote on YouTube.

You can also read the keynote slides are on Slideshare

Resources I used and mentioned in the keynote:

HTML5 beyond the hype

My second talk was about the current state of HTML5. My goal for this talk was not to dazzle and impress with great new technology and experimental features. Instead, I wanted to remind people how much we have available to us across browsers these days. A lot of very basic, but also very useful functionality of HTML5 and CSS3 is now safe to use. The hype around HTML5 at his inception and the subsequent disappointment when browsers didn’t support it immediately made us forget about these features. In this talk, I wanted to remind people of what can be used and how it could improve existing web solutions – even in the enterprise space.

There’s a screencast of the HTML5 session to YouTube.

Many people asked for the slide deck, so here it is on Slideshare

The resources I covered in the talk are listed here, in case you want to follow up with them:

All in all, I very much enjoyed my time at Jfokus. The organisation is very efficient, it was fun to talk to people and good to see that me joining a big company didn’t change much the way people approached me. I hope I managed to inspire some people to play with tech they have not before and to tell their bosses about things their companies should be doing.

Footnotes

Updated: .  Michael(tm) Smith <mike@w3.org>