Bruce LawsonReading List

This week’s reading list comes to you with a renewed vigour, and a zesty swagger in its stride, as several people sought me out at Google I/O to tell me how useful it is. Thanks for the feedback.

This is simply a list of stuff I’ve read this week, and posted links to over Twitter. However, I know that lots of real people don’t read Twitter all day, or it’s blocked at work, so this is an amalgamated list. Note that these links interested me, but I neither endorse them nor vouch for their accuracy.

Standards

Industry

Misc

  • Clippy.js – “Clippy.js is a full Javascript implementation of Microsoft Agent (AKA Clippy and friends), ready to be embedded in any website.”

Planet WebKitChrome 28 Beta: A more immersive web, everywhere

Today’s Beta channel release introduces several new developer features and a major under-the-hood performance improvement. Unless otherwise noted, changes apply to desktop versions of Chrome and Chrome for Android. We’ll be rolling out the update over the next few hours.

Fullscreen API on Chrome for Android

The Fullscreen API on Chrome for Android allows you to programmatically hide the browser UI and OS status bar. Just like on desktop, you can tell any piece of content to enter fullscreen mode by calling its webkitRequestFullScreen() function. The prefixed version will eventually be replaced by requestFullscreen(). Here you can see the Fullscreen API used in a zombie-inspired Chrome Experiment:


Faster page loads

Starting in today’s Beta, your apps get a free speed boost from Blink’s new threaded HTML parser. It has two under-the-hood performance benefits: (1) reduced jankiness by moving work off the main JavaScript thread, and (2) improved page loading speed through pipelining. Compared to the normal HTML parser, it loads DOM content about 10% faster and reduces the maximum stop time due to parsing by 40%.

Experimental new media features in Chrome for Android

In today's Beta, WebGL joins Web Audio and WebRTC as an option in about:flags in Chrome for Android. Used together or independently, these three features will allow you to create rich, powerful web experiences that work across device form factors. We’re still actively improving the implementations, but we invite you to start experimenting. To see them in action, watch the mobile web demo in the Google I/O 2013 keynote.

Deprecated features

The prefixed version of the Content Security Policy HTTP header is now deprecated, so please use Content-Security-Policy instead of X-WebKit-CSP. The prefixed version will still work for now, but future releases may not support it.

For Chrome Extensions, HTML-based notifications have been deprecated in favor of the new Rich Notifications Chrome API. Extensions developers who are using HTML notifications in their apps or extensions should migrate to the newer Rich Notifications API, as support for the existing createHTMLNotification() feature will stop working in a future release of Chrome.

Other developer features in this release
Visit chromestatus.com for a complete overview of Chrome’s developer features, and circle +Google Chrome Developers for more frequent updates.

For general information about what’s going on in Chromium and Blink, watch the recordings of the fireside chats with the Blink team and the Chrome team at Google I/O 2013.

Posted by Alexandre Elias, Software Engineer and Screen Space Conservationist

Bruce LawsonResponsive images – interim report

At Google I/O last week, I was hoping to hear Google’s thoughts on responsive images. Every developer I speak to wonders how to send the right image size to browsers, Google is obsessed with performance, Peter Beverloo and Paul Kinlan were explicit in their talk Mobile HTML: The Future of Your Sites that web sites should request the correctly-sized images. But how?

The <picture> element seems to be at an impasse; no-one from Google showed any enthusiasm (nor does any other browser vendor, seemingly). That’s OK; when I dreamed it up 18 months ago it was a strawman to get the conversation going in standards groups (I’d previously been told that there was no need for any standard, as everyone would have huge devices on reliable, fast wifi in the future).

We have the srcset attribute, with its wildly unintutive syntax, “ready for first implementation”, so supported nowhere yet. It’s disliked by oiks who actually make web sites and fails to satisfy the art direction use case that 27% of those oiks currently polyfill to address.

Google has proposed a Client Hints header with device resolution, viewport size, touch-enabled flag etc. The idea is that the server generates lots of different sizes of foo.png, and when a request comes for foo.png, the server looks at the Client Hints and sends the most appropriate size. Opera (and soon, Chrome) announces that it accepts webP images, so the server could intercept requests for foo.png and send foo.webp in its place for conforming browsers, thereby reducing bandwidth.

This server-side negotiation will be great for legacy pages as it means source code doesn’t need to be touched. I’ve heard it also touted as being much simpler to author, but I wonder about that; HTML isn’t hard to write, while implementing server-side image resizing and transcoding, intercepting headers, doing the negotiation and setting cache control is much harder for an HTML wrangler than writing HTML is. It also assumes everyone has full access to the server. Don’t forget that some uses of HTML (for example, ebooks) have no server.

But those are all for the future. What can you do now? Clever, but ultimately unsatisfying workarounds.

Firstly, have <img> with a dummy source, then replace the src with JavaScript once you know the dimensions and resolution of your device. This makes images dependent on JavaScript.

Alternatively, make your images into single-frame <video>s, and take advantage of media queries on the child <source> element to send differently sized images. Ian Devlin dreamed this up, and it’s evil genius – except that file sizes increase and <video> doesn’t have an alt attribute, so the “images” are therefore inaccessible.

A similarly inaccessible technique for webkit browsers is simply to use a <div> element instead of <img> and set an image-set background-image in CSS. Don’t do this; it’s inaccessible, and is single-engine.

The current best way is brilliant albeit somewhat hacky: Estelle Weyl‘s clown car technique. This uses a data-encoded SVG file, embedded using an <object> element. Unfortunately, there’s nowhere to hang alt text, and right-clicking to “save as” doesn’t behave as expected.

So that’s the interim report. Inaccessible and/ or single-engine, or JavaScript-dependant, or relying on server-side shenanigans which don’t address the important art-direction use-case.

Images are central to the web; we need a declarative HTML method to achieve them in our multi-device world.

I’ve done my part in suggesting a strawman, and although cleverer minds than mine tell me it’s a bad solution, in a year and a half no-one has told me what a good one looks like.

Fingers crossed.

Anne van KesterenFetching URLs

There are a ton of features in the web platform that take a URL. (As the platform is build around URLs, that makes a ton of sense, too.) XMLHttpRequest, <img>, background-image, <script>, @font-face, … The semantics around obtaining a resource from such a URL however are not very well defined. Are redirects followed? What if the server uses HTTP authentication? What if the server returns 700 as status code for the resource? Does a data URL work? Does about:blank work? Is the request synchronous? What if I use a skype URL? Or mailto? Is CORS used? What value will the Referer header have? Can I read data from the resource returned (e.g. via the canvas element)? Can I display it?

For what seems rather trivial, is actually rather complicated.

At the moment Ian Hickson has defined some of this in the HTML Standard. In an algorithm named fetch. Then CORS came about (for sharing cross-origin resources) and the idea was that it would neatly layer on top, but it ended up rather intertwined. And now there is another layer controlling fetching, named CSP. To reduce some of this intertwinedness and simplify defining new features that take a URL, I wrote the Fetch Standard. It supersedes HTML fetch and CORS and should be quite a bit clearer about the actual model as well as fix a number of edge cases.

It is not entirely done, but it is at the point where review would be much appreciated.

Planet MozillaIAccessible2. New features and performance.

After months of long discussions the IAccessible2 group released 1.3 version. It was probably most disputable release after the initial shipping. A number of assistive technology vendors requested a number of changes for better integration with the browser. So it took a time to find a common multiplier for all ideas.

So updated IAccessible2 API require some engineering love and thus it will be implemented in Firefox by parts. It'd be helpful to know which features are most crucial and needed for you as AT developer so we can give them a priority. To let us know please leave a comment in the bug or just ping me saying that. Alternatively you can vote for or comment right into the interesting bug (see sections below).

Roles


IA2_ROLE_COMPLEMENTARY_CONTENT is supposed for HTML5 aside element and WAI ARIA complementray role (see Mozilla bug).

States


A new IA2_STATE_PINNED state will be exposed on pinned tabs in Firefox UI (see Mozilla bug).

Technically speaking IA2_STATE_CHECKABLE was introduced in place of hacks used to workaround a lack of this state. For example Firefox exposes checkable object attribute (IAccessible2 hack) and STATE_SYSTEM_MARQUEED (pure MSAA approach). See Mozilla bug.

Relations


There is a bunch of new relations introduced to simplify a navigation through the hierarchy. Now you can quickly move to

Note, IServiceProvider is an alternative to described relations (strictly speaking a subsest of described relations). See Mozilla bug.

Also now you can get a next and previous elements in tab order (IA2_RELATION_NEXT_TABBABLE and IA2_RELATION_PREVIOUS_TABBABLE). It's useful for AT that implement own in-page navigation (see Mozilla bug).

New IA2_RELATION_NODE_PARENT_OF is a dual relation to IA2_RELATION_NODE_CHILD_OF. It's useful to get all logical children of the accessible (as you know a logical parent-child relation may be different from parent-child relation defined by accessible tree). Example is aria-owns (like aria-owns tree) and flat structured hierarchies like XUL tree (used for message list in Thunderbird) or ARIA tree where the hierarchy is supplied by aria-level like:

  <ul role="tree">
    <li role="treeitem" aria-level="1">Item 1</li>
    <li role="treeitem" aria-level="2">Item 1.1</li>
  </ul>

See Mozilla bug.

 

IAccessibleDocument


IAccessibleDocument is a new interface that allows you to get an anchor target, i.e. an element that the page was scrolled to after the page was loaded. Also an object of this interface is returned by relations like IA2_RELATION_CONTAINING_DOCUMENT and IA2_RELATION_CONTAINING_WINDOW. See Mozilla bug.

 

Manage the media content


The idea was to manage the media content by accessible actions (IA2 group got parsimonious for a new interface so the existing mechanism was reused).

If a screen reader starts, say, a subtitled video then it invokes IA2_ACTION_OPEN action. The browsers plays a portion of subtitled video and stops it until a screen reader invokes IA2_ACTION_COMPLETE action. This action is used to notify the browser that screen reader consumed the portion of played video subtitles, for example, it finished reading of them. If screen reader is no longer interesting to manage the media playback then it invokes IA2_ACTION_CLOSE action.

If more than one AT tries to manage the same media (i.e. AT that invoked 'open' action) then the browser holds on the video playback until all AT invoke 'complete' action. See Mozilla bug.

Performance


After IAccessible2 implementation by Firefox and AT it got clear that IAccessible2 API can be improved in certain ways to make the intercommunication between the browser and AT faster. So number of changes was introduced for performance matter.

Object attribute by name


Now you can obtain object attributes by name rather than get them all together and then parse the string (see Mozilla bug).

Relation by type


Now you have a method to get relation targets by type. You don't need to get all relations when you need a single one (see Mozilla bug).

Accessible containing the caret


You have a fast way to get a text accessible containing a caret in the subtree and the caret offset in  the text accessible. Previously the algorithm to find an accessible containing the caret failed on some cases. The idea of that approach was that a screen reader should crawl down the hierarchy asking for caret position until it reaches a deepest text accessible having the caret. It failed if you meet non text accessible on the way. As a workaround the screen readers navigated children. While the workaround worked well but it was really slow (see Mozilla bug).

Text attributes


The change is similar to object attributes. A light alternative to the method used to get all text attributes was added.  Now you can specify a string filter and get attributes you need (see Mozilla bug).

Get all hyperlinks


Usually AT needs all hyperlinks at once. Getting them one by one is unperformant (especially for out of process AT) and requires a cache implementation on the browser side. So a new method to get all hyperlinks has been introduced (see Mozilla bug).

IEBlogWeb Performance APIs Rapidly Become W3C Recommendations

The W3C Web Performance Working Group recently published three specifications as W3C Recommendations with full implementations from all major browser vendors, advancing developers’ ability to accurately measure the performance of Web applications and make the Web faster. Over the last three years, companies including Microsoft, Google, Mozilla, Intel, Facebook, and others have been working towards standardizing the Navigation Timing, High Resolution Time, and Page Visibility interfaces in the Working Group. Rapid adoption of these recommendations demonstrates what’s possible when the industry and community come together through the W3C.

To make the Web faster, developers need the ability to accurately measure the performance characteristics of Web applications and the ability to effectively use the underlying hardware to improve the performance of their applications. To solve these problems, the Web Performance Working Group worked on 15 different specifications that address those issues. The table below shows the maturity level of all the specifications currently edited by the Working Group.

Specification Editor’s Draft First Public Working Draft Last Call Candidate Rec Proposed Rec Rec
Navigation Timing Sept 2010 Oct 2010 Jan 2011 Mar 2011 July 2012 Dec 2012
Resource Timing Sept 2010 May 2011 Aug 2011 May 2012
User Timing Oct 2010 Aug 2011 Sept 2011 July 2012
Performance Timeline July 2011 Aug 2011 Sept 2011 July 2012
High Resolution Time Feb 2012 Mar 2012 Mar 2012 May 2012 Oct 2012 Dec 2012
Page Visibility Apr 2011 June 2011 July 2011 July 2012 Feb 2013 May 2013
Display Paint Notifications May 2011 June 2011 Feb 2012
Navigation Timing L2 (NEW) Jan 2013 Jan 2013
Efficient Script Yielding June 2011
High Resolution Time L2 (NEW) Apr 2013
Beacon (NEW) Mar 2013
Resource Priorities (NEW) Apr 2013
Navigation Error Logging (NEW) Apr 2013
Resource Error Logging (NEW) Apr 2013
Prerender (NEW)

The Navigation Timing, Resource Timing, User Timing, and Performance Timeline specifications help developers accurately measure the timing of the navigation of the document, fetching of resources on the page, and developer script execution. Prior to these APIs, this data wasn’t easily obtainable. Navigation Timing was published as a W3C Recommendation, and all major browser vendors support it. The other three interfaces are currently at the Candidate Recommendation stage awaiting two full implementations from browser vendors. IE10 is currently the only browser that implements all of these interfaces, however, other vendors are working on implementations.

To ensure these performance metrics are measured in the most accurate way possible, the High Resolution Time specification allows developers to measure operations with sub-millisecond accuracy. This interface not only benefits accurate measurements of performance metrics, but also allows better frame rate calculations and synchronization of animations or audio cues. This interface has been published as a W3C Recommendation, with all major browser vendors implementing the performance.now() method defined in the specification.

The Page Visibility API allows for programmatically determining the current visibility state of the page. Developers can use this data to make better CPU- and power-efficiency decisions, e.g., throttling down activity when the page is in the background tab. This specification has also been published as a W3C Recommendation, with all major browser vendors implementing it.

The Timing Control for Script-Based Animations, and Efficient Script Yielding specifications help developers write more CPU- and power-efficient Web applications. The requestAnimationFrame API, from the Timing Control for Script-Based Animations specification, allows for creating more efficient JavaScript animations. All browser vendors fully support this interface, with the Working Group actively working on publishing this specification as a Candidate Recommendation. The setImmediate API, from the Efficient Script Yielding specification, allows developers to efficiently yield control flow to the user agent and receive an immediate callback, efficiently leveraging the CPU. IE10 is the first browser to implement this interface.

This year the Working Group also started to look at new ideas, with editor’s drafts of those ideas currently being discussed in the Working Group. The Beacon API is intended to help scripts asynchronously transfer data to a Web server without blocking the unload event, which can negatively impact the perceived performance of the next navigation. The Resource Priorities API defines a means for Web developers to give the browser hints on the download priority of resources to help improve the page load time. As a corollary to the Timing specs, the Navigation Error Logging and Resource Error Logging specifications help developers understand the errors and availability of their applications. The Navigation Timing Level L2 specification adds High Resolution Time and Performance Timeline support to Navigation Timing, and High Resolution Time L2 specification adds Web Worker support. These are just some of the drafts the Working Group is currently defining, with more specification drafts on Prerender and other diagnostics areas forthcoming.

The W3C Web Performance Working Group is a great example of how quickly new ideas can become interoperable standards that developers can depend on in modern HTML5-enabled browsers. Together with industry and community leaders who participate in the Working Group, we hope to continue to make rapid progress on interoperable standards that will help developers make the Web faster.

Thanks,
Jatinder Mann
Internet Explorer Program Manager

Planet MozillaOpen licensing at the W3C

In response to the objection I made on behalf of Mozilla to the previous proposed charter of the HTML Working Group, the W3C has proposed a new charter that allows the group to experiment with an open document license for extension specifications to HTML. If you work for a member company of W3C, I'd encourage you to encourage your AC representative to support the open licensing experiment. Here's why:

W3C publishes specifications, generally written by members of the W3C's working groups, under W3C's copyright ownership [1] and licensed under the W3C Document License, which does not allow other parties to modify the documents. This means that if, for some reason, W3C is no longer the right place to standardize something, those who want to work on that standard elsewhere have to start over rather than starting from the W3C document.

I'm one of several people who would prefer that the W3C license its specifications in a way that allows others to modify and republish the documents (a license that allows forking). I want this to be possible so that if the W3C stops being the right venue for standardizing something, the work can continue elsewhere. I don't want this to be a regular occurrence, nor do I want those who do it to do it without appropriate credit to the specification's original authors. But I want it to be possible to do when it's necessary, so I don't want legal restrictions that prevent forking, only social ones.

I also want it to be possible so that the W3C can collaborate effectively with other venues (such as the WHATWG) that do publish documents under a forking license. Currently this sort of collaboration isn't possible because the W3C can and does take documents from elsewhere that allow modification (often with little credit), but the movement can't happen in the other direction.

For example, when Ian Hickson set out to write what has become HTML5, he had to start over from scratch because previous HTML specifications were not licensed in a way that they can be modified. Ian wrote a new HTML specification at the WHATWG under a license that does allow modifications, and the W3C is using this specification as the bulk of their HTML5 specification (often with little credit to its original author).

Having licenses that allow modification and forking is also a useful governance mechanism. Without such licenses, once W3C has become the place for a particular specification, it is under relatively little pressure to continue to be a good forum. Allowing modification puts a little more competitive pressure on W3C to remain a good forum for standardization. This will reduce the chance of the history of HTML standardization from around 1999-2006 repeating itself.

Some might counter that this lessens the point of standards. I disagree. Standards are useful where there is benefit from a common agreement on how things work. If that agreement still has more value than changing it, then the party modifying the specification won't change it. When the value of sharing a common standard isn't large enough to motivate people to make and keep an agreement, that probably means it isn't worthwhile to do so.

For these reasons, I (as Mozilla's representative to the W3C's Advisory Committee) objected to the previous draft charter of the HTML Working Group. While we'd like to see all the documents of the group published under a liberal license, our objection didn't ask for this. We asked only that the charter allow the group to experiment with liberal licensing on extension specifications. We hope this experiment will give other members of the W3C more experience in working in an environment with liberal licensing and make them more comfortable with extending the experiment to other documents in the future.

The new proposed charter is under review by the W3C's Advisory Committee (the committee consisting of one representative per W3C member company) through May 29. If you are a member of the Advisory Committee or know your company's representative on the Advisory Committee, I'd encourage you to support liberal licensing.

[1] Ownership is transferred through the membership agreement, the clause on "should the Submission be used as a contribution towards a W3C Activity" in the submission request, or the invited expert agreement.

Planet MozillaSix years revisited

Two years ago I blogged about how it had been six years since I wrote my first patch for Firefox. Today I get to say that it’s been six years since I started getting paid to do what I love, working for Mozilla. In that time I’ve moved to a new continent, found a wife (through Mozilla no less!), progressed from coder to module owner to manager, seen good friends leave for other opportunities and others join and watched (dare I say helped?) Mozilla, and the Firefox team in particular, grow from the small group that I joined in 2007 to the large company that will soon surpass 1000 employees.

One of the things I’ve always appreciated about Mozilla is how flexible they can be about where you work. Recently my wife and I decided to move away from the bay area to be closer to family. It was a hard choice to make as it meant leaving a lot of friends behind but one thing that made it easier was knowing that I wouldn’t have to factor work into that choice. Unlike many other companies I know there is no strict requirement that everyone work from the office. True it is encouraged and it sometimes makes sense to require it for some employees, particularly when starting out, but I knew that when it came time to talk to my manager about it he wouldn’t have a problem with me switching to working remote. Of course six years ago when I started I was living in the UK and remained so for my first two years at Mozilla so he had a pretty good idea that I could handle it, and at least this time I’m only separated from the main office by a short distance and no time-zones.

The web has changed a lot in the last six years. Back then we were working on Firefox 3, the first release to contain the awesomebar and a built-in way to download extensions from AMO. Twitter and Facebook had only been generally available for about a year. The ideas for CSS3 and HTML5 were barely written, let alone implemented. If you had told me back then that you’d be able to play a 3D game in your browser with no additional plugins, or watch videos without flash I’d have probably thought they were crazy pipe-dreams. We weren’t even Jitting our JS code back then. Mozilla, along with other browser makers, are continuing to prove that HTML, CSS and JS are winning combinations that we can build on to make the future of the web open, performant and powerful. I can’t wait to see what things will be like in another six years.

Planet MozillaGiving “image swivel” the vanilla web diet treatment

With my chapter for the upcoming Smashing Book 4 done and talking about “The Vanilla Web Diet” and some workshops in the making I thought it is a good idea to show in an example how to make a seemingly good solution better by rethinking it with the principles of the vanilla web diet in mind.

The solution I am targeting is the image swivel effect that was posted a few days ago on CSS tricks. I sent my solution to the author and he was very interested in getting more information as to the why. The effect as it stands works and does the job to inspire people to play with it. The comments showed that where lots of developers had a go at creating their own solutions fixing some of the issues I am about to cover here. My favourite probably is Eduardo García Sanz’s Pure CSS solution which to me is a good plan to achieve an effect like that if you don’t need touch support or keyboard access.


Before I get accused of “hating” – the code is OK as a demo to get creative juices flowing but it has many issues that would show up when used in production. It is also mouse-dependent and doesn’t work on touch devices. Therefore I’ll start by stating what I find troublesome about the published solution and then explain how to approach the problem to make it as simple and maintainable as possible. I also add support for mouse, keyboard and touch access and try to achieve the effect whilst not blocking people out.

If you prefer to see this as a screencast, here is one I recorded explaining the ideas, the issues I found with the original code and how my solution works. This is live and unscripted.

How much markup do we need?

Let’s start by looking at the solution that is shown in the CSS tricks article.
The HTML is the following:

<div id="faces">
  <div id="face-area">
    <div id="image-1" style="display: none;">
      <img src="/images/look-left-3.jpg">
    </div>
    <div id="image-2" style="display: none;">
      <img src="/images/look-left-2.jpg">
    </div>
    <div id="image-3" style="display: none;">
      <img src="/images/look-left-1.jpg">
    </div>
    <div id="image-4" style="display: none;">
      <img src="/images/look-center.jpg">
    </div>
    <div id="image-5" style="display: none;">
      <img src="/images/look-right-1.jpg">
    </div>
    <div id="image-6" style="display: none;">
      <img src="/images/look-right-2.jpg">
    </div>
    <div id="image-7" style="display: none;">
      <img src="/images/look-right-3.jpg">
    </div>
    <div id="the_faces_overlay">
      <div class="the_faces" data-number="1"></div>
      <div class="the_faces" data-number="2"> </div>
      <div class="the_faces" data-number="3"></div>
      <div class="the_faces" data-number="4"></div>
      <div class="the_faces" data-number="5"></div>
      <div class="the_faces" data-number="6"></div>
      <div class="the_faces" data-number="7"></div>
    </div> 
  </div><!-- END #face-area -->
</div> <!-- END #faces -->

Warning sign #1: Repeated HTML structures without logical connections

This is a very common mistake I see when starting an effect like that. We create something that contains the content and then something that gets acted upon to show the effect. Thus we lose the benefit of already interacting with the parent element and using event handling to find out where we are. I guess historically this is based on “CSS only solutions” that needed that kind of separation as you could not calculate or detect mouse position in CSS.

As we create two separate sets of markup for an effect we need to find a way to connect the element that was interacted with to the one we want to show. This means adding IDs to all the elements, classes to all the elements we interact with and a data attribute to tell which of the images to show. This is bad for maintainability. If we add an image, we need to also add an element to interact with. Great code is catered to making maintenance as easy as possible. Here we have a lot of dependencies to deal with when adding or removing an image.

Warning sign #2: Lack of semantic markup


The other gripe I have with this HTML is that it means nothing at all. As the order of the images is important, the right HTML construct to use here is an ordered list.

Warning sign #3: Lack of alternative content


Another big HTML mistake is adding images without an alt attribute. This means that screenreader users would get the file path of the images read out to them. Either provide a sensible alternative text or add a alt=”” to hide the image from screenreaders.

Warning sign #4: Dependency on the number of elements


One big warning sign to me here is that our effect is dependent on the number of images in the widget and that we need to have the data attributes and the IDs maintained together although they are on different elements.

The more elements we add means the more IDs we need to maintain. This is not what coding is about. Computers are good at calculating things for us.

Warning sign #5: Empty elements and inline styles

Whenever I see inline styles I know something went wrong. There is no point in them and if ever they should only be generated by code, not by humans. The same with empty HTML elements: you probably did some extra work that is not needed. HTML is there to contain content or provide interaction. If you have a lot of empty DIVs without an obvious templating use case, something went wrong.

Giving no power to CSS

The CSS of the solution is not much, and it doesn’t do much either. This is a shame seeing how much easier it is for a visual maintainer to change CSS rather than changing JavaScript.

body {
  background: #333 
}
#faces {
  height: 333px;
  width: 500px;
  margin: 0 auto;
  border: 8px solid white;
}
#face-area {
  height: 500px;
  width: 333px;
  position: relative;
}
#the_faces_overlay {
  position: absolute;
  width: 500px;
  top: 0;
  left: 0;
}
#faces .the_faces {
  height: 333px;
  width: 14.2857143%;
  float: left;
  margin: 0;
  padding: 0;
}

Warning sign #6: CSS dependent on the amount of elements

The glaring issue here is the “width: 14.2857143%;” which is calculated by dividing 100% into seven parts. This means that if you delete an image from the HTML, you also need to change the CSS width here. You should never be dependent on the amount of elements in your CSS, as those are prone to change. In this case especially there is no logical way to find out why this is the width. CSS calc() can at least make that obvious but in general it is a bad idea to create look and feel that is tied to a certain amount of elements.

Goldfish jQuery

The jQuery code to make the effect work is very short:

// Reveal the "center" image
var centerImage = $("#image-4").show();
// Bind hovers to each column
$(".the_faces").each(function() {
  $(this).on("mouseover", function() {
    $("#image-" + $(this).attr("data-number")).show();
  }).on("mouseout",function() {
    $("#image-" + $(this).attr("data-number")).hide();
  });
});
// Reset center image
$("#face-area").on("mouseleave", function() {
  centerImage.show();
}).on("mouseenter", function() {
  centerImage.hide();
});

It is, however, very demanding to the browser. Slowing down a browser can be done in many ways – the most damaging ones are heavy computation, accessing the DOM and lots and lots of event handling. The latter two is what we do here.

Warning sign #7: lack of data caching

We loop over all the elements with the class “.the_faces” and add a mouseover and mouseout handler to each of them. Every time these get fired, we read an attribute, create a string with it and access an element that has the ID of the string and show or hide it. Showing and hiding using the jQuery methods is another access to the DOM as it manipulates the display style property. We show and hide the “center image” upfront and also on another event hander on the overall parent element.

If we were to add touch and keyboard handlers we’d triple the amount of assigned event handlers as we apply them on each image.

I call this Goldfish code – we keep asking the browser for things we should already know. The widget interface we have here is static HTML - there is no loading of content, no changes in it. Therefore there is no point in continuously reading out what the data-number attribute is and ask the browser to find the element with a certain ID. Caching results is a very simple thing to do and the performance benefits are amazing.

Rethinking the solution the vanilla web diet way

I approached the solution by looking at what we need to do here:

  • We have a widget of a certain size with images in it
  • We have an unknown amount of images – it should be dead easy to remove or add one or replace them all
  • Moving the mouse over the widget should loop through the images, also touching the widget should do so and it would be nice to be able to flip forward and backward with the keyboard
  • Should things not work out it would be nice to have an display that still makes sense

The HTML - we don’t need IDs or data attributes

<a href="/productpage" id="rollover">
<ol>
  <li><img src="pics/IMG_0518.jpg" alt""></li>
  <li><img src="pics/IMG_0517.jpg" alt=""></li>
  <li><img src="pics/IMG_0516.jpg" alt=""></li>
  <li><img class="current" src="pics/IMG_0515.jpg" alt=""></li>
  <li><img src="pics/IMG_0519.jpg" alt=""></li>
  <li><img src="pics/IMG_0520.jpg" alt=""></li>
  <li><img src="pics/IMG_0521.jpg" alt=""></li>
</ol>
</a>

We should link this to something – after all a beautiful effect like that should react in some sale or deep-dive, right? Good thing is that in HTML5 links can contain other elements. So all we add is a link. This automatically gives us keyboard access to the widget – something otherwise we’d have to create by using roaming tabIndex.

As the order of the images is important, an OL is the right element to use. Each image has an empty alt attribute to ensure there is no hassle with screenreaders. Instead of defining which image to show as the first one in our JavaScript we keep this maintained in HTML, too, by adding a class of “current” to the image.

Showing and hiding with CSS

body {
  font-family: arial, sans-serif;
}
#rollover.js {
  display: block;
  margin: 2em;
  z-index: 3;
  position: relative;
  height: 270px;
  width: 200px;
  cursor: none;
}
#rollover.js img {
  width: 100%;
  position: absolute;
  top: 0;
  left: 0;
  visibility: hidden;
}
#rollover.js img.current {
  visibility: visible;
}

As our functionality is dependent on JavaScript, our styling should be, too. This we can achieve by adding a “js” class to the element when JavaScript is available and only apply the styles when needed. This CSS gives the widget a fixed size and positions all the images stacked inside it.

Instead of doing a hide() and show() in JavaScript, all we need to do is to apply a class of “current” to the element we need to show. All the hiding and showing is thus done in CSS which means that in the future we’d want to do other visual things with the “shown” and “hidden” images, all we need to change is the CSS.

(function(){
 
  if (document.querySelector) {
 
    var rollover = document.querySelector('#rollover');
    rollover.className = 'js';
 
    var images = rollover.querySelectorAll('img');
    var all = images.length;
    var width = rollover.offsetWidth;
    var ox = rollover.offsetLeft;
    var boundarywidth = width / all;
    var current = 0;
    var x = 0;
    var index = 0;
    var touched = false;
 
    var setcurrent = function(index) {
      if (images[index]) {
        images[current].className = '';
        images[index].className = 'current';
        current = index;
      }
    };
 
    var findindex = function(x) {
      index = parseInt((x - ox) / boundarywidth, 10);
      if (index !== current) {
        setcurrent(index);
      }
    };
 
    rollover.addEventListener('mousemove', function(ev) {
      if (!touched) {
        findindex(ev.clientX);
      }
    }, false);
 
    rollover.addEventListener('touchstart', function(ev) {
      touched = true;
    }, false);
 
    rollover.addEventListener('touchend', function(ev) {
      touched = false;
    }, false);
 
    rollover.addEventListener('touchmove', function(ev) {
      if (touched) {
        findindex(ev.changedTouches[0].clientX);
        ev.preventDefault();
      }
    }, false);
 
    rollover.addEventListener('keydown', function(ev) {
      var key = ev.char || ev.key || ev.which;
      if (key === 37) { index = index - 1;}
      if (key === 39) { index = index + 1;}
      if (index < 0) {index = 0;}
      if (index > all - 1) {index = all - 1;}
      setcurrent(index);
    }, false);
 
    if (rollover.querySelector('.current')) {
      for (var i = 0; i < all; i++) {
        if (images[i].className === 'current') {
          current = i;
          break;
        }
      }
    } else {
      setcurrent(current);
    }
  }
})();

Quite longer than the jQuery, but bear with me as this does a lot more and in a much less demanding fashion. We wrap our code in a closure to makes sure we don’t leave any nasty globals behind. That should always be the first step.

Then we test if the browser supports document.querySelector. This is the standard answer to jQuery’s $() and is supported by lots and lots of great browsers. It is not supported by old and outdated browsers, which is why it is a good idea to test for it. This means that old Internet Explorer versions will not get the effect but instead they get the images as a numbered list (as we made the CSS dependent on a class applied with JavaScript). This is good, as it means we don’t need to test on these old browsers, which is hard to do and frankly a waste of our time.

We get a reference to our rollover widget using document.querySelector() and add the “js” class to it. This hides all the images and sets up the look and feel of the widget – all maintained in CSS. No need to loop through a lot of elements or use inline styles to hide them.

Next we get references to all the DOM elements we need and calculate what we need to find out what to show when the mouse cursor or the finger is on a certain part of the widget.

First we get all the images and store them in images. This will not change while the page is open, and querySelectorAll() gives us references to all of them. We store the amount of images in all to compare against later on.

Next we do the thing dynamically that the original solution did by hand – calculate the width of the different strips of the interaction that shows and hides images. We do this by reading out the offsetWidth of the widget as it is defined in CSS, so we don’t know and don’t want to hard-wire any widths in our JavaScript. We find out how far left the widget is in the browser by reading the offsetLeft property and store that in ox. Then we calculate the width of the interaction boundaries by dividing the width of the widget by the amount of images and store that in boundarywidth.

This makes the widget flexible to change any time the CSS changes or the amount of images does. If you remove or add an image, the width of the boundaries is calculated newly. No need to change the CSS to reflect that width. We made this now maintainable simply by adding or removing list items.

We define the current shown image index as 0, preset an x variable as 0, define the current index of the detected movement as 0 and set touched to false.

The index will be the index of the image to be shown at a certain point in the interaction. The index of the currently shown one is be stored in current, x will be detected position of the mouse or finger on the screen and touched defines if the screen is currently being touched or not.

The setcurrent() function hides the last shown image and shows a new one by shifting the “current” class from one to the other. It then stores the new image index in current. This is a very simple way to show a new state in a collection of things that can only show one at a time. No need to ask the browser which one is visible when we can store this in a variable like current.

The findindex() function converts the detected horizontal position of the mouse cursor or the finger of the user into an index of our image array. All you need to do is to subtract the left position of the widget itself and divide the value by the boundarywidth. Convert it to an integer and compare it to the current index and if it differs, call setcurrent().

All that is left is to assign the event handlers to make the magic happen. The first is a mousemove listener on the widget that calls findindex() when no touch happened. The current horizontal mouse position is stored in clientX of the mousemove event.

Touch interaction needs to be initiated (at least in Chrome in my testing here) so we set the Boolean of touched to true when a touchstart happened and to false when touchend was detected.

When a user moves a finger over the widget the browser fires the touchmove event and the current horizontal position is stored in the clientX property of the changedTouches array. We only detect the first finger in this case.

Keyboard detection doesn’t give us a position on the screen, so all we do is manipulate the image index directly. We listen for a keydown event and check the code of the key that was pressed. If it is the left arrow we subtract one from the current index and in the case of the right arrow we add one to it. We ensure that the index stays in the allowed limits and call setcurrent().

The last thing to do is to show the current image. If there is one with the right class in the HTML we need to find out its index and we do that by looping over them until we find the right one. If there isn’t any we just apply the current class to the first image (as defined at the start of the script).

Many solutions to the same idea

I hope this gives you an idea of how to approach an effect like that when you want to put it in production. There are of course other ways of doing it, but I wanted to ensure a few things that get very often forgotten:

  • The whole effect is now generated from the HTML, so all you need to do to create a new swivel is add other images
  • The whole look and feel is defined in CSS and you can resize the widget without having to worry about the size of the different boundaries
  • It can be used with a keyboard, the mouse or on touch devices
  • The DOM interaction is kept to an absolute minimum which means the performance on low spec devices is much, much better
  • There is no jQuery dependency

Planet MozillaAccessible Mozilla: Tech overview of Firefox 22

Firefox 22 reached beta status (it will be released June 24). It's time to list accessibility improvements we made for this version.

ARIA


ARIA role="note" doesn't allow name from subtree (bug) anymore. The bug caused JAWS, for example, to announce role="note" content twice.

HTML


* HTML radio group position doesn't count hidden radio elements (bug). So if the page contains hidden input@type="radio" then a screen reader doesn't take them into account announcing the number of radios.

* HTML input@type="file" changed its hierachy. Now it contains a push button and a label. Be careful if you have dependences on this hierarchy (see bug).

* HTML5 header and footer has changed their mapping according to HTML spec:
footer element that is not a descendant of an article or section element. contentinfo role;

header element that is not a descendant of an article or section element.  banner role.

XUL


XUL label@value element now implements text interface (partially). You can obtain a text between offsets but you can't get it by words for example (refer to bug). XUL label is used wide in Firefox user interface (for example, in Options dialog). Let us know if you have problems with new implementation.

ATK


RELATION_NODE_PARENT_OF has been implemented. It's exposed for aria-owns markup and XUL trees (used in Thunderbird and Firefox bookmarks).

Text interface


As I wrote before we started text interface reimplementation. Firefox 22 got improved getTextAt and getTextBefore offsets at word boundaries. Note, in case of getTextAt we had to mimic WebKit behavior rather than follow the ATK spec to keep Orca working.

Attention. It might be important


* Document load event may be fired a bit later than we used to do that, it will be fired right after all events contained in the queue at the time when document got loaded (see bug).

* IServiceProvider interface is implemented as a tear off (bug).

W3C Team blogPerspectives on Encrypted Media Extension Reaching First Public Working Draft

The HTML Working Group has announced their decision to release a First Public Working Draft of the Encrypted Media Extension (EME) specification. A preliminary version of the document has been public for some time, prompting the Free Software Foundation and others to petition W3C not to publish this draft.

The purpose of this post is to inform the community that, while we welcome and value input from all parties, we intend to continue to work on content protection, and publish this draft.

The Requirement from our Community

I intentionally refer to "content protection." Different publishers use the Web differently, some choosing to make content available free of charge, others preferring to control access. Most people would agree that individuals and institutions in general should have the right to limit access to proprietary information, or charge for access to content they own. Against this backdrop, the W3C Director has established that work on content protection for the Web is in scope for the HTML Working Group. This would address a specific set of requirements on HTML that came to the HTML Working Group from the Web and TV Interest Group.

How W3C Develops Web Standards

It is useful to review the W3C process to develop Web standards. It is a consensus process whereby we bring together vast and diverse interested parties to collaborate and achieve consensus to address the never-ending ways in which the Web drives increased value to society. The key objective is to maximize interoperability and openness - values that have served us well. Once we receive requirements for enhanced functionality, we address those requirements in W3C Working Groups. Once a Working Group has been chartered with a particular scope, the group has autonomy in how it satisfies requirements within that scope. It is thus up to the HTML Working Group to do their best to address identified content-protection requirements with the ultimate goal of enhancing interoperability on the Web. At the current time, the only content-protection proposal put forth within the HTML Working Group is the EME specification. Other discussions about content protection and alternative solutions to the requirements are taking place in the Restricted Media Community Group, which could feed into the HTML Working Group standards effort.

It is typical at this early stage of development for there to be issues; EME is an early draft not a final Recommendation. The HTML Working Group will publish revisions, seek comment, respond to issues, and pursue consensus decisions, all part of the usual W3C process. All W3C specifications are developed under the W3C Patent Policy, with a goal of assuring that the final standards can be implemented on a Royalty-Free (RF) basis. The Working Group expects to see open source implementations of the EME specification.

The DRM Debate and How it Relates to Web Standards

Here is our understanding of why EME is a contentious specification, despite broad agreement that some form of content protection is needed for the Web. The EME specification defines Application Programming Interfaces (APIs) that would provide access to content decryption modules (CDMs), part of Digital Rights Management (DRM) systems. W3C is not standardizing CDM technology, but there is a concern that standardizing APIs could encourage CDM usage - which some view as being in opposition to open Web principles.

While this viewpoint is important to consider as part of the debate, we have heard multiple principled and practical arguments on both sides of the issue.

We all aspire for a rich Web experience. Principled arguments for content protection begin by pointing out that the Web should be capable of hosting all kinds of content and that it must be possible to compensate creative work. Without content protection, owners of premium video content - driven by both their economic goals and their responsibilities to others - will simply deprive the Open Web of key content. Therefore, while the actual DRM schemes are clearly not open, the Open Web must accommodate them as best possible, as long as we don't cross the boundary of standards with patent encumbrances; or standards that cannot be implemented in open source. It has also been noted that a number of widely deployed proprietary technologies are used with the Web, including some video codecs and technologies accessed through plug-in APIs. We are not supportive of proprietary video codecs but recognize that to have far-reaching standards that support interoperability it is essential to include connections to such proprietary elements, some of which may be replaced in time with open standards.

Some have argued that we should not standardize interfaces to CDMs which would have the effect of cordoning off protected content into its own walled garden. This would be a mistake. It is W3C's overwhelming responsibility to pursue broad interoperability, so that people can share information, whether content is protected or available at no charge. A situation where premium content is relegated to applications inaccessible to the Open Web or completely locked down devices would be far worse for all.

There have also been critiques about EME regarding its impact on open source software development, specifically the question as to whether it can be implemented in open source. It is worth noting that the proposed (non-proprietary) APIs would work equally well with proprietary CDMs as with non-proprietary content protection schemes that could be implemented in open source software. The latter might not offer the same degree of content protection - they could be breakable - and would rely more on social convention than on impenetrability. Candidly, we don't see these solutions as being acceptable to content owners for premium video today, but that could change in time, or it could be acceptable to others in the community with different content requirements.

Next Steps

W3C as an organization will weigh a variety of complex considerations to determine the right balance for the Open Web Platform. There are principles and practical arguments on both sides of the debate. If we engage with all, I believe we have a much better chance of success than if we do not. We invite the community to review the First Public Working Draft of the EME specification. We also invite those with alternate proposals for addressing the requirements to consider joining the HTML Working Group, or to discuss them in the Restricted Media Community Group.

Planet MozillaA few questions and answers about “mobile web” and sites vs. apps

I just got asked to provide a few answers for a survey amongst “mobile web experts” and thought it’d be good to re-use those here. So here goes:

What is the difference between a web site and a web application?


There are a few differences. On a very basic level applications are catered for doing things whereas web sites are more catered for offering content for consumption. Web sites started as structured, interlinked academical documents. Later on we added multimedia content to make them much more engaging but all in all they are fixed in their state.
Applications are more dynamic. They allow for customisation of the interface and store the state of what happens so that when you get back to it you can go on from where you left off.

The use case of an app should always be to do something with it. This could be as simple as voting on a how much you like a kitten photo and go as far as editing video content live in your browser or on your device. Basic examples would be a webmail client as an application and a pure image gallery to click through as a web site.

Web sites are static whereas web applications have atomic updates and in of themselves have a very small footprint as most of the content gets loaded subsequently and changes every time you use it.

All in all it is a sliding scale though as for example an image gallery can easily become an application if it allows you to upload your own images or edit and remix the existing ones in your browser. That is one of the main benefits of web technology – it is very flexible and allows for quick and simple changes to the final product without being hindered by a complex compilation, packaging and deployment process.

What kind of features should a web site have to be qualified as a web application?


Again, there are many things to consider. One main thing is that an app does one thing and one thing well. It is there to help you do something.

Technically it should behave like the fat client apps of old: it should retain my state and settings, allow me to customise the interface to my needs and it needs to work offline. The latter is not a technical necessity in terms of definition but to me crucial usability. Seeing how flaky our connections are – I am writing this on a plane – our apps should make people as effective as possible and this means we shouldn’t be dependent on a connection. The interface should be usable whilst we are off the grid and sync as soon as we go online.

Customisation and personalisation of the interface and interactivity to me make an application. This could just mean a game where I can change my character and get extras the more I play. A proper “web” application to me also should use the web whenever it can. For example I am very frustrated with playing a game on my phone and when I go to my tablet playing the same game my score and achievements aren’t synced although the device knows who I am. Why be online then?

A lot of badly designed web apps are just wrappers for content like a news feed. For example turning a blog into an app is a pointless exercise. It just adds the overhead of having to install the wrapper, update and uninstall it when I am fed up with the blog but doesn’t give me the “do something” part that defines an app.

If I don’t interact with it other than reading there is no point in making it an app. You lose a lot of the flexibility of the web when packaging HTML as native apps with the most important thing being opaque updates. An app that is hosted on the web can be patched and upgraded without the end user having to download megabytes of data. That is incredibly useful on slow or flaky connections. Instead of the whole app you only download the changes.

What is the difference between a mobile-friendly site and a mobile web app?


A mobile-friendly site is a web site that detects the capabilities of the device it is displayed on and doesn’t make assumptions about how big my screen is and that I have a mouse and keyboard or not. It runs in the browser of the device and is thus hindered by its limitations – which in the case of older Android devices are quite limited indeed. It caters to the mobile space in terms of display changes – single column display, larger fonts, larger buttons.

A mobile web app is an application built with web technologies running in an own, full-screen wrapper and not inside a browser. It takes advantage of all the things the current environment allows for. For example it stores content and itself locally instead of having to be online and requesting everything new every time you open it. It can access the special features of certain environments like swipe gestures, accelerometer for interaction or accessing the camera to get content in. Its purpose is for me to do things with it and not just visit it like I visit a web site.

What would you consider is the key feature that made HTML5 (as opposed to Flash and Java) the number 1 choice for developing mobile web apps?


Flexibility is the super power of HTML5. It is easy to make an HTML5 app adapt to a new environment and to make an interface that shows and hides content and functionality dependent on the capability of the device or environment. Both Java and Flash are not “web native”, meaning you need to initiate and execute an own code environment inside a browser before you see the results. This hinders interactivity of the containing document and the content of the Flash movie or Java Applet. Whilst both Java and Flash have incredibly good development tools and capabilities once they are available they are very demanding to the hardware they run on. That’s why iOS devices don’t have Flash. There is already a fully capable environment available – the browser – and executing another one inside it means using a lot of resources. On mobile devices this means shorter battery life and the device heating up quickly.

With HTML5 we have the opportunity to improve what mobile, web enabled devices already have to have – a browsing environment. These are available open source and for free (Firefox, Chromium and others) and developers can build apps without needing to sign up for one company and get their SDK to get started.

All environments have their uses and there are things that are better done in Flash than in HTML5. Betting on open technologies and browsers means though that you are very likely to be flexible enough to cater to the next environment around the corner. The web has always evolved and mutated around the needs of the market. That’s why multimedia in HTML5 is just another element of the document and not a black hole that can not interact with the rest of the browser or the document.

Where would you say the mobile web is heading? Do you see a future for the mobile web?


There is no mobile web. There is the web. Right now we start consuming it more and more on mobile devices. That is cool. And the web is totally ready for that as it is flexible enough to adapt.

If you use web standard technologies to build applications that expect a certain device, a fixed size of a screen, a special way of user interaction or expect a fast connection you are building a very limited and very quickly outdated piece of software.

Over the last years we should have learned that hardware is a commodity and susceptible to very sudden change. An amazing piece of hardware that is the hot new thing now can tomorrow be embarrassingly outdated.

When you build your web apps to only cater for that you try to be native code and that is a race you can not win. A lot of native apps are built to promote a new piece of hardware and to get people to upgrade. That is a very old technique of selling more products called in-built obsolescence.

Web apps should be beyond this. Our job is to give end users the best possible experience on the current devices but not make them a necessity. We did this mistake in the past which is why you see web applications that tell you that you need a “modern browser like Internet Explorer 6” and “at least 800×600 resolution”.

Native apps on high-end devices are doing really well right now, but I can foresee that people will get bored of having to buy a new phone every year just to get new functionality that is not that important when you consider the cost. The web will stay and always be the open alternative for those who want to consume and create on their own terms instead of being dependent on the success or goals of a certain company.

Planet MozillaThe Elusive Universal Web Bytecode

It's often said that the web needs a bytecode. For example, the very first comment in a very recent article on video codecs on the web says
A proper standardized bytecode for browsers would (most likely) allow developers a broader range of languages to choose from as well as hiding the source code from the browser/viewer (if that's good or not is subjective of course).
 And other comments continue with
Just to throw a random idea out there: LLVM bytecode. That infrastructure already exists, and you get to use the ton of languages that already have a frontend for it (and more in the future, I'm sure).
[..]
I also despise javascript as a language and wish someone would hurry up replacing it with a bytecode so we can use decent languages again.
[..]
Put a proper bytecode engine in the browser instead, and those people that love javascript for some unknowable reason could still use it, and the rest of us that use serious languages could use them too.
[..]
Honestly, .Net/Mono would probably be the best bet. It's mature, there are tons of languages targeting it, and it runs pretty much everywhere already as fast as native code
Ignoring the nonproductive JS-hating comments, basically the point is that people want to use various languages on the web, and they want those languages to run fast. Bytecode VMs have been very popular since Java in the 90's, and they show that multiple languages can run in a single VM while maintaining good performance, so asking for a bytecode for the web seems to make sense at first glance.

But already in the quotes above we see the first problem: Some people want one bytecode, others want another, for various reasons. Some people just like the languages on one VM more than another. Some bytecode VMs are proprietary or patented or tightly controlled by a single corporation, and some people don't like some of those things. So we don't actually have a candidate for a single universal bytecode for the web. What we have is a hope for an ideal bytecode - and multiple potential candidates.

Perhaps though not all of the candidates are relevant? We need to pin down the criteria for determining what is a "web bytecode". The requirements as mentioned by those requesting it include
  • Support all the languages
  • Run code at high speed
To those we can add two additional requirements that are not mentioned in the above quotations, but are often heard:
  • Be a convenient compiler target
  • Have a compact format for transfer
In addition we must add the requirements that anything that runs on the web must fulfill,
  • Be standardized
  • Be platform-independent
  • Be secure
JavaScript can already do the last three (it's already on the web, so it has to). Can it do the first four? I would say yes:
  • Support all the languages: A huge list of languages can compile into JavaScript, and that includes major ones like C, C++, Java, C#, LLVM bytecode, and so forth. There are some rough edges - often porting an app requires changing some amount of code - but nothing that can't be improved on with more work, if the relevant communities focus on it. C++ compilers into JavaScript like Emscripten and Mandreel have years of work put into them and are fairly mature (for example see the Emscripten list of demos). GWT (for Java) has likewise been used in production for many years; the situation for C# is perhaps not quite as good, but improving, and even things like Qt can be compiled into JavaScript. For C#, Qt, etc., it really just depends on the relevant community being focused on the web as one of its targets: We know how to do this stuff, and we know it can work.
  • Run code at high speed: It turns out that C++ compiled to JavaScript can run at about half the speed of native code, which in some cases outperforms Java, and is expected to get better still. Those numbers are when using the asm.js subset of JavaScript, which basically structures the compiler output into something that is easier for a JS engine to optimize. It's still JavaScript, so it runs everywhere and has full backwards compatibility, but it can be run at near-native speed already today.
  • Be a convenient compiler target: First of all, the long list of languages from before shows that many people have successfully targeted JavaScript. That's the best proof that JavaScript is a practical compiler target. Also, there are many languages that compile into either C or LLVM bytecode, and we have more than one compiler capable of compiling those to the web, and one of them is open source, so all those languages have an easy path. Finally, while compiling into a "high-level" language like JavaScript is quite convenient, there are downsides, in particular the lack of support for low-level control flow primitives like goto; however, this is addressed by reusable open source libraries like the Relooper.
  • Have a compact format for transfer: It seems intuitive that a high-level language like JavaScript cannot be compact - it's human-readable, after all. But it turns out though that JS as a compiler target is already quite small, in fact comparable to native code when both are gzipped. Also, even in the largest and most challenging examples, like Unreal Engine 3, the time spent to parse JS into an AST does not need to be high. For example, in that demo it takes just 10 seconds on my machine to both parse and fully optimize the output of over 1 million lines of C++ (remember that much of that optimization time would need to be done no matter what format the code is in, because it has to be a portable one).
So arguably JavaScript is already very close to providing what a bytecode VM is supposed to offer, as listed in the 7 requirements above. And of course this is not the first time that has been said, see here for a previous discussion from November 2010. In the 2.5 years since that link, the case for that approach has gotten significantly stronger, for example, JavaScript's performance on compiled code has improved substantially, and compilers to JavaScript can compile very large C++ applications like Unreal Engine 3, both as mentioned before. At this point the main missing pieces are, first (as already mentioned) improving language support for ones not yet fully mature, and second, a few platform limitations that affect performance, notably lack of SIMD and threads with shared state.

Can JavaScript fill the gaps of SIMD and mutable-memory threads? Time will tell, and I think these things would take significant effort, but I believe it is clear that to standardize them would be orders of magnitude simpler and more realistic than to standardize a completely new bytecode. So a bytecode has no advantage there.

Some of the motivation for a new bytecode appears to come from an elegance standpoint: "JavaScript is hackish", "asm.js is a hack", and so forth, but a new from-scratch bytecode would be (presumably) a thing of perfection. That's an understandable sentiment, but technology has plenty of such things, witness the persistence of x86, C++, and so forth (some would add imperative programming to that list). It's not only true of technology but human civilization as well, for example no natural language has the elegance of Esperanto, and our currently-standing legal and political systems are far from what a from-scratch redesign would arrive at. But large long-standing institutions are easier to improve continuously rather than to completely replace. I think it's not surprising that that's true for the web as well.

(Note that I'm not saying we shouldn't try. We should. But we shouldn't stop trying at the same time to also improve the current situation in a gradual way. My point is that the latter is more likely to succeed.)

Elegance aside, could a from-scratch VM be better than JavaScript? In some ways of course it could, like any redesign from scratch of anything. But I'm not sure that it could fundamentally be better in substantial ways. The main problem is that we just don't know how to create a perfect "one bytecode to rule them all" that is
  • Fast - runs all languages at their maximal speed
  • Portable - runs on all CPUs and OSes
  • Safe - sandboxable so it cannot be used to get control of users' machines
The elusive perfect universal bytecode would need to do all three, but it seems to me that we can only pick two.

Why is this so, when supposedly the CLR and JVM show that the trifecta is possible? The fact is that they do not, if you really take "fast" to mean what I wrote above, which is "runs all languages at their maximal speed" - that's what I mean by "perfect" in the context of the last paragraph. For example, you can run JavaScript on the JVM, but it won't come close to the speed of a modern JS VM. (There are examples of promising work like SPUR, but that was done before the leaps in JS performance that came with CrankShaft, TypeInference, IonMonkey, DFG, etc.).

The basic problem is that to run a dynamic language at full speed, you need to do the things that JavaScript engines, LuaJIT, etc. do, which includes self-modifying code (architecture-specific PICs), or even things like entire interpreters in handwritten optimized assembly. Making those things portable and safe is quite hard - when you make them portable and safe, you make them more generic pretty much by definition. But CPUs have significant-enough differences that doing generic things can lead to slower performance.

The problems don't stop there. A single "bytecode to rule them all" must make some decisions as to its basic types. LuaJIT and several JavaScript VMs represent numbers using a form of NaNboxing, which uses invalid values in doubles to store other types of values. Deciding to NaNbox (and in what way) or not NaNbox is typically a design desicion for an entire VM. NaNboxing might be well and good for JS and Lua, but it might slow down other languages. Another example is how strings are implemented: IronPython, Python on .NET, ran into issues with how Python expects strings to work as opposed to .NET.

Yet another area where decisions must be made is garbage collection. Different languages have different patterns of usage, both determined by the language itself and the culture around the language. For example, the new garbage collector planned for LuaJIT 3.0, a complete redesign from scratch, is not going to be a copying GC, but in other VMs there are copying GCs. Another concern is finalization: Some languages allow hooking into object destruction, either before or after the object is GC'd, while others disallow such things entirely. A design decision on that matter has implications for performance. So it is doubtful that a single GC could be truly optimal for all languages, in the sense of being "perfect" and letting everything run at maximal speed.

So any VM must make decisions and tradeoffs about fundamental features. There is no obvious optimal solution that is right for everything. If there were, all VMs would look the same, but they very much do not. Even relatively similar VMs like the JVM and CLR (which are similar for obvious historic reasons) have fundamental differences.

Perhaps a single VM could include all the possible basic types - both "normal" doubles and ints, and NaNboxed doubles? Both Pascal-type strings and C-type strings? Both asynchronous and synchronous APIs for everything? Of course all these things are possible, but they make things much more complicated. If you really want to squeeze every last ounce of performance out of your VM, you should keep it simple - that's what LuaJIT does, and very well. Trying to support all the things will lead to compromises, which goes against the goal of a VM that "runs all languages at their maximal speed".

(Of course there is one way to support all the things at maximal speed: Use a native platform as your VM. x86 can run Java, LuaJIT and JS all at maximal speed almost by definition. It can even be sandboxed in various ways. But it has lost the third property of being platform-independent.)

Could we perhaps just add another VM like the CLR alongside JavaScript, and get the best of both worlds that way, instead of putting everything we need in one VM? That sounds like an interesting idea at first, but it has technical difficulties and downsides, is complex, and would likely regress existing performance.

Do we actually need "maximal speed"? How about just "reasonable speed"? Definitely, we can't hold out for some perfect VM that can do it all. In the last few paragraphs I've been talking about a "perfect" bytecode VM that can run everything at maximal speed. My point is that it's important to realize that there is no such VM. But, with some compromise we definitely can have a VM that runs many things at very high speeds. Examples of such VMs are the JVM, CLR, and as mentioned before JavaScript VMs as well, since they can run one very popular dynamic language at maximal speed, and they can run statically typed code compiled from C++ about as well or even better than some bytecode VMs (with the already-discussed caveats of SIMD and shared-mutable threads).

For that reason, switching from JavaScript to another VM would not be a strictly better solution in all respects, but instead just shift us to another compromise. For example, JavaScript itself would be slower on the CLR, but C# would be faster, and I'm not sure which of the two can run C++ faster, but my bet is that both can run it at about the same speed.

So I don't think there is much to gain, technically speaking, from considering a new bytecode for the web. The only clear advantage such an approach could give is perhaps a more elegant solution, if we started from scratch and designed a new solution with less baggage. That's an appealing idea, and in general elegance often leads to better results, but as argued earlier there would likely be no significant technical advantages to elegance in this particular case - so it would be elegance for elegance's sake.

I purposefully said we don't need a new bytecode in the last paragraph. We already have JavaScript, which I have claimed is quite close to providing all the advantages that a bytecode VM could. Note that this wasn't entirely expected - not any language can in a straightforward way be transformed into a more general target for other languages. It just so happens though that JavaScript did have just enough low-level support (bitwise operations being 32-bit, for example) to make it a practical C/C++/LLVM IR compiler target, which made it worth investing in projects like the Relooper that work around some of its other limitations. Combined with the already ongoing speed competition among JavaScript engines, the result is that we now have JavaScript VMs that can run multiple languages at high speed.

In summary, we already have what practically amounts to a bytecode VM in our browsers. Work is not complete, though: While we can port many languages very well right now, support for other languages is not quite there yet. If you like a language that is not yet supported on the web, and you want it to run on the web, please contribute to the relevant open source project working on doing that (or start one if there isn't one already). There is no silver bullet here - no other bytecode VM that if only we decided on it, we would have all the languages and libraries we want on the web, "for free" - there is work that needs to be done. But in recent years we have made huge amounts of progress in this area, both in infrastructure for compiling code to JavaScript and in improvements to JavaScript VMs themselves. Let's work together to finish that for all languages.

Jeremy KeithBy any other name

I’m not a fan of false dichotomies. Chief among them on the web is the dichotomy between documents and applications, or more broadly, “websites vs. web apps”:

Remember when we were all publishing documents on the web, but then there was that all-changing event and then we all started making web apps instead? No? Me neither. In fact, I have yet to hear a definition of what exactly constitutes a web app.

I’ve heard plenty of descriptions of web apps; there are many, many facets that could be used to describe a web app …but no hard’n’fast definitions.

One pithy observation is that “a website has an RSS feed; a web app has an API.” I like that. It’s cute. But it’s also entirely inaccurate. And it doesn’t actually help nail down what a web app actually is.

Like obscenity and brunch, web apps can be described but not defined.

I think that Jake gets close by describing sites as either “get stuff” (look stuff up) or “do stuff”. But even that distinction isn’t clear. Many sites morph from one into the other. Is Wikipedia a website up until the point that I start editing an article? Are Twitter and Pinterest websites while I’m browsing through them but then flip into being web apps the moment that I post something?

I think there’s a much more fundamental question here than simply “what’s the difference between a website and a web app?” That more fundamental question is…

Why?

Why do you want to make that distinction? What benefit do you gain by arbitrarily dividing the entire web into two classes?

I think this same fundamental question applies to the usage of the term “HTML5”. That term almost never means the fifth iteration of HTML. Instead it’s used to describe everything from CSS to WebGL. It fails as a descriptive term for the same reason that “web app” does: it fails to communicate the meaning intended by the person using the term. You might say “HTML5” and mean “requires JavaScript to work”, but I might hear “HTML5” and think you mean “has a short doctype.” I think the technical term for a word like this is “buzzword”: a word that is commonly used but without any shared understanding or agreement.

In the case of “web app”, I’m genuinely curious to find out why so many designers, developers, and product owners are so keen to use the label. Perhaps it’s simply fashion. Perhaps “website” just sounds old-fashioned, and “web app” lends your product a more up-to-date, zingy feeling on par with the native apps available from the carefully-curated walled gardens of app stores.

In his recent talk at Port 80, Jack Franklin points to one of the dangers of the web app/site artificial split:

We’re all building sites that people visit, do something, and leave. Differentiating websites vs. web apps is no good to anyone. A lot of people ignore new JavaScript tools, methods or approaches because those are just for “web apps.”

That’s a good point. A lot of tools, frameworks, and libraries pitch themselves as being intended for web apps even though they might be equally useful for good ol’-fashioned websites.

In my experience, there’s an all-too-common reason why designers, developers, and product owners are eager to self-identify as the builders of web apps. It gives them a “get out of jail free” card. All the best practices that they’d apply to websites get thrown by the wayside. Progressive enhancement? Accessibility? Semantic markup? “Oh, we’d love to that, but this is a web app, you see… that just doesn’t apply to us.”

I’m getting pretty fed up with it. I find myself grinding my teeth when I hear the term “web app” used without qualification.

We need a more inclusive term that covers both sites and apps on the web. I propose we use the word “thang.”

“Check out this web thang I’m working on.”

“Have you seen this great web thang?”

“What’s that?” “It’s a web thang.”

Now all I need is for someone to make a browser plugin (along the lines of the cloud-to-moon and cloud-to-butt plugins) to convert every instance of “website” or “web app” to “web thang.”


Tagged with

Anne van KesterenMaking the web platform more suitable for “apps”

At Mozilla we’re trying to bring the web platform closer to what is taken for granted in the “walled gardens” of our time (Apple’s App Store, Google Play, and friends). A big thing we need to solve is offline. As applications, sites should just work without network connectivity. Some variant of “NavigationController” (the name is bad) will give us that, but we need to iterate on it more. And in particular we need to test it to make sure performance is adequate and the API simple enough.

We have an API for end-user notifications, but after the site is closed clicking the notification from the notification center will fail (what should happen?) and if there are multiple browsing contexts with the same site open there is also some ambiguity as to which should receive focus. The permission grant is per-origin, but a single origin can host multiple sites. Push notifications face similar issues. The site is not open, but a push notification for it comes in, where should it be delivered?

The idea we have been toying around with is a worker that could be fired up whenever there is some external event that cannot be directly managed by the site (e.g. when the site is not open). This idea is not new, Google suggested it long ago, but it did not take off. A change from their model would be to not make these workers persistent, but rather short-lived so they are not too wasteful. Part of the application logic would move to the server, and push notifications can be used to wake the worker (we have been using “event worker” as a name) to e.g. notify the user or synchronize state for when the user navigates to the relevant site next.

Planet Mozilla“just use technology $x” is a terrible answer to a question

A few days ago a vertical video went viral of the student Jeff Bliss telling his teacher off for being a bad teacher who just hands out materials without explaining anything.

And we all applauded him for his gall and his eagerness to learn and to point out the obvious flaws in the education system. Teachers are not paid enough, are under more stress to be seen as someone who has students with good grades rather than understanding and have to deliver course materials they don’t believe in but are forced to go through as those are the ones that are easy to mark.

Terrible, isn’t it? So why do people in our little world of web development constantly and voluntarily become this bad, bored and ineffective teacher?

What am I talking about? The thing I mentioned in large detail in my talk at BaconConf this year but here it is for the generation who wants things in 140 chars or as a cute image with large letters on it:

Whenever you answer a question of another developer with “just use $x” you breed an expert idiot. You did not teach them anything, you showed a way out of learning.

In my ongoing task to de-clutter my life I just un-subscribed from several communities on Google+, namely the HTML5 community and the JavaScript one. Not because I am not interested in these matters any more, quite the opposite: because I care much about these communities and all I found there is frustration and annoyance. Nearly every single question new developers have is answered in three ways:

  • Use jQuery – here is a plugin
  • Just Google for it
  • You’ll need to use framework $x / JavaScript and/or HTML5 is not good enough for that

None of these answers are satisfactory if you really want to help someone who needs to solve a problem and learn how to repeat the solution in the future. The latter in most cases is a plain lie and shows that you are blaming a technology for your lack of interest in it.

What gets answered far too quickly is the “how” – how to achieve a massively complex result (which yet has to be proven to be really necessary) without thinking about it or understanding the solution that you apply. We assume that is enough and that we are doing something good – we let a new developer have a positive experience of having something achieved very quickly and that will obviously entice him or her to learn more and go explore in more detail later on.

That is not necessarily the case. We showed people a shortcut and how to focus on the outcome and hope the step where they understand what they are doing comes later. Sadly in a lot of cases this never comes but it fills people with fake confidence that gets shattered once they have their first job interview in a real company who cares about what they build.

If you want to teach people, make them understand the “why” and let them find their own “how”. That is much harder work, but also much more rewarding when you see them grow and explore.

We do not teach people how to write by copying and pasting the style of other authors. We tell them about similes, metaphors, rhetoric devices, orthography and grammar rules. Why bother with that? We could just show them TXTSPK and explain that anyone who knows English will understand what they try to convey. The reason why we do it is that we teach the fun of playing with language and finding your own style.

“But I don’t have time for that – I just want to help someone solving their problem”

Is a very common, admirable, but misguided idea. What you do is advertise the solution that made most sense to you as you already solved the problem. You steal the learning experience away from the other person and the way we learn is our most personal asset and differs vastly from person to person.

If you don’t want to really teach and see people grow and learn on their own terms, please stop spouting truisms and “best quick solutions” in places where people come to learn. If all they want is for you to solve their issue, they should hire you to do so for them.

Don’t be the grumpy teacher you learned to first despise and later on pity. We can do better on the web.

Bruce LawsonReading List

Here’s your reading list for the next 2 weeks – I’m off to Google i/o where I’ll be speaking at the “Web Platform Fireside Chat” 3pm (to 3.40pm) on the Friday, Room 5. Come and say hi if you’re going to San Francisco (be sure to wear some flowers in your hair).

Standards’n'shit

Misc

Planet Mozilla#justcode

As developers we are incredibly lucky. We work in a very growing and immensely well paid market, our companies shower us with benefits, companies offer us jobs rather than having to send out hundreds of CVs on the off-chance and even the mass media and politicians start talking about “coding” being a skill everybody needs.

Quite some part of this success is based on the stubbornness we showed in the past. When we got a task to build something we didn’t give up on it and said it is impossible. Instead we went back in our corner and tried and failed and tried again with sparks flying and code explosions happening until we achieved what we wanted. Think Dr. Bunsen Honedew’s laboratory instead of Statler and Waldorf.

This gave especially the web a strange “hack it together” reputation that many people keep bringing up when it comes to replacing JavaScript for example with “more organised and professional” languages. But you know what? I really think when it comes to the web, this is its main strength.

The fun of coding

whimsy

As explained earlier in my Flash is not the enemy post, whimsy and spontaneous ideas is what made the web a larger media outlet than it was. It wasn’t the large sites that got non-technical people excited. It was the funny animation and short-lived game that you could mail to your friends.

Therefore I think it is important to celebrate this for yourself from time to time. Personally I find myself extremely lucky to have been at the right time (and moving around to the right places) when the web exploded into an offering of amazingly cool things and while I am sure as hell not proud of the code I had to write to get things done in the past, I am happy that I did and that I didn’t give up or wait until someone else solves my problems for me.

Having just taught a workshop on HTML5 at Industryconf I found that we are losing a bit on that. Attendees were worried that they need to learn a lot of libraries and find the right plugins to get started and once shown that they have the power to do most of what they want using the things browsers come with out of the box got quickly into enjoying themselves reaching new levels.

One thing I did with the attendees is a To-Do List App in plain HTML/JS/CSS (No sound).

This is what the Mozilla Webmaker Project is about – to get non-programmers excited about building things for the web. And it is incredibly exciting to see some of these events as a “professional”.

I think it is very important to never forget about the wonder we experienced the first time we made something show up on screen or wrote our first condition that printed out “is amazing” when you entered your name or “is boring” when it was another one.

Be fearless

A lot of times being creative means being fearless. Watching Bret Victor’s talks and seeing his Learnable programming course and Seb Lee-Delisle’s training courses they consist of one main thing – play with things and worry about them breaking later. Amazing results happen when the outcome and the input get as close together as possible – not when things happen using dozens of abstractions.

This does not have to be visual from the get-go though. The MPEG-1 decoder in pure JavaScript for example is pure byte-shifting but blew me away in its fearlessness of what could go wrong.

Go, code!

Why not have a go? Take 10 minutes, half and hour, an hour out of your life right now and use it to #justcode something, anything. Just play with an idea, put it on JSFiddle, Codepen, JSBin, Dabblet, or whatever other amazing tool we have right now and share it.

Don’t build a perfect plugin, don’t build a solution dependent on preprocessors and libraries. Go vanilla and just play with what we have in browsers today. CSS Animations and Transitions, Canvas, Audio and Video, HTML5 and Friends – we have so many cool toys to play with. Don’t explore the main use case either. Yes, Canvas is for putting things on the screen, but it is also about reading image data.

We got were we are by playing with things. Never forget this and never stop playing.

Planet Mozilla Reforming the W3C Advisory Board

I am running for the W3C Advisory Board (AB) for one of four open seats on a reform and openness platform. If your company or organization is a member of the W3C (World Wide Web Consortium), please ask your AC Representative to vote (W3C member-only link) for myself and Chris Wilson in support of more openness and reform.

In short, the W3C Advisory Board keeps track of W3C process issues (W3C member-only link) and ... manages the evolution of the [W3C] Process Document. It also advises the W3C Team on topics like licensing.

I have participated in a couple of Advisory Board meetings by invitation in the past. I was asked to contribute experience and opinions related to more rapid and more open standards development practices. Though cautiously optimistic at the time, I have to admit that I have not seen as much progress as I'd hoped or expected. Community Groups with a more open license are nice but insufficient exception. The recently revised HTML Working Group Charter is another (more on that below).

So with that experience, why am I running? Two big reasons.

First, in the recent W3C TAG election, everyone who was elected was from the reform slate. The remaining unelected reformer was subsequently appointed. Politically, it seems the W3C membership and leadership are now more supportive of reform than in the past, top down, starting with the Technical Architecture Group.

Second, there are additional reformers running for the AB who are dedicated to process simplification and more openness, in particular, more open licensing:

If have you have W3C Member Access, you can read all nominees' statements. I'll update this list as I receive public confirmation from fellow reformers.

From my experience interacting with the other existing/continuing AB members, I know that some of them do express a desire for simplification and are supportive of more openness, both in process and in the specifications we produce (e.g. with more open licensing).

If we are able to elect a set of reformers to the AB, they plus the existing reform-friendly members have a good chance of effecting real change in the AB, and thus potentially the W3C as a whole.

My official nomination statement is posted at W3C but as that link is member-only (no reason for it to be in my opinion), I'm republishing it here, openly since I wrote it. Pardon the third person perspective.

Tantek Çelik is Mozilla's web standards lead and a 15 year participant in the W3C. Having contributed to working groups such as CSS and HTML, he has helped edit and produce several key Recommendations for the open web platform including: CSS 2.1, Selectors, and CSS3 Color.

Tantek's contribution to open web standards began with leading the implementation of the Tasman rendering engine in Microsoft Internet Explorer 5 for Macintosh, a watershed achievement of solid CSS1, HTML4, and PNG 1.0 support in the year 2000.

In addition to W3C activities, Tantek participated in the development of vCard4 at IETF, contributes to the WHATWG, and co-founded the microformats.org standards community.

As a longstanding champion of pushing for increasing openness in the methods and means of open standards development, he has both helped working groups (like CSS) do so, and has written extensively on the topic, e.g.: http://tantek.com/2011/168/b1/practices-good-open-web-standards-development

Tantek hopes to bring his practical experience working with (and helping evolve) a variety of standards organization models and processes to the W3C Advisory Board, to improve and modernize W3C's processes accordingly.

Tantek holds B.S. & M.S. degrees in Computer Science from Stanford University and shares his thoughts on his personal website: http://tantek.com/

There are two big changes in particular that I'd like to help make happen:

  • More open licensing. W3C Community Groups are allowed to publish with an open license (though a bespoke one W3C came up with). Official W3C Working Groups are not. My goal here is to have W3C adopt a standard open license (e.g. CC0 or CC-BY) for all specifications moving forward. Other modern standards organizations and communities like the WHATWG and microformats.org have already done so (with CC0) to great benefit (and no apparent downside). Just last week a new HTML Working Group charter was proposed that allows for the possibility of publishing HTML Extension Specifications with CC-BY. This is an excellent first step. W3C as a whole can benefit from more open licensing.
  • More open process. 15 years ago the CSS Working Group conducted the vast majority of its technical discussions behind closed doors, member-only mailing lists. Today the opposite is true, CSS is developed in the open, on open IRC channel, open wiki, and open email lists. I believe we can and should do the same for the process of running the W3C itself. Transparency in how we create standards is an end in its own right and the more we practice it, the more confidence, support, and trust we can build in a broader community of the work that we do.

If you work for a member of the W3C, please consider asking your AC Representative (W3C member-only link) to vote for myself and Chris Wilson for the W3C Advisory Board (W3C member-only link). I'll update the list of reform candidates as I receive confirmation of their public positions, and link to their blog posts accordingly. Thanks for your consideration.

Footnotes

Updated: .  Michael(tm) Smith <mike@w3.org>