W3C/SMPTE Joint Workshop on Professional Media Production on the Web

Hey everyone. My name is Patrick, I'm a program manager at Microsoft and I'll be speaking to you today about the EyeDropper API.

Sampling colors creative application, is something that's very useful. For example, when you use something like PowerPoint and you want to change the color of the outline of an object, you're able to use an EyeDropper tool to go and grab the color from a different object so that you don't have to remember what color it was, or maybe remember the code by heart.

Developer tools in browsers also have that available. If you want to change a color property in the styles panel of dev tools, usually there's an EyeDropper icon that allows you to go and just click part of the webpage so that you can grab that color right away and not have to remember the hex code, for example.

Photoshop or Photoshop-like applications typically allow you to grab colors from the artwork as well directly.

Unfortunately, on the web, you cannot do this. So if you're developing a creative application with web technologies today, you cannot do this.

There's a way you can do this on Chromium browsers. If you use an input type color element in Chromium browsers, once you click on that element, you'll have a dropdown and it will contain an EyeDropper icon that allows you to do this.

But this is non-standard and it doesn't work the same in Firefox, for example, which has the typical or default color picker palette shown here.

Also the input type color is hard to style with CSS, and it's an extra HTML element that you might not want to add to your markup. It's better if you can drive that feature directly from JavaScript.

So that's why we created this EyeDropper API. It's a new API. It's specified right now as a WICG draft spec. It's gone through W3C TAG review and it's been implemented by engineers at Microsoft Edge in the Chromium open source project so that it's available to any Chromium-based browsers at the moment. This was also done in close collaboration with Google Chrome folks.

So let's see what the feature looks like. I'm going to go through a couple demos here. I have one of a kind of a silly game where you're supposed to find the color at the bottom that matches the little colored square at the top, using the EyeDropper. And you can either succeed by finding the right color or fail. And also if you go above the 10 seconds limit, you'll just go back to the normal cursor and you fail the round.

Here is another demo, which is a sort of a swapping color application where you can load an image in the canvas and then you can click on the first button to sample that color from the image. Let's say this color here, and then click on the second button to go and replace that color with another one that you sample from anywhere.

And then that third demo is a kind of a random Mondrian art generator, where you can change the color of the blocks to anything else that you want by clicking on that.

Now let's take a look at what the code looks like for those demos. And I'm going to use the code from the game demo because that's the one that uses the most features of the API.

At the heart of the EyeDropper API is the EyeDropper class. If you want to use the EyeDropper, instantiate the class then you get an object that you can call the open method on.

If you call the open method, right away the EyeDropper opens up and the user can select a color from the entire screen. The method returns a promise, which will either reject or resolve. It will reject when the user escapes from the EyeDropper mode, by pressing the escape button on the keyboard, for example. And it will resolve when the user clicks on a pixel in the screen and it will resolve to the value of the pixel color. There's also a way to abort the EyeDropper mode.

If at any time your application changes state in such a way that the EyeDropper should stop, you can send an abort signal to the EyeDropper, passing it to the open method as an option parameter. And then that will escape the EyeDropper mode.

And that's really all there is to the API. It's a very simple API at the moment. It only does one thing, but it does it very well.

Now there are concerns related to security and privacy here. If a random website is able to collect the colors of any pixels on my screen, that could be a problem. If they start tracking the coordinates of my mouse together with that, they could start revealing information that I'm not supposed to be exchanging on the web.

But the API addresses those concerns in the following ways. First of all, you cannot use the open method outside of a user action. There needs to be a user intent. So in response to the user click, for example, then you can use the open method. Also once the open method has been called, the browser is supposed to be showing the EyeDropper mode in a very, very obvious way, which is why you can see the big magnifying glass appearing. So that it's very obvious to the user that something's going on, that's not the normal cursor that typically appears.

Third, the API cannot collect the colors from any pixel as the user is moving the mouse. There has to be a user intent again, which is typically click on a pixel and that's when you can get the color, otherwise you can't.

And finally, the user is in control of the whole experience and they can still escape out of the EyeDropper mode by using the escape key on the keyboard.

So that's it for the API. We would love your feedback about it, specifically, do you think that the API should be extended further? Are there other capabilities that you have in mind for something like this? For example, multiple color selection or getting the screen coordinates or anything like that?

So I will leave you with the links to the spec, the demos, and where to leave feedback about this API. And thank you for your time and have a good day.