12586 2011-05-03 02:13:48 +0000 Suggest using Typed Array for ImageData pixels. Add a new 'buffer' attribute that references ArrayBuffer of the pixel data. Convert 'data' (CanvasPixelArray) to Uint8Array view of 'buffer'. 2012-02-29 23:32:09 +0000 1 1 1 Unclassified HTML WG LC1 HTML Canvas 2D Context unspecified Other other CLOSED FIXED http://www.whatwg.org/specs/web-apps/current-work/#canvaspixelarray canvas P3 enhancement --- 13800 1 contributor contributor bzbarsky cam ian jonas kbr mike public-html-admin public-html-wg-issue-tracking travil w3.org public-html-bugzilla oldest_to_newest 47856 0 contributor 2011-05-03 02:13:48 +0000 Specification: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html Section: http://www.whatwg.org/specs/web-apps/current-work/complete.html#the-canvas-state Comment: Suggest using Typed Array for ImageData pixels. Add a new 'buffer' attribute that references ArrayBuffer of the pixel data. Convert 'data' (CanvasPixelArray) to Uint8Array view of 'buffer'. Posted from: 99.20.208.90 User agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.205 Safari/534.16 47857 1 w3.org 2011-05-03 02:20:07 +0000 If necessary 'CanvasPixelArray' could be alias for Uint8Array type if it is believed that scripts are actually checking the type of CanvasPixelArray. This accomplishes a couple things: - consolidate binary array data types (focus browser optimization resources) - allows developers to create a Uint32Array from the raw ArrayBuffer to access whole pixels at a time. Typed Array views and the CanvasPixelArray definition are largely compatible (with the typed array definition actually having more power). 47858 2 bzbarsky 2011-05-03 02:28:41 +0000 Note that Uint8Array doesn't have the same overflow/rounding semantics that CanvasPixelArray does. In Gecko, CanvasPixelArray is implemented as a new typed array type called Uint8ClampedArray. 51721 3 ian 2011-07-29 01:30:21 +0000 It's not quite a typed array. For example, some of the TypedArray constructors don't make much sense, since the canvas pixel array is based on the underlying buffer and so the same dimensions can have different buffer sizes. It might make sense to define some sort of relationship, though. What's the implementation status of CanvasPixelArray in the various browsers, in terms of their relationship to TypedArrays? Does anyone else have an underlying ArrayBuffer like Gecko? bz: Is the TypedArrayness of CanvasPixelArray in Gecko visible to JS in any way? 51725 4 bzbarsky 2011-07-29 02:29:14 +0000 > bz: Is the TypedArrayness of CanvasPixelArray in Gecko visible to JS in any > way? Yep. The CanvasPixelArray is just a typed array, and you can do all the normal typed array stuff to it from JS: get the underlying ArrayBuffer, create other views on the data, construct subarrays, etc. In fact, there is no CanvasPixelArray interface in Gecko at all at the moment; if you ask the .data of an imageData whether it's instanceof Uint8ClampedArrays you should get back true. Furthermore, you can create Uint8ClampedArrays using the normal typed array constructors and then work with them. And since the .data of imagedata is not readonly in Gecko at the moment you can assign one of those (or some other typed array, or a regular array) to it. I'm not sure all of this necessarily makes sense from a spec point of view; just describing what Gecko does right now. 51754 5 w3.org 2011-07-29 14:13:34 +0000 Actually, I think the gecko approach makes a lot of sense from a spec consistency and usefulness perspective. Typed arrays are the native data format for 3D Canvas (WebGL) interaction. It seems consistent that 2D Canvas data formats would also be typed arrays. As I noted in comment #1 it would be really useful/efficient in some situations to be able to address 2D image data as whole pixels (Uint32Array or Uint32ClampedArray or whatever). Also, being able to move data back and forth easily between 2D Canvas and WebSockets, File API, etc seems like another plus from a spec consistency/simplicity perspective. 51800 6 kbr 2011-07-30 00:40:51 +0000 In WebKit, CanvasPixelArray is currently a distinct type from the rest of the typed arrays. It owns its storage, which is distinct from that of the Canvas's backing store, and it doesn't expose it in the form of an ArrayBuffer. It would definitely be possible to respecify it as Uint8ClampedArray and add aliases (i.e., CanvasPixelArray == Uint8ClampedArray) for backward compatibility. I'd be glad to see this unification for all of the reasons mentioned here. If we do go down this route, I would strongly prefer to add only Uint8ClampedArray to the typed array spec, and not clamped versions of the other typed arrays, to avoid an explosion of the typed array types. 52772 7 mike 2011-08-04 05:04:07 +0000 mass-move component to LC1 54667 8 ian 2011-08-11 03:12:52 +0000 Ok, Kenneth, can you add Uint8ClampedArray? Then I'll redefine the canvas in those terms. No need to make an alias, I doubt anyone is relying on the old name since not everyone implements it anyway. 56685 9 kbr 2011-09-13 01:09:28 +0000 I apologize for the delay. Uint8ClampedArray has been added to the typed array editor's draft: https://www.khronos.org/registry/typedarray/specs/latest/ Please send me any feedback, in particular any concerns over inheriting from Uint8Array (which made the IDL for both types slightly simpler). 56688 10 bzbarsky 2011-09-13 04:06:22 +0000 There's an open item on whether after clamping Uint8ClampedArray should round or truncate. 58495 11 kbr 2011-10-20 02:02:50 +0000 (In reply to comment #10) > There's an open item on whether after clamping Uint8ClampedArray should round > or truncate. (Sorry for the delay, I didn't see this comment until now.) After clamping, it should not matter; the values are guaranteed to be within the range that Uint8ClampedArray can represent internally. Rounding and truncation should produce the same results. Am I missing something? 58496 12 bzbarsky 2011-10-20 02:16:14 +0000 Yes, you are. Consider a computation that in fact always produces values in the range 0-255 but is subject to the usual floating point rounding errors. If you truncate, you will never get 255 out of it. For many computations the end result of this as the computation is iterated is for all the color channels to collapse to 0... So you really do want rounding. 58497 13 kbr 2011-10-20 02:33:55 +0000 (In reply to comment #12) > Yes, you are. Consider a computation that in fact always produces values in > the range 0-255 but is subject to the usual floating point rounding errors. If > you truncate, you will never get 255 out of it. > > For many computations the end result of this as the computation is iterated is > for all the color channels to collapse to 0... > > So you really do want rounding. Sorry for missing this on first glance. It looks like this was fixed with the [Clamp] tweaks in the September 27 update to the Web IDL editors' draft. See http://dev.w3.org/2006/webapi/WebIDL/#changes . 58545 14 bzbarsky 2011-10-20 14:59:07 +0000 Yeah, IDL [Clamp] does the right thing. If the typed array spec is defined via WebIDL, then we're probably all set. (Though if it does, it needs various verbiage about sets of supported indices and such that it seems to be missing.) 61362 15 ian 2011-12-09 23:26:48 +0000 EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document: http://dev.w3.org/html5/decision-policy/decision-policy.html Status: Accepted Change Description: part of the trunk merge Rationale: Concurred with reporter's comments. 64795 16 kbr 2012-02-29 23:32:09 +0000 (In reply to comment #14) > Yeah, IDL [Clamp] does the right thing. If the typed array spec is defined via > WebIDL, then we're probably all set. (Though if it does, it needs various > verbiage about sets of supported indices and such that it seems to be missing.) Sorry for letting this go so long, but the editor's draft of the typed array spec now defines the supported indexed properties. https://www.khronos.org/registry/typedarray/specs/latest/#7