12427 2011-04-05 22:17:33 +0000 User interface of spatial media fragment URI in <img> or <video> resource, or in browser navigation 2011-08-04 05:35:19 +0000 1 1 1 Unclassified HTML WG LC1 HTML5 spec unspecified PC All RESOLVED WONTFIX P2 normal --- 1 silviapfeiffer1 ian ayg cdouble ian jackalmage mike philipj public-html-admin public-html-wg-issue-tracking raphael.troncy public-html-bugzilla oldest_to_newest 47171 0 silviapfeiffer1 2011-04-05 22:17:33 +0000 Problem: what is a browser to display when the browser navigates to a image or video resource with a spatial media fragment URI, or when the @src attribute of a <img> or <video> element contains a spatial media fragment URI Proposal: a cropped (spliced) image or video is displayed to the user by hiding the other pixels from the user. Section affected for image: http://www.whatwg.org/specs/web-apps/current-work/#update-the-image-data In particular, this section explains what the image element's image data is: "The resouce obtained in this fashion is the img element's image data." This should state that when the resource is obtained with a spatial media fragment, the img element's image data is restricted to the rectangular region defined by the spatial media fragment. Section affected for video: http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#concept-video-intrinsic-width The intrinsic width and height of a video element are influenced by provision of a spatial media fragment URI. The width and height are restricted to the width and height of the rectangle provided in the spatial media fragment URI. The browser further needs to remember the x and y offset from which that region is extracted. Further, for video, a definition of the video's image data is required similar to how there is a definition of the img element's image data. This is particularly important, since video data can be imported into canvas elements through the drawImage function, so it needs to be clear what the image data is. http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-drawimage 47172 1 jackalmage 2011-04-05 23:47:19 +0000 Isn't this up to the Media Fragments spec to define? The answer should be, imo, that a spatial fragment represents precisely that - a fragment of a larger resource. Only the fragment would be displayed. 47178 2 silviapfeiffer1 2011-04-06 01:13:39 +0000 (In reply to comment #1) > Isn't this up to the Media Fragments spec to define? > > The answer should be, imo, that a spatial fragment represents precisely that - > a fragment of a larger resource. Only the fragment would be displayed. The issue is that you cannot deliver just the fragment pixels, so it's up to the User Agent to restrict the image pixels to just the requested ones. Otherwise it would be simple. So, since the UA receives all the pixels, it needs to be specified what pixels the UA actually has to deal with. The Media Fragment spec already makes a recommendation, but is that sufficient for browsers to actually be assured of what pixels to work with from within a HTML5 page? In particular in a cross-browser consistent manner? 47183 3 silviapfeiffer1 2011-04-06 06:27:43 +0000 Note: related earlier discussion http://lists.w3.org/Archives/Public/public-media-fragment/2010May/0061.html 47184 4 silviapfeiffer1 2011-04-06 06:33:52 +0000 It is actually necessary to specify where a media fragment URI is executed and where not in a browser. For example in a download link such as in right-clicks, the UA can't execute the fragment, since that would require creation of a consistent new resource and not just several byte range requests (unless browsers actually repackage the content, which is certainly an option). 47185 5 cdouble 2011-04-06 06:51:32 +0000 (In reply to comment #4) > ...since that would require creation of a > consistent new resource and not just several byte range requests (unless > browsers actually repackage the content, which is certainly an option). Byte range requests wouldn't work for spatial media fragments would they? For the formats that immediately come to mind you'd have to download the actual resource and then crop. You can't byte range request into a spatial area. 47186 6 silviapfeiffer1 2011-04-06 07:17:33 +0000 (In reply to comment #5) > (In reply to comment #4) > > ...since that would require creation of a > > consistent new resource and not just several byte range requests (unless > > browsers actually repackage the content, which is certainly an option). > > Byte range requests wouldn't work for spatial media fragments would they? For > the formats that immediately come to mind you'd have to download the actual > resource and then crop. You can't byte range request into a spatial area. No, you cannot do byte ranges on spatial media fragments in general. Indeed, cropping is necessary, which would be browser-internal only. My above comment was also for temporal media fragments, where it is possible though probably not trivial to re-package the content as a shorter piece of audio/video. 47191 7 philipj 2011-04-06 08:44:41 +0000 What about the percent syntax #xywh=percent:25,25,50,50 ? If that does not map to a integer number of pixels, should rounding be applied and in which direction? I presume that videoWidth and videoHeight should also reflect the cropped dimensions, so the question applies there as well. I note also in passing that if Opera implements support for Media Fragment, there's no plan for supporting the xywh syntax, as you can already achieve cropping with CSS. If there are compelling use cases where CSS doesn't do a good job, then I might reconsider, of course. 47200 8 silviapfeiffer1 2011-04-06 12:52:28 +0000 (In reply to comment #7) > What about the percent syntax #xywh=percent:25,25,50,50 ? If that does not map > to a integer number of pixels, should rounding be applied and in which > direction? That should probably go into the media fragment spec. Good point to make - I would say always round up. > I presume that videoWidth and videoHeight should also reflect the cropped > dimensions, so the question applies there as well. Yes, indeed, so that has to go into the HTML spec (or at least refer to the media frag spec). > I note also in passing that if Opera implements support for Media Fragment, > there's no plan for supporting the xywh syntax, as you can already achieve > cropping with CSS. If there are compelling use cases where CSS doesn't do a > good job, then I might reconsider, of course. Link below has a use case.. (In reply to comment #3) > Note: related earlier discussion > http://lists.w3.org/Archives/Public/public-media-fragment/2010May/0061.html Sorry, I meant this one: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027581.html 47202 9 philipj 2011-04-06 13:01:48 +0000 (In reply to comment #8) > (In reply to comment #7) > > What about the percent syntax #xywh=percent:25,25,50,50 ? If that does not map > > to a integer number of pixels, should rounding be applied and in which > > direction? > > That should probably go into the media fragment spec. Good point to make - I > would say always round up. OK, raised in http://lists.w3.org/Archives/Public/public-media-fragment/2011Apr/0011.html > Link below has a use case.. > http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027581.html I'm not sure I understand this. What is "image sprite support" and why is cutting an image using the canvas 2d api not good enough? 47208 10 jackalmage 2011-04-06 15:56:15 +0000 (In reply to comment #9) > > http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027581.html > > I'm not sure I understand this. What is "image sprite support" and why is > cutting an image using the canvas 2d api not good enough? Consider the general case of CSS spriting, which is currently done in a hacky way with background-position. Spriting into an <img> is just one instance of this type of spriting that can't be done even hackily at present (without doing something like running script to use <canvas>). 47211 11 philipj 2011-04-06 17:43:38 +0000 (In reply to comment #10) > (In reply to comment #9) > > > http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027581.html > > > > I'm not sure I understand this. What is "image sprite support" and why is > > cutting an image using the canvas 2d api not good enough? > > Consider the general case of CSS spriting, which is currently done in a hacky > way with background-position. Spriting into an <img> is just one instance of > this type of spriting that can't be done even hackily at present (without doing > something like running script to use <canvas>). Is the goal of this to save bandwidth by putting several images into a single physical image? 47212 12 jackalmage 2011-04-06 17:51:08 +0000 (In reply to comment #11) > (In reply to comment #10) > > (In reply to comment #9) > > > > http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027581.html > > > > > > I'm not sure I understand this. What is "image sprite support" and why is > > > cutting an image using the canvas 2d api not good enough? > > > > Consider the general case of CSS spriting, which is currently done in a hacky > > way with background-position. Spriting into an <img> is just one instance of > > this type of spriting that can't be done even hackily at present (without doing > > something like running script to use <canvas>). > > Is the goal of this to save bandwidth by putting several images into a single > physical image? Not bandwidth, but connections, yes. Spriting isn't the best approach for saving connections (better is resource packages, or a reworking of the internet architecture like SPDY), but it's a decent stop-gap solution. 47227 13 philipj 2011-04-07 07:57:31 +0000 My concern with supporting #xywh for still images is that it would require rewriting code all over the place (<img>, CSS backgrounds/borders) that previously used the physical size of the image to also consider the fragment size. This is of course possible, but it's really a question if it's worth the implementation cost (and future bugs) to support these use cases, which IMO are rather marginal. If HTTP pipelining were completely inadequate, I'd have expected to see JavaScript libraries to work around this using CSS or canvas in common use. 47237 14 jackalmage 2011-04-07 15:46:40 +0000 (In reply to comment #13) > If HTTP pipelining were completely inadequate, I'd have expected to see > JavaScript libraries to work around this using CSS or canvas in common use. You do, in the form of CSS Spriting, which is very widespread and recommended by all major performance optimization guides. It's an ugly hack, though - background-position was never meant to be used in this way. I'm helping with a bug on an internal page right now where background-position rounds in different directions at certain zoom levels, causing the sprited image to appear to vibrate when you animate between the sprites - my best suggestion so far was to use <canvas> to sprite the images instead. ^_^ 49733 15 ian 2011-06-16 21:37:30 +0000 (In reply to comment #0) > Problem: what is a browser to display when the browser navigates to a image or > video resource with a spatial media fragment URI That's more or less out of scope of the HTML spec, no? > or when the @src attribute of > a <img> [...] element contains a spatial media fragment URI Are there any image MIME types that support the media fragment syntax? If not, then the answer is "nothing different". > or <video> For video, the spec just defers to the video format; if the video format supports the media fragment syntax then that'll be defined by the video format spec. In conclusion, this seems out of scope for HTML. EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document: http://dev.w3.org/html5/decision-policy/decision-policy.html Status: Rejected Change Description: no spec change Rationale: see above 54010 16 mike 2011-08-04 05:35:19 +0000 mass-move component to LC1