This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 24338 - Spec should have Fetch for Blob URLs
Summary: Spec should have Fetch for Blob URLs
Status: RESOLVED FIXED
Alias: None
Product: WHATWG
Classification: Unclassified
Component: Fetch (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: Unsorted
Assignee: Anne
QA Contact: sideshowbarker+fetchspec
URL:
Whiteboard:
Keywords:
Depends on: 23147 24998 25081 25987
Blocks:
  Show dependency treegraph
 
Reported: 2014-01-20 19:39 UTC by Arun
Modified: 2014-11-25 13:51 UTC (History)
5 users (show)

See Also:


Attachments

Description Arun 2014-01-20 19:39:10 UTC
The File API should have a "Fetch" for Blob URLs that uses fetch.spec.whatwg.org "fetch a URL" algorithm but applied to Blob URLs, and works alongside the "parse a URL" step described in Bug 17765.
Comment 1 Glenn Maynard 2014-01-29 17:08:44 UTC
We don't want a special separate "fetch" algorithm that everyone has to use to support blob URLs.  There's only one Fetch algorithm for all types of URLs, just as there's only one Parse algorithm for all types of URLs.
Comment 2 Arun 2014-01-29 17:33:11 UTC
It won't be a separate "fetch."  It'll be the same fetch.
Comment 3 Anne 2014-02-05 11:25:48 UTC
I can take care of this in Fetch. What I need for this from File API are synchronous and asynchronous read concepts for blobs.
Comment 4 Arun 2014-02-27 22:26:43 UTC
Done. 
http://dev.w3.org/2006/webapi/FileAPI/#reading-data-section

Anne can you review this before I close it?
Comment 5 Anne 2014-03-17 15:59:09 UTC
I filed bug 25081. Is the idea that Fetch defines the fetching?
Comment 6 Arun 2014-03-19 18:00:48 UTC
(In reply to Anne from comment #5)
> I filed bug 25081. Is the idea that Fetch defines the fetching?

Yes, per Comment 3, assuming that the read operation delivers to Fetch everything it needs.
Comment 7 Anne 2014-03-19 18:33:54 UTC
I will just use this bug for Fetch then.
Comment 8 Arun 2014-05-13 18:45:45 UTC
(In reply to Anne from comment #7)
> I will just use this bug for Fetch then.

I think Bug 25081 is mostly resolved pending review from you. I've added Bug 24998 as a dependency here too.
Comment 9 Arun 2014-06-02 20:16:37 UTC
File API now has the following pieces in place:

1. Asynchronous read operation that only queues tasks, doesn't "mix" synchronous bits with asynchronous bits (which was a problem/bug in previous editor's drafts), and ties in with error handling with failure reasons usable by other specifications. That's http://dev.w3.org/2006/webapi/FileAPI/#readOperationSection

2. Serializing a Blob URL, extracting identifiers, and extracting origins e.g. http://dev.w3.org/2006/webapi/FileAPI/#DefinitionOfScheme

3. Blob URL Store and Revocation List e.g. http://dev.w3.org/2006/webapi/FileAPI/#lifeTime

4. Request/response guidance: http://dev.w3.org/2006/webapi/FileAPI/#requestResponseModel

I'm pretty sure these are the last pieces to add to the Fetch spec.
Comment 10 Anne 2014-07-03 10:08:32 UTC
https://github.com/whatwg/fetch/commit/a53a57dfc66911686488111d6cd19297cdb5c354 landed the other day. Still need the actual extracting the body part somehow.
Comment 11 Anne 2014-10-30 17:50:00 UTC
http://dev.w3.org/2006/webapi/FileAPI/#readOperation looks good enough.
Comment 12 Arun 2014-11-18 20:50:25 UTC
Resolving per Comment 11
Comment 13 Anne 2014-11-19 11:43:42 UTC
Arun, I wonder if the current setup works after all... Sorry for the churn.

https://fetch.spec.whatwg.org/#basic-fetch sets response's body to a new body and sets the length of that body.

http://dev.w3.org/2006/webapi/FileAPI/#readOperation creates a new body and returns that (while pushing to it in the background).

I could reference #readOperation and then set length on the result and set response's body to the result, but that seems somewhat ugly, no?


It seems either #readOperation should also set length or #readOperation should take a body as argument to read the blob into.
Comment 14 Arun 2014-11-19 23:42:53 UTC
(In reply to Anne from comment #13)
> Arun, I wonder if the current setup works after all... Sorry for the churn.
> 
> https://fetch.spec.whatwg.org/#basic-fetch sets response's body to a new
> body and sets the length of that body.
> 
> http://dev.w3.org/2006/webapi/FileAPI/#readOperation creates a new body and
> returns that (while pushing to it in the background).
> 
> I could reference #readOperation and then set length on the result and set
> response's body to the result, but that seems somewhat ugly, no?
> 
> 
> It seems either #readOperation should also set length or #readOperation
> should take a body as argument to read the blob into.


OK, I think this read operation from FileAPI should set length, and not transmitted; length is the payload's available bytes, whereas transmitted is what portions of the payload are actually "transmitted."

So, the (asynchronous) read operation makes available a body with increasing length for response.

Is this closer right? Change is read operation in File API doesn't set transmitted; diffs are lines 881 and 891:

http://dev.w3.org/cvsweb/2006/webapi/FileAPI/Overview.html.diff?r1=1.272;r2=1.273;only_with_tag=MAIN;f=h

With the read operation defined here:

http://dev.w3.org/2006/webapi/FileAPI/#readOperationSection
Comment 15 Anne 2014-11-20 10:13:55 UTC
No, length needs to match size. We could rename transmitted to something more neutral I suppose that works for local resources as well. Not sure what though.
Comment 16 Arun 2014-11-20 14:45:46 UTC
(In reply to Anne from comment #15)
> No, length needs to match size. We could rename transmitted to something
> more neutral I suppose that works for local resources as well. Not sure what
> though.

OK, setting body's length to match blob's size is pretty easy to do.

But, should the read operation in File API touch 'transmitted' at all? In fact, is the 'read operation' what actually transmits the bytes to the requesting API?
Comment 17 Anne 2014-11-20 15:18:30 UTC
(In reply to Arun from comment #16)
> OK, setting body's length to match blob's size is pretty easy to do.
> 
> But, should the read operation in File API touch 'transmitted' at all? In
> fact, is the 'read operation' what actually transmits the bytes to the
> requesting API?

/transmitted/ is just an integer used by progress events in XMLHttpRequest. I assume we want progress events even for blob URLs. We could use a different term from /transmitted/ if you want. Perhaps /progress/?

What transmits the bytes is pushing them to body. And handing that body to the requesting API.

There's various ways of structuring this, I'm not sure what you think is best. My suggestions are in the last paragraph of comment 13.
Comment 18 Arun 2014-11-20 16:03:27 UTC
(In reply to Anne from comment #13)
> Arun, I wonder if the current setup works after all... Sorry for the churn.
> 
> https://fetch.spec.whatwg.org/#basic-fetch sets response's body to a new
> body and sets the length of that body.
> 
> http://dev.w3.org/2006/webapi/FileAPI/#readOperation creates a new body and
> returns that (while pushing to it in the background).
> 
> I could reference #readOperation and then set length on the result and set
> response's body to the result, but that seems somewhat ugly, no?
> 
> 
> It seems either #readOperation should also set length or #readOperation
> should take a body as argument to read the blob into.


We could set /length/ in the read operation to match blob's size.

But the problem with this is if the (asynchronous) read operation generates an error mid-read. In that case, you will only have a portion of /length/; body is returned asynchronously.

Or I could increment /transmitted/ and /length/ as bytes are pushed to body.

Which is better?
Comment 19 Anne 2014-11-20 16:23:37 UTC
Setting /length/ still makes sense. But we should signal the error state somehow I suppose. Domenic, any idea how the low-level streams primitive will do that?
Comment 20 Arun 2014-11-20 16:49:17 UTC
(In reply to Anne from comment #19)
> Setting /length/ still makes sense. But we should signal the error state
> somehow I suppose. Domenic, any idea how the low-level streams primitive
> will do that?

The read operation does signal error! I think Fetch should handle it if it does.

(BTW, this plumbing is what the filesystem: URL scheme will use as well.)
Comment 21 Anne 2014-11-20 16:55:00 UTC
(In reply to Arun from comment #20)
> The read operation does signal error! I think Fetch should handle it if it
> does.

It's just unclear how that works. You returned the stream you keep pushing bytes to and continue to run some set of steps in parallel. Then you terminate those steps with a failure reason, but how is the caller going to find out? If it's not indicated on the stream somehow it seems that just goes lost.
Comment 22 Arun 2014-11-20 17:17:46 UTC
(In reply to Anne from comment #21)
> (In reply to Arun from comment #20)
> > The read operation does signal error! I think Fetch should handle it if it
> > does.
> 
> It's just unclear how that works. You returned the stream you keep pushing
> bytes to and continue to run some set of steps in parallel. Then you
> terminate those steps with a failure reason, but how is the caller going to
> find out? If it's not indicated on the stream somehow it seems that just
> goes lost.


OK, I can make this better.

1. In other algorithms or steps on the platform, we use "If x returns failure, then..". Right now, read operation terminates with a failure reason. I can have it explicitly return the failure reason, or make the return steps better, so that Fetch can say "if read returns failure..."One way to do this is to return body, but if a failure reason occurs, return that.

2. Or, we can have something set on the body itself in addition to /length/ and /transmitted/. I'd propose a property on body that helps determine if operations on body are failures, including a msg that accompanies the failure reason.
Comment 23 Anne 2014-11-20 17:25:07 UTC
Yeah, something on body itself seems like the only thing that would work. And as body is a stream, I wondered what Domenic had done there, but we could proceed with exposing some kind of "terminate reason" on body, just as we have for responses.
Comment 24 Domenic Denicola 2014-11-21 22:33:38 UTC
Streams have an errored state, and in the errored state they have an error reason (which should typically be an instanceof Error). That sounds kind of perfect for what you want?
Comment 25 Arun 2014-11-24 18:15:59 UTC
Resolving this. Fetch's body now has an error flag which is set by the read operation:

http://dev.w3.org/2006/webapi/FileAPI/#readOperationSection
Comment 26 Anne 2014-11-25 13:51:30 UTC
This was actually my bug :-)

Anyway, fixed in Fetch: https://github.com/whatwg/fetch/commit/ecf06611c36b4beae4a50fd039957dd8f7238a22