This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 12620 - 5MB isn't enough
Summary: 5MB isn't enough
Status: RESOLVED INVALID
Alias: None
Product: WebAppsWG
Classification: Unclassified
Component: Web Storage (editor: Ian Hickson) (show other bugs)
Version: unspecified
Hardware: All All
: P2 normal
Target Milestone: ---
Assignee: Ian 'Hixie' Hickson
QA Contact: public-webapps-bugzilla
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-06 05:42 UTC by Oscar Godson
Modified: 2011-06-01 18:20 UTC (History)
7 users (show)

See Also:


Attachments

Description Oscar Godson 2011-05-06 05:42:48 UTC
In regards to Disk Space:

The spec is amazing. It really is. It allows us, developers, to create some cool things without having to know complicated DB/SQL APIs. I know SQL and NoSQL DBs like Mongo and Couch and what I love about localStorage is that I personally can write my own wrapper and store data however I want. It's an empty bucket, or a blank piece of paper if you will. There is one issue with it tho. 5MBs is NOT enough to start in the long run. 5MBs is great for a cookie replacement, but localStorage can be so much more. With LS we can store base64 encoded media like images, video, and audio. Think about having a completely offline Photoshop that saves images as base64 encoded data!

Right now most vendors DONT allow us to increase the space even if we want to. I think 10MBs would be a better start and if they want to increase it, it should be a spec requirement to allow the end user to increase it.

Currently browser vendors say increasing it would be a bad idea because the end user has to wait for entire localStorage repo to load which is nonsense. They can make it an asynchronous API to the browser. A user sets or gets and item it then queries the browser to retrieve the data. Because it's all on the client side the responses will be almost instantaneous.

Please, give us more, or at the very least, more control over how much data we want to store.

Thanks! I'd love to hear your feedback. Here is a blog post i wrote about this issue back in January:
http://oscargodson.posterous.com/localstorage-and-where-it-failed

Oscar Godson
oscargodson@gmail.com
@oscargodson
Comment 1 Aryeh Gregor 2011-05-06 16:12:33 UTC
localStorage is, unfortunately, *not* an asynchronous API.  It's synchronous.  IndexedDB is being defined purely asynchronously to avoid the problem of blocking the page's functioning.

As I understand it, the bigger issue here is just that it's not practical to let every site in the world permanently use up dozens of megabytes of your disk space.  Browsers are planning to allow some way of permitting select sites to use more storage, but they're not sure what good UI for it is, so that the user can make an informed decision.  Ideally they'd draw some judgment based on things like how much disk space is available, how much the user visits the site, things like that.

But you really have to talk to the browser implementers about this, not the spec writers.  The reason the spec says 5 MB is because implementers weren't willing to commit to allowing all sites to use more than that.  The spec follows the implementations, not the other way around -- a spec is just a way for implementers to coordinate, not something that can make them do something they don't want to.
Comment 2 Oscar Godson 2011-05-06 18:02:07 UTC
Just as an FYI, i posted this here because the form on the Web Storage spec doesnt work. It says I don't have JS enabled, which obviously, I do. It then sent me here... it was a feedback on the disk space section. I can post it elsewhere, I just figured this was the kind of feedback you were looking for...? If not, what kind of info are you looking for in those forms if its not about the spec? Not at all trying to be sarcastic or an ass, actually curious.

Everyone says that it has to be synchronous but I honestly can't find that anywhere in the spec. It could be async easily and I actually don't understand why it couldnt?

Also, yes, obviously allowing 1GB or unlimited data on all sites wouldn't work, but im just saying give the choice to the user. If, for example, photoshop.com needed 10GBs of data I would give it to them as I need it for the site and I know what its being used for. Now, if i go to some random site after googling something and it asks to store that much I would say no. I dont see what the issue is if its the user making the choice. Arent we making sites and browser for our end users? IndexedDB, again, is nice and powerful, but so, so, so much more complicated than localStorage. I'll learn it as it looks super cool, but for most stuff it seems like its a little heavy.

Lastly, I know, I talked to the Chrome team awhile ago and, if you read my blog, they basically just called the entire idea of raising it even 5MBs stupid even tho all the other people aside from the Chrome team wanted it. They also said localStorage in general was just a stupid idea, so when I saw the suggestion box I thought it was the right place to do it. Sorry!
Comment 3 Aryeh Gregor 2011-05-06 19:12:29 UTC
(In reply to comment #2)
> Just as an FYI, i posted this here because the form on the Web Storage spec
> doesnt work. It says I don't have JS enabled, which obviously, I do. It then
> sent me here... it was a feedback on the disk space section. I can post it
> elsewhere, I just figured this was the kind of feedback you were looking
> for...? If not, what kind of info are you looking for in those forms if its not
> about the spec? Not at all trying to be sarcastic or an ass, actually curious.

Yeah, it seems to be broken.  I filed bug 12628 on it.  This is good enough, anyway.  The component is wrong, but Ian is the editor of both specs, so it doesn't matter much.  I'll change it to the right component for tidiness.

We're looking for feedback on the spec, yes.  However, a lot of what the spec says is there for a good reason.  There'd be no point in changing the spec to say a minimum of 20 MB if browsers didn't want to go along.  The 5 MB limit was probably the result of considerable discussion.  It's enough for basic usage without risking using up much of the user's disk.  Individual browsers can supply more than the 5 MB requirement, much more if they want.  But they're not likely to all have a much higher limit all the time, because it might use up too much disk space over time, since localStorage data isn't supposed to be deleted if possible.

There wasn't any way for you to know this, though.  Thanks for the feedback anyway.  The editor will get around to officially responding to it at some point.

> Everyone says that it has to be synchronous but I honestly can't find that
> anywhere in the spec. It could be async easily and I actually don't understand
> why it couldnt?

In JavaScript terms, synchronous APIs are ones where you do something and then the next line of code doesn't run until the first line is complete.  For instance, if you do

  localStorage["foo"] = 1;
  alert(localStorage["foo"]);

then it will alert "1" (if nothing else is modifying localStorage at the same time, like the same site in another tab).  For this to be reliably true, the first line must completely finish, with the data submitted to at least temporary storage, before the second line is run.  That's what we mean by "synchronous".

An asynchronous API in JavaScript is where you register an event handler.  For instance, code similar to the above in IndexedDB would be something like . . . well, I won't try to replicate it, too complicated.  But this should give the basic idea.  If store has been initialized to an object store:

  store.add({foo: 1}).onsuccess = function(event) {
    alert("foo has been set to 1");
  }
  alert("Some other code");

What this does is call store.add() to do the actual adding of the new value.  That does nothing except return an IDBRequest object immediately.  The code sets the onsuccess property of the object to a function that will be fired when the add() operation is successful.  Then it does nothing else at present, so it alerts "Some other code".  When the add() is successful, the browser queues a success event to fire at the object the next time it needs to run a task.  Then that alerts "foo has been set to 1".

The key thing here is that if it takes a while to do reads or writes or such, script doesn't stop running.  The event just won't fire for a while.  JavaScript in browsers is single-threaded, so if the browser gets stuck executing a particular line, all script has to freeze.  This typically means the whole page freezes.  With purely asynchronous APIs, nothing will freeze even if some event takes a while.

> Also, yes, obviously allowing 1GB or unlimited data on all sites wouldn't work,
> but im just saying give the choice to the user. If, for example, photoshop.com
> needed 10GBs of data I would give it to them as I need it for the site and I
> know what its being used for. Now, if i go to some random site after googling
> something and it asks to store that much I would say no. I dont see what the
> issue is if its the user making the choice.

This is a possibly valid argument, but what the specification says isn't likely to change what browsers do here.  For open-source browsers like Firefox or Chromium, you can try asking on a relevant discussion list.  Generally, browsers prefer not to ask the user to make too many choices, especially when most users have only a vague idea of what disk space is and can't give an informed answer.  Browsers will change when they have authors asking them to change.

> IndexedDB, again, is nice and powerful, but so, so, so much
> more complicated than localStorage. I'll learn it as it looks super cool, but
> for most stuff it seems like its a little heavy.

Yeah, it seems awfully hard to deal with . . .

> Lastly, I know, I talked to the Chrome team awhile ago and, if you read my
> blog, they basically just called the entire idea of raising it even 5MBs stupid
> even tho all the other people aside from the Chrome team wanted it. They also
> said localStorage in general was just a stupid idea, so when I saw the
> suggestion box I thought it was the right place to do it. Sorry!

There are a lot of implementers who are unhappy with localStorage and don't want to encourage its use too much, yeah.  The fact that it's synchronous (= can easily freeze your entire web page perceptibly every time it's invoked) has a lot to do with that.  Kind of a shame, since IndexedDB is horrendously complicated, but there you have it.  (Seems IDB has fallen into the "let's make a highly general and complicated API and hope authors paper over it with something usable" trap.)
Comment 4 Oscar Godson 2011-05-06 20:36:13 UTC
I know what async means in JavaScript, but for whatever reason it didn't cross my mind that _normal_ localStorage works like:

localStorage['foo']='bar';
localStorage['foo'] //return bar

I've been using a homebrew lib so long I think I just sort of forgot about underlying blocking nature of the API.

Changing it would be a hell of a task now after I saw it. Sure it wont do any good, but I'll post on their bug trackers with a possible solution where setItem() and stuff has callbacks like:

localStorage.setItem('foo', 'bar', function(data,err){
  try{
    console.log(data)//returns localStorage['foo'];
  }
  catch(err){
    if(err == 'QUOTA_EXCEEDED_ERR'){
      alert('Oops, you are out of space!');
    }
  }
})

...or whatever.

P.S. about the "alerts"... i dont think they are trying hard *enough* to hide those, actually! I get annoying alerts for:
WebKit Notification
Geolocation
Password
Password changes
Out of date plugins
...and more

Thanks for your feedback!
Comment 5 Aryeh Gregor 2011-05-06 20:56:36 UTC
Note: if you no longer think this bug needs the consideration of the editor, you can close it as RESOLVED INVALID or such so he doesn't have to read this whole conversation.