Storage module design and API for review

gistfile1.md

Storage

Motivation

There are 3 major types of storage is browsers up to date:

• IndexedDB. The future.
• localStorage. Limited in size and can be observed through the storage event.
• WebSQL. Dropped by the W3C but it's still very present in the wild, with notable platforms like iOS and PhoneGap.

Green: IndexedDB; blue: WebSQL; yellow: localStorage

In the mobile world local cache and storage is very important for various reasons:

• faster loading times
• unreliable connections
• offline use

My particular use case is an app to be used "in the field" where you probably don't have a connection and you need to do data entry. My solution is to save to a local storage and sync to the server when a connection is available. The 5 MB of localStorage may not be enough in my case, but I won't be able to tell until the app is used by real users.

Open questions

• Is there enough interest in this considering the availability of localStorage?
• Is it too early to use these technologies?

Architecture

The goal is to use conditional loading to load one of 3 or 4 sub-modules: indexeddb, websql, localstorage and a fallback to memory.

• IndexedDB uses a key-based storage organized in "object stores".
• WebSQL uses SQL tables. WebSQL can be used as a fallback for object stores by matching a table to a store and using tables with two columns: a key and a value column which saves the object serialized as JSON.
• IndexedDB uses structured cloning which is similar to JSON.parse(JSON.stringify(obj)).

Open questions

• Is falling back to localStorage a good idea considering the limited size and observability?
• Is the whole concept OK or is it too hackish?
• Should this include over-the-wire databases via some protocol?

API design

The API aims to find the lowest common denominator between all implementations. A database requires:

• A name. Required by both IndexedDB and WebSQL
• A version. Required by both IndexedDB and WebSQL
• A structure, which means the name of each object store, tied to the version. Required by both IndexedDB and WebSQL
• A size in bytes. Required by WebSQL

A Storage constructor receives all these parameters and creates an instance with properties with the names of each object store:

var storage = new Y.Storage({
  name: 'my database',
  version: 1,
  size: 5242880,
  stores: ['fooStore', 'barStore']
});

Assert.isObject(storage.fooStore); // true
Assert.isObject(storage.barStore); // true

A Storage instance also has a close method that closes the database connection/session.

A Store object is created for each requested store in a property of the same name. A store can perform the following actions:

• get. Retrieve an object by key
• put. Insert or update an object by key
• remove. Remove an object by key
• clear. Remove all objects from the store
• count. Count the objects in the store

All methods return a promise.

Example:

storage.fooStore.put('some key', { foo: 'bar' }).then(...);

Open questions

• Should Storage inherit from Base in order to use attributes for configuration and maybe fire close or error events? A versionchange event is very important.

Quite interesting "latency compensation".
Shouldn't YUI support this inside the core?