JSON schema specification

[yathit]

IndexedDB indexes are defined procedurally during database opening sequence on upgradeneeded event. Creating and deleting index requires a lot of boilerplate code and needed to compare with existing indexes. Instead of defining index in procedural, it is easier to defined indexes in JSON in database connection and let underlying database to do the job.

For example

var schema = {
  version: 1, // optional,
  stores: [{
    name: 'author',
    keyPath: 'email',
    autoIncrement: false,
    indexes: [
      {
        name: 'born',
        keyPath: 'born',
        unique: false,
        multiEntry: false
      }
    ]
  }]
};

indexedDB.openFrom('name', schema).then(function (db) {
}, function (e) {
});

If version is not specified, version will be increment as necessary. Additionally, in this case, database connection will be reconnect for new version on the event of version change as illustrated in the following code snippet. This will solved a lot of pain point.

db.onversionchange = function(e) {
  db.close();
  indexedDB.open(db.name).onsuccess = function(newDb) {
    db = newDb;
  }
};

Some use case require to modify existing data. To allow for that db instance should add oldVersion property. Probably oldSchema property may needed to cover most use case.

[alekseykulikov]

I think this feature can be implemented in user land. IndexedDB API is low level enough to cover this case. For example I built idb-schema which can be combined with idb-factory for similar results:

import Schema from 'idb-schema'
import { open } from 'idb-factory'

const schema = new Schema()
.version(1)
  .addStore('books', { key: 'isbn' })
  .addIndex('byTitle', 'title', { unique: true })
  .addIndex('byAuthor', 'author')
.version(2)
  .getStore('books')
  .addIndex('byDate', ['year', 'month'])
.version(3)
  .addStore('magazines')
  .addIndex('byPublisher', 'publisher')
  .addIndex('byFrequency', 'frequency')

open('mydb1', schema.version(), schema.callback()).then((db) => {})

Handling of onversionchange should be done in app. For example having 2 tabs running on top of 2 different db versions can be an issue.

[brettz9]

As far as defining stores and indexes via JSON, I've made a suggestion that JSON Schema itself specify IndexedDB properties so that a store and indexes can be built out of such JSON Schema documents (as well as add constraints above and beyond those within IndexedDB), and also made a suggestion for allowing JSON Schema (and its IndexedDB features) to be incrementally upgraded by JSON-based means.

(And FWIW, in the case of incremental versioning of JSON without JSON Schema, I'm working on such code for db.js built on top of @alekseykulikov's idb-schema and hope to add as a separate library.)

As far as whether to add support for JSON-based schema upgrades within IndexedDB, there are, as intimated in the OP, limitations to what an upgrade can accomplish non-programmatically. For example, it may be possible that you have a string identifier stored in a number of places but with a version upgrade, you want to pad it with 0's up to a certain length. For this, the programmatic approach (as used inidb-schema with its addCallback method) would be required (and even the idb-schema approach is currently limited if you want to use promises within the callback since the upgradeneeded transaction is too short-lived as a result of issue #42 (FWIW, I've made some PRs, starting with # 12 to override onsuccess to allow promise-based "upgrade" content modifications to be made, but I hope such workarounds would become unnecessary, especially since it requires some hoops to simulate treating the onsuccess "upgrade" as though part of the single genuine upgrade transaction).

Although this can be done in user land, it would be handy to have out-of-the-box for the common use cases.

While the onversionchange trick mentioned in the OP may work in some cases, it could become problematic if the old code is still seeking to add data to a now-deleted store, etc. I think service workers would be required if you want to recover without a page refresh.

[brettz9]

Since the JSON Schema project was not interested in incorporating IndexedDB elements, here's an alternative approach which covers the OP use case along with providing an even greater justification for defining such a format...

1. First define a standard JSON-based serialization of structured clones along with an HTTP content type and headers to go with it (to indicate key at least if out-of-line or keyPath if inline)? This would be useful in-and-of itself as cloned data could be reconstructed as easily as, e.g., getting a full structured clone as opposed to JSON via XHR from a server. Other methods like PUT could be employed to allow RESTful updates to individual stores. The various HTTP Patch protocols (e.g., JSON Patch or JSON Merge Patch) could be leveraged to allow a well-known means of partially updating an IndexedDB store on a server (or even a client, I suppose if headers could go along with local URLs as per Support URLs into records #57).

   As far as a possible implementation format, FWIW, in IndexedDBShim, we are representing cyclic objects and cloneable built-in objects part of the SCA via the typeson format (specifically utilizing the structured clone representation in the typeson-registry).

2. Create another JSON format which builds on top of the individual store format to include all JSON-serialized store content (say within an array) as well as the necessary meta-information to reconstruct a full database (keyPath, indexes, version number, etc.).

   Normal REST principles could then be applied at either level. Perhaps some HTTP method could allow retrieval of a count of the stores or such.

3. Finally, upgrade paths for client-side as well as server-side use, across multiple versions could be specified by JSON Patch (or if people were inclined to start over to get a more compact but equally expressive format, I've given a sample of my own here). (JSON Merge Patch is nicely succinct and intuitive but not expressive enough to cover all cases.)

[inexorabletash]

Closing this out in favor of the more general #224

(I still don't think we should do this; it seems like putAll() #69 + libraries are a better solution.)

[inexorabletash]

Oops, forgot to actually close.