Support autoIncrement and indexes properties

[brettz9]

I would like to see standardization on the following properties for any given schema (especially the root): autoIncrement and keyPath, as allowed for stores in IndexedDB. Perhaps these could be within a store-metadata property or such.

I'd also like to see an indexes property which could standardize on at least three subproperties (as used in IndexedDB indexes), unique, multiEntry, and keyPath (and I'd hope the i18n work (I still intend to expand on this in PR #12 ) would hopefully also tie into Mozilla's locale property so that such a property would not need to be added in this location as well). Perhaps a name property could also be provided to allow for complete store and index generation.

Such properties could be used for auto-generating database stores based on the schemas, and one would not need to rebuild according to database implementation. IndexedDB is a particularly suitable choice, imo, since it is (at least seems it will be) ubiquitous in browsers, and can also be used as an API in server-side implementations (the most promising, imo, at least for Node, appears to me to be IndexedDBShim (with this PR), and possibly also for NoSQL implementations not having an IndexedDB API.

[Ixrec]

This sounds like a great use case for a schema, but I'm not seeing why any new JSON Schema keywords would be required to write such a schema.

{
    "type": "object",
    "properties": {
        "indexes": {
            "type": "array",
            "items": { ... }
        },
        "columns": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": { "type": "string" },
                    "autoIncrement": { "type": "boolean" },
                    "keyPath": { "type": "string" },
                    ...
                }
            }
        }
    }
}

What's wrong with something like this?

[awwright]

Under which conditions would this validate?

If these properties aren't for validation, it would probably be better suited as a different meta-schema altogether, like how we have JSON Hyper-Schema.

[brettz9]

@Ixrec : I'd like to see a specific syntax standardized because I think it would be a general enough use case for applications to be able to optionally process this (e.g., to import into a database--actually it'd probably be nice to have syntax to indicate the name of a database as well). While custom schemas are great for application-specific needs, IndexedDB storage is a very broad need, so it would be nice to be able to just pass over a schema or part of a schema to a general tool that could handle the db creation. I know one could publish one's own schema and library for this purpose, but I think it is something worth standardizing, if nothing else than for publicizing this possibility. I think it is instructive that even one of the founders of XML discouraged the creation of new XML languages and reinventing the wheel despite XML having been designed for new language creation: https://www.tbray.org/ongoing/When/200x/2006/01/08/No-New-XML-Languages

@ACubed : There are internal constraints for the values supplied within the schema: If there is a store keyPath, the schema type must be "object". If a store's autoIncrement is true, the store keyPath must not be an empty string or array. If an index's multiEntry is true, its keyPath must not be an array. keyPath (for stores or indexes) must abide by the following requirements: http://w3c.github.io/IndexedDB/#dfn-valid-key-path .

There would be some validation between a schema indicating a keyPath and an instance document (I think the following would be what should probably be applied: http://w3c.github.io/IndexedDB/#dfn-evaluate-a-key-path-on-a-value ), but in some cases at least, invalidity could be detected ahead of time within the JSON Schema schema alone.

But validation here would I think mostly not apply to validating a schema against a single instance document--it would more come into play when supplying a set of instance documents and the schema (i.e., a unique index would require that the indicated property be unique across all instance documents).

However, there is some need for normal validation with a type I neglected to mention in my OP: the need for a key type which itself would have certain restrictions on instances that would need to be validated: http://w3c.github.io/IndexedDB/#h-key-construct , but even here, its primary value would be validating against a schema AND the targeted instance document(s) (to ensure the key exists within the set of referenced document(s)).

[handrews]

This feels similar to an issue we dealt with at Riverbed where we wanted to use the same schema wherever a specific resource's representation appeared, but had trouble concisely expressing "this id field is auto-generated on create and read-only afterwards."

This just meant that the field should be omitted (and if supplied would be ignored) on a POST create, after which it behaves like any other readOnly field as expressed in hyper-schema.

I think that these keywords would clearly express those semantics. Like readOnly, it does feel more like a hyper-schema feature than a validation filter.

[awwright]

@handrews It seems most of these are pretty common problems to HTTP...

Ideally, the URI is the only part of the resource that's server managed. You create a resource with a POST to a resource-creating script, and a server allocates a URI within its namespace that it manages.

Sometimes the server needs to manage values inside resources too, though. You can treat this as semantically identical to a PUT operation performed immediately after a client performs an update.

I think a minimum requirement for considering adding validation-keyword behavior to JSON Schema is that it's an operation the server would reject with a 4xx client error.

So for things like giving resources a server-assigned ID (that's not a URI), maybe a serial/incrementing number, ISBN, UUID or something, then I would just configure the server to ignore client updates on that property. (Mayyybe return a 4xx if the client tries to change it... but this makes things harder for clients, any PUT they perform has to contain the existing ID that they might not otherwise even be interested in.)

A server must, at the very least, permit a client to GET a document, then PUT the exact same document back without problem. If a server is configured to send a property to a client, but prohibit that property when being sent to the server, I would argue you're actually dealing with two separate media types and therefore two separate JSON Schemas (if that is what's being used).

So I would think a "readonly" property here suffices... It indicates the property is only supposed to be updated by the server, and a client can read it, but there's no sense in trying to change it.

[handrews]

@awwright yeah, I can buy that argument, and I definitely like the "operation the server would reject with a 4xx client error" criteria.

What you describe with the '"readOnly": true' property and ignoring change attempts to such fields is essentially what we ended up doing (including using a readOnly field in the JSON schema, which I guess is technically only available in hyper-schema.

If a client attempted to update the auto-generated id, that was probably a 400 (I don't recall off the top of my head). But things like timestamps that are automatically updated on each request were just silently ignored if they did not match. That way the server could just blindly update the timestamp according to their documented (in the human-readable description) semantics, and clients didn't need to worry about stripping the timestamps out in GET-modify-PUT cycles.

Anyway, we now return to our regularly scheduled programming addressing @brettz9 's actual question :-)

[awwright]

Correct me if I'm wrong, but this doesn't seem to add constraints on JSON instances, it's strictly declarative. If this is just providing annotations for IndexedDB (and it only seems to affect IndexedDB), you could just write it into your IndexedDB handling.

Further, I think indexes should be resources unto themselves, not a property of collections. Constraints on collections could feasibly also be treated the same way, as a resource that describes an existing collection. You could write your own $schema vocabulary for this end.

[brettz9]

@awwright : There are a few constraints to JSON instances. See my comment at #17 (comment) .

Sorry I'm suffering from some health problems, so not able to get into involved discussions here atm. I think I've already made my case in the OP and comments though, so if you would kindly consider them (as well as consider the fact that some operations need not reach the server if they are being used offline, e.g., with IndexedDB), I say go ahead with your decisions.

[awwright]

Consensus seems to be that this sounds a little too specific to IndexedDB, so I think I'll close this. Something to consider instead would be writing a custom vocabulary, with a custom value for "$schema", that adds this functionality.

If you've got some example code, feel free to share that so I can take a look at it. IndexedDB with JSON Schema still happens to be something I'm interested in!

Hope you get better soon!