Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datastore: Namespace should be a dataset-level configuration #122

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .npmignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
test
coverage
regression
test.js
test.js
key.json
49 changes: 30 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,11 @@ If you're running this client on Google Compute Engine, you need to construct a
~~~~ js
var gcloud = require('gcloud'),
datastore = gcloud.datastore,
ds = new datastore.Dataset({ projectId: YOUR_PROJECT_ID });
ds = new datastore.Dataset({
projectId: YOUR_PROJECT_ID,
/* namespace is optional, if not provided the default is used. */
namespace: 'namespace1'
});
~~~~

Elsewhere, initiate with project ID and private key downloaded from Developer's Console.
Expand All @@ -85,6 +89,8 @@ Elsewhere, initiate with project ID and private key downloaded from Developer's
var gcloud = require('gcloud'),
ds = new gcloud.datastore.Dataset({
projectId: YOUR_PROJECT_ID,
/* namespace is optional, if not provided the default is used. */
namespace: 'namespace1',
keyFilename: '/path/to/the/key.json'
});
~~~~
Expand All @@ -98,12 +104,12 @@ TODO
Get operations require a valid key to retrieve the key identified entity from Datastore. Skip to the "Querying" section if you'd like to learn more about querying against Datastore.

~~~~ js
ds.get(datastore.key('Company', 123), function(err, entity) {});
ds.get(ds.key('Company', 123), function(err, entity) {});

// alternatively, you can retrieve multiple entities at once.
ds.get([
datastore.key('Company', 123),
datastore.key('Product', 'Computer')
ds.key('Company', 123),
ds.key('Product', 'Computer')
], function(err, entities) {});
~~~~

Expand All @@ -113,15 +119,15 @@ To learn more about keys and incomplete keys, skip to the Keys section.

~~~~ js
ds.save({
key: datastore.key('Company', null), data: {/*...*/}
key: ds.key('Company', null), data: {/*...*/}
}, function(err, key) {
// First arg is an incomplete key for Company kind.
// console.log(key) will output ['Company', 599900452312].
});
// alternatively, you can save multiple entities at once.
ds.save([
{ key: datastore.key('Company', 123), data: {/*...*/} },
{ key: datastore.key('Product', 'Computer'), data: {/*...*/} }
{ key: ds.key('Company', 123), data: {/*...*/} },
{ key: ds.key('Product', 'Computer'), data: {/*...*/} }
], function(err, keys) {
// if the first key was incomplete, keys[0] will return the generated key.
});
Expand All @@ -135,10 +141,10 @@ ds.delete(['Company', 599900452312], function(err) {});
// alternatively, you can delete multiple entities of different
// kinds at once.
ds.delete([
datastore.key('Company', 599900452312),
datastore.key('Company', 599900452315),
datastore.key('Office', 'mtv'),
datastore.key('Company', 123, 'Employee', 'jbd')
ds.key('Company', 599900452312),
ds.key('Company', 599900452315),
ds.key('Office', 'mtv'),
ds.key('Company', 123, 'Employee', 'jbd')
], function(err) {});
~~~~

Expand Down Expand Up @@ -177,14 +183,14 @@ stored as properties is not currently supported.

~~~~ js
var q = ds.createQuery('Company')
.filter('__key__ =', datastore.key('Company', 'Google'))
.filter('__key__ =', ds.key('Company', 'Google'))
~~~~

In order to filter by ancestors, use `hasAncestor` helper.

~~~ js
var q = ds.createQuery('Child')
.hasAncestor(datastore.key('Parent', 123));
.hasAncestor(ds.key('Parent', 123));
~~~

##### Sorting
Expand Down Expand Up @@ -225,19 +231,24 @@ You can generate IDs without creating entities. The following call will create
100 new IDs from the Company kind which exists under the default namespace.

~~~~ js
ds.allocateIds(datastore.key('Company', null), 100, function(err, keys) {
ds.allocateIds(ds.key('Company', null), 100, function(err, keys) {

});
~~~~

You may prefer to create IDs from a non-default namespace by providing
an incomplete key with a namespace. Similar to the previous example, the
call below will create 100 new IDs, but from the Company kind that exists
You may prefer to create IDs from a non-default namespace.

Initialize a `Dataset` object with a namespace and allocate IDS.
Similar to the previous example, the call below will create 100
new IDs, but from the Company kind that exists
under the "ns-test" namespace.

~~~~ js
var incompleteKey = datastore.key('ns-test', 'Company', null);
ds.allocateIds(incompleteKey, 100, function(err, keys) {
var ds = new gcloud.datastore.Dataset({
// ...
namespace: 'ns-test'
});
ds.allocateIds(ds.key('Company', null), 100, function(err, keys) {
});
~~~~

Expand Down
78 changes: 54 additions & 24 deletions lib/datastore/dataset.js
Original file line number Diff line number Diff line change
Expand Up @@ -55,28 +55,30 @@ var SCOPES = [
* @alias module:datastore/dataset
*
* @param {object} options
* @param {string} options.id - Dataset ID. This is your project ID from the
* Google Developers Console.
* @param {string} options.projectId - Dataset ID. This is your project ID from
* the Google Developers Console.
* @param {string} options.keyFileName - Full path to the JSON key downloaded
* from the Google Developers Console.
* @param {string=} options.namespace - Namespace. If none provided, the default
* namespace is used.
*
* @example
* ```js
* var dataset = new Dataset({
* id: 'my-project',
* projectId: 'my-project',
* keyFileName: '/path/to/keyfile.json'
* });
* ```
*/
function Dataset(opts) {
opts = opts || {};
var id = opts.projectId;

this.connection = new conn.Connection({
keyFilename: opts.keyFilename,
scopes: SCOPES
});
this.id = id;
this.id = opts.projectId;
this.namespace = opts.namespace || null;
this.transaction = this.createTransaction_();
}

Expand All @@ -88,17 +90,12 @@ function Dataset(opts) {
* @example
* ```js
* var query = dataset.createQuery(['Lion', 'Chimp']);
* var zooQuery = dataset.createQuery('zoo', ['Lion', 'Chimp']);
* ```
* @return {module:datastore/query}
*/
Dataset.prototype.createQuery = function(ns, kinds) {
if (!kinds) {
kinds = ns;
ns = '';
}
Dataset.prototype.createQuery = function(kinds) {
kinds = util.arrayize(kinds);
return new Query(ns, kinds);
return new Query(kinds);
};

/**
Expand Down Expand Up @@ -204,7 +201,7 @@ Dataset.prototype.runQuery = function(q, callback) {
* dataset.transaction(function(transaction, done) {
* // From the `transaction` object, execute dataset methods as usual.
* // Call `done` when you're ready to commit all of the changes.
* transaction.get(datastore.key('Company', 123), function(err, entity) {
* transaction.get(ds.key('Company', 123), function(err, entity) {
* if (err) {
* transaction.rollback(done);
* return;
Expand Down Expand Up @@ -235,25 +232,20 @@ Dataset.prototype.runInTransaction = function(fn, callback) {
* @example
* ```js
* // The following call will create 100 new IDs from the Company kind, which
* // exists under the default namespace.
* var incompleteKey = datastore.key('Company', null);
* dataset.allocateIds(incompleteKey, 100, function(err, keys) {});
*
* // You may prefer to create IDs from a non-default namespace by providing an
* // incomplete key with a namespace. Similar to the previous example, the call
* // below will create 100 new IDs, but from the Company kind that exists under
* // the "ns-test" namespace.
* var incompleteKey = datastore.key('ns-test', 'Company', null);
* // exists under the dataset's namespace.
* var incompleteKey = ds.key('Company', null);
* dataset.allocateIds(incompleteKey, 100, function(err, keys) {});
* ```
*/
Dataset.prototype.allocateIds = function(incompleteKey, n, callback) {
if (entity.isKeyComplete(incompleteKey)) {
throw new Error('An incomplete key should be provided.');
}
// TODO(jbd): Add namespace to keys.
var incompleteKeys = [];
var proto = entity.keyToKeyProto(incompleteKey);
for (var i = 0; i < n; i++) {
incompleteKeys.push(entity.keyToKeyProto(incompleteKey));
incompleteKeys.push(proto);
}
this.transaction.makeReq(
'allocateIds',
Expand All @@ -270,14 +262,52 @@ Dataset.prototype.allocateIds = function(incompleteKey, n, callback) {
});
};

/**
* @borrows {module:datastore/entity~Key} as key
*
* @example
* ```js
* var key = ds.key('Company', 123);
* ```
*/
Dataset.prototype.key = function() {
return new entity.Key(this.namespace, [].slice.call(arguments));
};

/**
* @borrows {module:datastore/entity~Int} as int
*
* @example
* ```js
* var anInteger = ds.int(7);
* ```
*/
Dataset.prototype.int = function(value) {
return new entity.Int(value);
};

This comment was marked as spam.


/**
* Helper function to get a Datastore Double object.
*
* @borrows {module:datastore/entity~Double} as double
*
* @example
* ```js
* var aDouble = ds.double(7);
* ```
*/
Dataset.prototype.double = function(value) {
return new entity.Double(value);
};

/**
* Create a new Transaction object using the existing connection and dataset.
*
* @private
* @return {module:datastore/transaction}
*/
Dataset.prototype.createTransaction_ = function() {
return new Transaction(this.connection, this.id);
return new Transaction(this.connection, this.id, this.namespace);
};

/**
Expand Down
59 changes: 24 additions & 35 deletions lib/datastore/entity.js
Original file line number Diff line number Diff line change
Expand Up @@ -93,15 +93,12 @@ var SIGN_TO_ORDER = {
*
* @example
* ```js
* var key = new Key('Company', 123);
* var key = new Key('ns', ['Kind, 123]);
* ```
*/
function Key() {
if (arguments.length > 1) {
this.path_ = [].slice.call(arguments);
} else {
this.path_ = arguments[0];
}
function Key(namespace, path) {
this.namespace = namespace;
this.path = path;
}

module.exports.Key = Key;
Expand Down Expand Up @@ -220,15 +217,16 @@ module.exports.entityFromEntityProto = entityFromEntityProto;
* ```
*/
function keyFromKeyProto(proto) {
var keyPath = [];
var ns = null;
if (proto.partition_id && proto.partition_id.namespace) {
keyPath.push(proto.partition_id.namespace);
ns = proto.partition_id.namespace;
}
proto.path_element.forEach(function(path) {
keyPath.push(path.kind);
keyPath.push(Number(path.id) || path.name || null);
var path = [];
proto.path_element.forEach(function(el) {
path.push(el.kind);
path.push(Number(el.id) || el.name || null);
});
return new Key(keyPath);
return new Key(ns, path);
}

module.exports.keyFromKeyProto = keyFromKeyProto;
Expand All @@ -243,7 +241,7 @@ module.exports.keyFromKeyProto = keyFromKeyProto;
*
* @example
* ```js
* var keyProto = keyToKeyProto(new Key('Company', 1));
* var keyProto = keyToKeyProto(new Key(null, 'Company', 1));
*
* // keyProto:
* // {
Expand All @@ -257,21 +255,19 @@ module.exports.keyFromKeyProto = keyFromKeyProto;
* ```
*/
function keyToKeyProto(key) {
var keyPath = key.path_;
if (keyPath.length < 2) {
var proto = {};
if (key.path.length < 2) {
throw new Error('A key should contain at least a kind and an identifier.');
}
var namespace = null;
var start = 0;
if (keyPath.length % 2 === 1) {
// the first item is the namespace
namespace = keyPath[0];
start = 1;
if (key.namespace) {
proto.partition_id = {
namespace: key.namespace
};
}
var path = [];
for (var i = start; i < (keyPath.length - start); i += 2) {
var p = { kind: keyPath[i] };
var val = keyPath[i+1];
for (var i = 0; i < key.path.length; i += 2) {
var p = { kind: key.path[i] };
var val = key.path[i+1];
if (val) {
// if not numeric, set key name.
if (isNaN(val)) {
Expand All @@ -282,14 +278,7 @@ function keyToKeyProto(key) {
}
path.push(p);
}
var proto = {
path_element: path
};
if (namespace) {
proto.partition_id = {
namespace: namespace
};
}
proto.path_element = path;
return proto;
}

Expand Down Expand Up @@ -342,8 +331,8 @@ module.exports.formatArray = formatArray;
*
* @example
* ```js
* isKeyComplete(new Key('Company', 'Google')); // true
* isKeyComplete(new Key('Company', null)); // false
* isKeyComplete(new Key(ns, ['Company', 'Google'])); // true
* isKeyComplete(new Key(ns, ['Company', null])); // false
* ```
*/
module.exports.isKeyComplete = function(key) {
Expand Down
Loading