Skip to content

Commit

Permalink
Drop support of value types other than string and Buffer (#192)
Browse files Browse the repository at this point in the history
  • Loading branch information
vweevers authored Aug 18, 2019
1 parent b442455 commit f008114
Show file tree
Hide file tree
Showing 4 changed files with 24 additions and 78 deletions.
45 changes: 6 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Your data is discarded when the process ends or you release a reference to the s

## Data types

Keys can be strings or Buffers. Any other key type will be irreversibly stringified. Unlike [`leveldown`] though, `memdown` does not stringify values. This means that in addition to Buffers, you can store any JS value without the need for [`encoding-down`]. The only exceptions are `null` and `undefined`. Keys and values of that type are rejected.
Keys and values can be strings or Buffers. Any other key type will be irreversibly stringified. The only exceptions are `null` and `undefined`. Keys and values of that type are rejected.

```js
const db = levelup(memdown())
Expand All @@ -53,18 +53,17 @@ db.put('example', 123, (err) => {
valueAsBuffer: false
}).on('data', (entry) => {
console.log(typeof entry.key) // 'string'
console.log(typeof entry.value) // 'number'
console.log(typeof entry.value) // 'string'
})
})
```

If you desire normalization for values (e.g. to stringify numbers), wrap `memdown` with [`encoding-down`]. Alternatively install [`level-mem`] which conveniently bundles [`levelup`], `memdown` and [`encoding-down`]. Such an approach is also recommended if you want to achieve universal (isomorphic) behavior. For example, you could have [`leveldown`] in a backend and `memdown` in the frontend.
If you desire non-destructive encoding (e.g. to store and retrieve numbers as-is), wrap `memdown` with [`encoding-down`]. Alternatively install [`level-mem`] which conveniently bundles [`levelup`], `memdown` and [`encoding-down`]. Such an approach is also recommended if you want to achieve universal (isomorphic) behavior. For example, you could have [`leveldown`] in a backend and `memdown` in the frontend.

```js
const encode = require('encoding-down')
const db = levelup(encode(memdown()))
const db = levelup(encode(memdown(), { valueEncoding: 'json' }))

// The default value encoding is utf8, which stringifies input.
db.put('example', 123, (err) => {
if (err) throw err

Expand All @@ -73,46 +72,14 @@ db.put('example', 123, (err) => {
valueAsBuffer: false
}).on('data', (entry) => {
console.log(typeof entry.key) // 'string'
console.log(typeof entry.value) // 'string'
console.log(typeof entry.value) // 'number'
})
})
```

## Snapshot guarantees

A `memdown` store is backed by [a fully persistent data structure](https://www.npmjs.com/package/functional-red-black-tree) and thus has snapshot guarantees. Meaning that reads operate on a snapshot in time, unaffected by simultaneous writes. Do note `memdown` cannot uphold this guarantee for (copies of) object references. If you store object values, be mindful of mutating referenced objects:

```js
const db = levelup(memdown())
const obj = { thing: 'original' }

db.put('key', obj, (err) => {
obj.thing = 'modified'

db.get('key', { asBuffer: false }, (err, value) => {
console.log(value === obj) // true
console.log(value.thing) // 'modified'
})
})
```

Conversely, when `memdown` is wrapped with [`encoding-down`] it stores representations rather than references.

```js
const encode = require('encoding-down')

const db = levelup(encode(memdown(), { valueEncoding: 'json' }))
const obj = { thing: 'original' }

db.put('key', obj, (err) => {
obj.thing = 'modified'

db.get('key', { asBuffer: false }, (err, value) => {
console.log(value === obj) // false
console.log(value.thing) // 'original'
})
})
```
A `memdown` store is backed by [a fully persistent data structure](https://www.npmjs.com/package/functional-red-black-tree) and thus has snapshot guarantees. Meaning that reads operate on a snapshot in time, unaffected by simultaneous writes.

## Test

Expand Down
2 changes: 1 addition & 1 deletion UPGRADING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This document describes breaking changes and how to upgrade. For a complete list

## v5 (unreleased)

Support of keys other than strings and Buffers has been dropped. Internally `memdown` now stores keys as Buffers which solves a number of compatibility issues ([#186](https://github.com/Level/memdown/issues/186)). If you pass in a key that isn't a string or Buffer, it will be irreversibly stringified.
Support of keys & values other than strings and Buffers has been dropped. Internally `memdown` now stores keys & values as Buffers which solves a number of compatibility issues ([#186](https://github.com/Level/memdown/issues/186)). If you pass in a key or value that isn't a string or Buffer, it will be irreversibly stringified.

## v4

Expand Down
14 changes: 8 additions & 6 deletions memdown.js
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ MemIterator.prototype._next = function (callback) {
key = key.toString()
}

if (this.valueAsBuffer && !Buffer.isBuffer(value)) {
value = Buffer.from(String(value))
if (!this.valueAsBuffer) {
value = value.toString()
}

this._tree[this._incr]()
Expand Down Expand Up @@ -138,7 +138,9 @@ MemIterator.prototype._outOfRange = function (target) {
}

MemIterator.prototype._seek = function (target) {
// TODO: conversions - i.e. string keys, with buffer target.
if (target.length === 0) {
throw new Error('cannot seek() to an empty target')
}

if (this._outOfRange(target)) {
this._tree = this.db._store.end
Expand Down Expand Up @@ -172,7 +174,7 @@ MemDOWN.prototype._serializeKey = function (key) {
}

MemDOWN.prototype._serializeValue = function (value) {
return value
return Buffer.isBuffer(value) ? value : Buffer.from(String(value))
}

MemDOWN.prototype._put = function (key, value, options, callback) {
Expand All @@ -197,8 +199,8 @@ MemDOWN.prototype._get = function (key, options, callback) {
})
}

if (options.asBuffer !== false && !Buffer.isBuffer(value)) {
value = Buffer.from(String(value))
if (!options.asBuffer) {
value = value.toString()
}

setImmediate(function callNext () {
Expand Down
41 changes: 9 additions & 32 deletions test.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,10 @@ var test = require('tape')
var suite = require('abstract-leveldown/test')
var concat = require('level-concat-iterator')
var memdown = require('.').default
var MemIterator = require('.').MemIterator
var ltgt = require('ltgt')
var Buffer = require('safe-buffer').Buffer
var noop = function () { }

// Temporary fix for abstract seek tests, which currently assume
// that buffers are stored the same as strings (i.e. as byte arrays).
var baseSeek = MemIterator.prototype._seek
MemIterator.prototype._seek = function (target) {
return baseSeek.call(this, String(target))
}

var testCommon = suite.common({
test: test,
factory: function () {
Expand Down Expand Up @@ -364,7 +356,7 @@ test('put multiple times', function (t) {
})
})

test('put key as string, get as buffer and vice versa', function (t) {
test('put as string, get as buffer and vice versa', function (t) {
t.plan(7)

var db = testCommon.factory()
Expand All @@ -375,13 +367,13 @@ test('put key as string, get as buffer and vice versa', function (t) {
db.put('a', 'a', function (err) {
t.ifError(err, 'no put error')

db.get(Buffer.from('a'), { asBuffer: false }, function (err, value) {
db.get(Buffer.from('a'), { asBuffer: true }, function (err, value) {
t.ifError(err, 'no get error')
t.is(value, 'a', 'got value')
t.same(value, Buffer.from('a'), 'got value')
})
})

db.put(Buffer.from('b'), 'b', function (err) {
db.put(Buffer.from('b'), Buffer.from('b'), function (err) {
t.ifError(err, 'no put error')

db.get('b', { asBuffer: false }, function (err, value) {
Expand All @@ -392,7 +384,7 @@ test('put key as string, get as buffer and vice versa', function (t) {
})
})

test('put key as string, iterate as buffer', function (t) {
test('put as string, iterate as buffer', function (t) {
t.plan(4)

var db = testCommon.factory()
Expand All @@ -403,23 +395,23 @@ test('put key as string, iterate as buffer', function (t) {
db.put('a', 'a', function (err) {
t.ifError(err, 'no put error')

concat(db.iterator({ keyAsBuffer: true, valueAsBuffer: false }), function (err, entries) {
concat(db.iterator({ keyAsBuffer: true, valueAsBuffer: true }), function (err, entries) {
t.ifError(err, 'no concat error')
t.same(entries, [{ key: Buffer.from('a'), value: 'a' }])
t.same(entries, [{ key: Buffer.from('a'), value: Buffer.from('a') }])
})
})
})
})

test('put key as buffer, iterate as string', function (t) {
test('put as buffer, iterate as string', function (t) {
t.plan(4)

var db = testCommon.factory()

db.open(function (err) {
t.ifError(err, 'no error from open')

db.put(Buffer.from('a'), 'a', function (err) {
db.put(Buffer.from('a'), Buffer.from('a'), function (err) {
t.ifError(err, 'no put error')

concat(db.iterator({ keyAsBuffer: false, valueAsBuffer: false }), function (err, entries) {
Expand Down Expand Up @@ -455,21 +447,6 @@ test('number keys', function (t) {
})
})

test('object value', function (t) {
t.plan(2)

var db = testCommon.factory()
var obj = {}

db.open(noop)
db.put('key', obj, noop)

db.get('key', { asBuffer: false }, function (err, value) {
t.ifError(err, 'no get error')
t.ok(value === obj, 'same object')
})
})

function stringBuffer (value) {
return Buffer.from(String(value))
}
Expand Down

0 comments on commit f008114

Please sign in to comment.