Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Drop support of value types other than string and Buffer #192

Merged
merged 3 commits into from
Aug 18, 2019
Merged

Drop support of value types other than string and Buffer #192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d2c7344
Drop support of value types other than string and Buffer
vweevers Aug 17, 2019
b6c298c
Test mixed use of string/buffer on values too
vweevers Aug 17, 2019
ca4ef70
Rename test (where my bikeshed at)
vweevers Aug 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 6 additions & 39 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Your data is discarded when the process ends or you release a reference to the s

## Data types

Keys can be strings or Buffers. Any other key type will be irreversibly stringified. Unlike [`leveldown`] though, `memdown` does not stringify values. This means that in addition to Buffers, you can store any JS value without the need for [`encoding-down`]. The only exceptions are `null` and `undefined`. Keys and values of that type are rejected.
Keys and values can be strings or Buffers. Any other key type will be irreversibly stringified. The only exceptions are `null` and `undefined`. Keys and values of that type are rejected.

```js
const db = levelup(memdown())
Expand All @@ -53,18 +53,17 @@ db.put('example', 123, (err) => {
valueAsBuffer: false
}).on('data', (entry) => {
console.log(typeof entry.key) // 'string'
console.log(typeof entry.value) // 'number'
console.log(typeof entry.value) // 'string'
})
})
```

If you desire normalization for values (e.g. to stringify numbers), wrap `memdown` with [`encoding-down`]. Alternatively install [`level-mem`] which conveniently bundles [`levelup`], `memdown` and [`encoding-down`]. Such an approach is also recommended if you want to achieve universal (isomorphic) behavior. For example, you could have [`leveldown`] in a backend and `memdown` in the frontend.
If you desire non-destructive encoding (e.g. to store and retrieve numbers as-is), wrap `memdown` with [`encoding-down`]. Alternatively install [`level-mem`] which conveniently bundles [`levelup`], `memdown` and [`encoding-down`]. Such an approach is also recommended if you want to achieve universal (isomorphic) behavior. For example, you could have [`leveldown`] in a backend and `memdown` in the frontend.

```js
const encode = require('encoding-down')
const db = levelup(encode(memdown()))
const db = levelup(encode(memdown(), { valueEncoding: 'json' }))

// The default value encoding is utf8, which stringifies input.
db.put('example', 123, (err) => {
if (err) throw err

Expand All @@ -73,46 +72,14 @@ db.put('example', 123, (err) => {
valueAsBuffer: false
}).on('data', (entry) => {
console.log(typeof entry.key) // 'string'
console.log(typeof entry.value) // 'string'
console.log(typeof entry.value) // 'number'
})
})
```

## Snapshot guarantees

A `memdown` store is backed by [a fully persistent data structure](https://www.npmjs.com/package/functional-red-black-tree) and thus has snapshot guarantees. Meaning that reads operate on a snapshot in time, unaffected by simultaneous writes. Do note `memdown` cannot uphold this guarantee for (copies of) object references. If you store object values, be mindful of mutating referenced objects:

```js
const db = levelup(memdown())
const obj = { thing: 'original' }

db.put('key', obj, (err) => {
obj.thing = 'modified'

db.get('key', { asBuffer: false }, (err, value) => {
console.log(value === obj) // true
console.log(value.thing) // 'modified'
})
})
```

Conversely, when `memdown` is wrapped with [`encoding-down`] it stores representations rather than references.

```js
const encode = require('encoding-down')

const db = levelup(encode(memdown(), { valueEncoding: 'json' }))
const obj = { thing: 'original' }

db.put('key', obj, (err) => {
obj.thing = 'modified'

db.get('key', { asBuffer: false }, (err, value) => {
console.log(value === obj) // false
console.log(value.thing) // 'original'
})
})
```
A `memdown` store is backed by [a fully persistent data structure](https://www.npmjs.com/package/functional-red-black-tree) and thus has snapshot guarantees. Meaning that reads operate on a snapshot in time, unaffected by simultaneous writes.

## Test

Expand Down
2 changes: 1 addition & 1 deletion UPGRADING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This document describes breaking changes and how to upgrade. For a complete list

## v5 (unreleased)

Support of keys other than strings and Buffers has been dropped. Internally `memdown` now stores keys as Buffers which solves a number of compatibility issues ([#186](https://github.com/Level/memdown/issues/186)). If you pass in a key that isn't a string or Buffer, it will be irreversibly stringified.
Support of keys & values other than strings and Buffers has been dropped. Internally `memdown` now stores keys & values as Buffers which solves a number of compatibility issues ([#186](https://github.com/Level/memdown/issues/186)). If you pass in a key or value that isn't a string or Buffer, it will be irreversibly stringified.

## v4

Expand Down
14 changes: 8 additions & 6 deletions memdown.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ MemIterator.prototype._next = function (callback) {
key = key.toString()
}

if (this.valueAsBuffer && !Buffer.isBuffer(value)) {
value = Buffer.from(String(value))
if (!this.valueAsBuffer) {
value = value.toString()
}

this._tree[this._incr]()
Expand Down Expand Up @@ -138,7 +138,9 @@ MemIterator.prototype._outOfRange = function (target) {
}

MemIterator.prototype._seek = function (target) {
// TODO: conversions - i.e. string keys, with buffer target.
if (target.length === 0) {
throw new Error('cannot seek() to an empty target')
}

if (this._outOfRange(target)) {
this._tree = this.db._store.end
Expand Down Expand Up @@ -172,7 +174,7 @@ MemDOWN.prototype._serializeKey = function (key) {
}

MemDOWN.prototype._serializeValue = function (value) {
return value
return Buffer.isBuffer(value) ? value : Buffer.from(String(value))
}

MemDOWN.prototype._put = function (key, value, options, callback) {
Expand All @@ -197,8 +199,8 @@ MemDOWN.prototype._get = function (key, options, callback) {
})
}

if (options.asBuffer !== false && !Buffer.isBuffer(value)) {
value = Buffer.from(String(value))
if (!options.asBuffer) {
value = value.toString()
}

setImmediate(function callNext () {
Expand Down
41 changes: 9 additions & 32 deletions test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,10 @@ var test = require('tape')
var suite = require('abstract-leveldown/test')
var concat = require('level-concat-iterator')
var memdown = require('.').default
var MemIterator = require('.').MemIterator
var ltgt = require('ltgt')
var Buffer = require('safe-buffer').Buffer
var noop = function () { }

// Temporary fix for abstract seek tests, which currently assume
// that buffers are stored the same as strings (i.e. as byte arrays).
var baseSeek = MemIterator.prototype._seek
MemIterator.prototype._seek = function (target) {
return baseSeek.call(this, String(target))
}

var testCommon = suite.common({
test: test,
factory: function () {
Expand Down Expand Up @@ -364,7 +356,7 @@ test('put multiple times', function (t) {
})
})

test('put key as string, get as buffer and vice versa', function (t) {
test('put as string, get as buffer and vice versa', function (t) {
t.plan(7)

var db = testCommon.factory()
Expand All @@ -375,13 +367,13 @@ test('put key as string, get as buffer and vice versa', function (t) {
db.put('a', 'a', function (err) {
t.ifError(err, 'no put error')

db.get(Buffer.from('a'), { asBuffer: false }, function (err, value) {
db.get(Buffer.from('a'), { asBuffer: true }, function (err, value) {
t.ifError(err, 'no get error')
t.is(value, 'a', 'got value')
t.same(value, Buffer.from('a'), 'got value')
})
})

db.put(Buffer.from('b'), 'b', function (err) {
db.put(Buffer.from('b'), Buffer.from('b'), function (err) {
t.ifError(err, 'no put error')

db.get('b', { asBuffer: false }, function (err, value) {
Expand All @@ -392,7 +384,7 @@ test('put key as string, get as buffer and vice versa', function (t) {
})
})

test('put key as string, iterate as buffer', function (t) {
test('put as string, iterate as buffer', function (t) {
t.plan(4)

var db = testCommon.factory()
Expand All @@ -403,23 +395,23 @@ test('put key as string, iterate as buffer', function (t) {
db.put('a', 'a', function (err) {
t.ifError(err, 'no put error')

concat(db.iterator({ keyAsBuffer: true, valueAsBuffer: false }), function (err, entries) {
concat(db.iterator({ keyAsBuffer: true, valueAsBuffer: true }), function (err, entries) {
t.ifError(err, 'no concat error')
t.same(entries, [{ key: Buffer.from('a'), value: 'a' }])
t.same(entries, [{ key: Buffer.from('a'), value: Buffer.from('a') }])
})
})
})
})

test('put key as buffer, iterate as string', function (t) {
test('put as buffer, iterate as string', function (t) {
t.plan(4)

var db = testCommon.factory()

db.open(function (err) {
t.ifError(err, 'no error from open')

db.put(Buffer.from('a'), 'a', function (err) {
db.put(Buffer.from('a'), Buffer.from('a'), function (err) {
t.ifError(err, 'no put error')

concat(db.iterator({ keyAsBuffer: false, valueAsBuffer: false }), function (err, entries) {
Expand Down Expand Up @@ -455,21 +447,6 @@ test('number keys', function (t) {
})
})

test('object value', function (t) {
t.plan(2)

var db = testCommon.factory()
var obj = {}

db.open(noop)
db.put('key', obj, noop)

db.get('key', { asBuffer: false }, function (err, value) {
t.ifError(err, 'no get error')
t.ok(value === obj, 'same object')
})
})

function stringBuffer (value) {
return Buffer.from(String(value))
}
Expand Down