Skip to content
This repository has been archived by the owner on Aug 3, 2023. It is now read-only.

Feat kv commands #405

Merged
merged 62 commits into from
Sep 13, 2019
Merged
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
731f9b6
feat: use cloudflare-rs
ashleymichal Aug 2, 2019
5169f34
feat: #340 first draft of wrangler kv add <namespace>
ashleymichal Aug 2, 2019
59fdb64
Add documentation for kv create
ashleymichal Aug 5, 2019
22e9052
refactor: handle success in endpoint caller
ashleymichal Aug 7, 2019
b8bc14b
feat: kv delete <namespace>
ashleymichal Aug 5, 2019
1ff3c02
feat: kv rename
ashleymichal Aug 5, 2019
b5fbdd2
Merge pull request #403 from cloudflare/alewis/feat-rename-namespace
ashleymichal Aug 7, 2019
2e4a9ee
feat: kv list (raw output)
ashleymichal Aug 5, 2019
3900fb6
Pretty up list output
ashleymichal Aug 7, 2019
1c05b10
correct rename output example in docs
ashleymichal Aug 7, 2019
c0dd867
Merge pull request #404 from cloudflare/alewis/feat-list-namespaces
ashleymichal Aug 7, 2019
22fc8c6
Update cloudflare-rs invocations to match updates
ashleymichal Aug 14, 2019
1022df1
Merge pull request #437 from cloudflare/alewis/update-dependencies
ashleymichal Aug 15, 2019
13ec991
Merge remote-tracking branch 'origin/master' into feat-kv-commands
ashleymichal Aug 15, 2019
bcea204
update cloudflare-rs invocations
ashleymichal Aug 19, 2019
3e1b854
Merge remote-tracking branch 'origin/master' into feat-kv-commands
ashleymichal Aug 19, 2019
8f3a14e
Add gets for single keys from Workers KV (#454)
gabbifish Aug 21, 2019
a1ff27a
Add individual write subcommand for Workers KV (#458)
gabbifish Aug 21, 2019
df8219a
Add bulk upload to Workers KV (#445)
gabbifish Aug 21, 2019
408c34e
Implementation of single and bulk deletion from KV (#452)
gabbifish Aug 21, 2019
ed43c93
Merge branch 'master' into feat-kv-commands
ashleymichal Aug 21, 2019
725ea2f
Add key listing functionality for workers kv subcommands (#462)
gabbifish Aug 22, 2019
464667b
make error message about lack of workers kv entitlements more clear
gabbifish Aug 22, 2019
eb1bca8
fix #406: remove panics and ensure required args for kv subcommands a…
gabbifish Aug 27, 2019
acb6892
clarify Workers KV upgrade error
gabbifish Aug 27, 2019
7c27abb
Gabbi/improve bulk error handling (#466)
gabbifish Aug 27, 2019
e87e5eb
Merge branch 'master' into feat-kv-commands
ashleymichal Aug 27, 2019
4752e23
Remove logic for uploading directories from bulk (#485)
ashleymichal Aug 30, 2019
d8a04c6
Update kv docs
ashleymichal Aug 30, 2019
4f1c239
Gabbi/update subcommand invocations (#501)
gabbifish Aug 30, 2019
a1d72bf
Merge pull request #503 from cloudflare/alewis/kv-docs
ashleymichal Sep 3, 2019
570f5ef
Gabbi/url encode (#507)
gabbifish Sep 3, 2019
70b7ea9
Update usage of cloudflare::Credentials::User -> UserAuthKey
ashleymichal Sep 4, 2019
f1469a5
Gabbi/add interactive delete (#511)
gabbifish Sep 4, 2019
c5a4995
Merge pull request #520 from cloudflare/alewis/update-dependencies
ashleymichal Sep 4, 2019
5f20b4e
refactor: pass project to kv commands in prep for environments
ashleymichal Sep 5, 2019
e3800e2
Merge pull request #526 from cloudflare/alewis/refactor-pass-project
ashleymichal Sep 5, 2019
ee872bc
refactor: organize kv by command type
ashleymichal Sep 5, 2019
264de89
Merge pull request #527 from cloudflare/alewis/refactor-directories
ashleymichal Sep 6, 2019
8fb1f56
Use url encoded strings for list keys prefixes
gabbifish Sep 9, 2019
ad5b33b
Gabbi/more detailed subcommand docs (#537)
gabbifish Sep 9, 2019
bfe8d92
Alewis/refactor key list (#531)
ashleymichal Sep 9, 2019
e71a91d
Fix merge conflict in src/main.rs
gabbifish Sep 9, 2019
4237934
s/project/target
gabbifish Sep 9, 2019
aac2161
fix with steve's feedback
gabbifish Sep 10, 2019
f55263d
use cloudflare-rs package from crates.io (#549)
gabbifish Sep 11, 2019
28de238
Gabbi/add kv env support (#542)
gabbifish Sep 11, 2019
bf19955
remove `kv:namespace rename` subcommand (#555)
gabbifish Sep 11, 2019
f667c7b
Gabbi/perscriptive namespace titles (#557)
gabbifish Sep 11, 2019
69ac09a
Print out namespaces list as json by default (#541)
gabbifish Sep 11, 2019
267caf9
Gabbi/merge master in (#562)
gabbifish Sep 12, 2019
bd9f4f5
warn user to remove binding if deleted + check input for bindings (#560)
gabbifish Sep 12, 2019
a8926e7
Re-order kv commands docs to be more user friendly (#558)
EverlastingBugstopper Sep 12, 2019
109b3a5
Update readme to point people to KV commands docs (#568)
gabbifish Sep 12, 2019
e71e699
fix links to environmenets
gabbifish Sep 12, 2019
399cf3c
Allow use of same json file to bulk delete as bulk put
ashleymichal Sep 12, 2019
ec5ff3f
Merge pull request #577 from cloudflare/alewis/bulk-delete
ashleymichal Sep 12, 2019
d33ce50
Merge 09/12 merges to master into feat-kv-commands (#580)
gabbifish Sep 12, 2019
81ac1e6
Fix panic when `kv: ` command does not have subcommand (#579)
gabbifish Sep 12, 2019
6c3d133
Merge branch 'master' into feat-kv-commands
ashleymichal Sep 13, 2019
17e98f4
Resolve merge conflicts in README
ashleymichal Sep 13, 2019
293ff89
Fix links to environments docs
ashleymichal Sep 13, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
977 changes: 609 additions & 368 deletions Cargo.lock

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ build = "build.rs"
atty = "0.2.11"
binary-install = "0.0.3-alpha"
clap = "2.32.0"
cloudflare = "0.3.1"
config = "0.9.2"
console = "0.7.5"
dirs = "1.0.5"
Expand All @@ -35,8 +36,14 @@ base64 = "0.10.1"
lazy_static = "1.3.0"
text_io = "0.1.7"
exitfailure = "0.5.1"
prettytable-rs = "0.8.0"
notify = "4.0.12"
ws = "0.9.0"
url = "2.1.0"
walkdir = "2.2.9"
percent-encoding = "1.0.1"
http = "0.1.1"
regex = "1"

[dev-dependencies]
assert_cmd = "0.11.1"
Expand Down
258 changes: 140 additions & 118 deletions README.md

Large diffs are not rendered by default.

19 changes: 19 additions & 0 deletions docs/environments.md → docs/content/environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,25 @@ name = "my-worker-staging"

Your default `wrangler build`, `wrangler preview`, and `wrangler publish` commands will all build with `webpack.dev.js`, as will `wrangler build -e staging`, `wrangler preview -e staging`, and `wrangler publish -e staging`. `wrangler build -e production`, `wrangler preview -e production`, and `wrangler publish -e production` would all use your `webpack.config.js` file.

### KV Namespaces with environments

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was renamed. I might be wrong but I believe I saw a link to this doc printed out in my terminal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct

@gabbifish do you mind updating the instances of /docs/environments.md to /docs/content/environments.md?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do!

You can specify different kv namespaces for different environments.

```toml
name = "my-worker"
type = "webpack"
account_id = "12345678901234567890"
workers_dot_dev = true
kv-namespaces = [
{ binding = "KV", id = "06779da6940b431db6e566b4846d64db" }
]

[env.production]
kv-namespaces = [
{ binding = "KV", id = "bd46d6484b665e6bd134b0496ad97760" }
]
```

## Invalid configurations

### Multiple types
Expand Down
330 changes: 330 additions & 0 deletions docs/content/kv_commands.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,330 @@
# 🗂️ `kv`

## Overview

The `kv` subcommand allows you to store application data in the Cloudflare network to be accessed from Workers, using
[Workers KV](https://www.cloudflare.com/products/workers-kv/).
KV operations are scoped to your account, so in order to use any of these commands, you need to:

* have a Wrangler project set up with your `account_id` configured in the `wrangler.toml`
ashleymichal marked this conversation as resolved.
Show resolved Hide resolved
* call commands from within a Wrangler project directory.

## Getting Started

To use Workers KV with your Worker, the first thing you must do is create a KV namespace. This is done with
the `kv:namespace` subcommand.

The `kv:namespace` subcommand takes as a new binding name as an argument. It will create a Worker KV namespace
whose title is a concatenation of your Worker's name (from `wrangler.toml`) and the binding name you provide:
```console
$ wrangler kv:namespace create "MY_KV"
🌀 Creating namespace with title "worker-MY_KV"
✨ Success: WorkersKvNamespace {
id: "e29b263ab50e42ce9b637fa8370175e8",
title: "worker-MY_KV",
}
✨ Add the following to your wrangler.toml:
kv-namespaces = [
{ binding: "MY_KV", id: "e29b263ab50e42ce9b637fa8370175e8" }
]
```
Make sure to add the `kv-namespaces` output above to your `wrangler.toml`. You can now
access it from a Worker with code like:
```js
let value = await MY_KV.get("my-key");
```
The full KV API for Workers can be found [here](https://developers.cloudflare.com/workers/reference/storage/).

To put a value to your KV namespace via Wrangler, use the `kv:key put` subcommand.
```console
$ wrangler kv:key put --binding=MY_KV "key" "value"
✨ Success
```
You can also specify which namespace to put your key-value pair into using `--namespace-id` instead of `--binding`:
```console
$ wrangler kv:key put --namespace-id=e29b263ab50e42ce9b637fa8370175e8 "key" "value"
✨ Success
```

Additionally, KV namespaces can be used with [environments](./environments.md)! This is useful for when you have code that refers to
a KV binding like `MY_KV`, and you want to be able to have these bindings point to different namespaces (like
one for staging and one for production). So, if you have a `wrangler.toml` with two environments:

```toml
[env.staging]
kv-namespaces = [
{ binding: "MY_KV", id: "e29b263ab50e42ce9b637fa8370175e8" }
]

[env.production]
kv-namespaces = [
{ binding: "MY_KV", id: "a825455ce00f4f7282403da85269f8ea" }
]
```

To insert a value into a specific KV namespace, you can use
```console
$ wrangler kv:key put --env=staging --binding=MY_MV "key" "value"
✨ Success
```

Since `--namespace-id` is always unique (unlike binding names), you don't need to pass environment variables for them (they will be unused).

There are way more helpful Wrangler subcommands for interacting with Workers KV, like ones for bulk uploads and deletes--check them out below!

## Concepts

Most `kv` commands require you to specify a namespace. A namespace can be specified in two ways:

1. With a `--binding`:
```sh
wrangler kv:key get --binding=KV "my key"
```
1. With a `--namespace_id`:
```sh
wrangler kv:key get --namespace-id=06779da6940b431db6e566b4846d64db "my key"
```

Most `kv` subcommands also allow you to specify an environment with the optional `--env` flag. This allows you to publish workers running the same code but with different namespaces. For example, you could use separate staging and production namespaces for KV data in your `wrangler.toml`:

```toml
type = "webpack"
name = "my-worker"
account_id = "<account id here>"
route = "staging.example.com/*"
workers_dot_dev = false

kv-namespaces = [
{ binding = "KV", id = "06779da6940b431db6e566b4846d64db" }
]

[env.production]
route = "example.com/*"
kv-namespaces = [
{ binding = "KV", id = "07bc1f3d1f2a4fd8a45a7e026e2681c6" }
]
```

With the wrangler.toml above, you can specify `--env production` when you want to perform a KV action on
the namespace `KV` under `env.production`. For example, with the wrangler.toml above, you can get a value
out of a production KV instance with:

```console
wrangler kv:key get --binding "KV" --env=production "my key"
```

To learn more about environments, check out the [environments documentation](./environments.md).

## `kv:namespace`

### `create`

Creates a new namespace.

Takes an optional `--env` [environment](./environments.md) argument.

#### Usage

```console
$ wrangler kv:namespace create "MY_KV"
🌀 Creating namespace with title "worker-MY_KV"
✨ Success: WorkersKvNamespace {
id: "e29b263ab50e42ce9b637fa8370175e8",
title: "worker-MY_KV",
}
✨ Add the following to your wrangler.toml:
kv-namespaces = [
{ binding: "MY_KV", id: "e29b263ab50e42ce9b637fa8370175e8" }
]
```

### `list`

Outputs a list of all KV namespaces associated with your account id.

#### Usage
The example below uses the `jq` command line tool to pretty-print output.

```console
$ wrangler kv:namespace list | jq '.'
[
{
"id": "06779da6940b431db6e566b4846d64db",
"title": "TEST_NAMESPACE"
},
{
"id": "32ac1b3c2ed34ed3b397268817dea9ea",
"title": "STATIC_CONTENT"
}
]
```

### `delete`

Deletes a given namespace.

Requires `--binding` or `--namespace-id` argument.

Takes an optional `--env` [environment](./environments.md) argument.

#### Usage

```console
$ wrangler kv:namespace delete --binding=KV
Are you sure you want to delete namespace f7b02e7fc70443149ac906dd81ec1791? [y/n]
yes
🌀 Deleting namespace f7b02e7fc70443149ac906dd81ec1791
✨ Success
```

## `kv:key`

### `put`

Writes a single key/value pair to the given namespace.

Requires `--binding` or `--namespace-id` argument.

Optional params include:

1. `--env`: The [environment](./environments.md) argument.
1. `--ttl`: Number of seconds for which the entries should be visible before they expire. At least 60. Takes precedence over 'expiration' option.
1. `--expiration`: Number of seconds since the UNIX epoch, indicating when the key-value pair should expire.
1. `--path`: Read value from the file at a given path. *This is good for security-sensitive operations, like uploading keys to KV; uploading from a file prevents a key value from being saved in areas like your terminal history.*

#### Usage

```console
$ wrangler kv:key put --binding=KV "key" "value" --ttl=10000
✨ Success
```

```console
$ wrangler kv:key put --binding=KV "key" value.txt --path
✨ Success
```

### `list`

Outputs a list of all keys in a given namespace.

Requires `--binding` or `--namespace-id` argument.

Optional params include:

1. `--env`: The [environment](./environments.md) argument.
1. `--prefix`: A prefix to filter listed keys.

#### Usage

The example below uses the `jq` command line tool to pretty-print output.

```console
$ wrangler kv:key list --binding=KV --prefix="public" | jq '.'
[
{
"name": "public_key"
},
{
"name": "public_key_with_expiration",
"expiration": "2019-09-10T23:18:58Z"
}
]
```

### `get`

Reads a single value by key from the given namespace.

Requires `--binding` or `--namespace-id` argument.

Takes an optional `--env` [environment](./environments.md) argument.

#### Usage

```console
$ wrangler kv:key get --binding=KV "key"
value
```

### `delete`

Removes a single key value pair from the given namespace.

Requires `--binding` or `--namespace-id` argument.

Takes an optional `--env` [environment](./environments.md) argument.

#### Usage

```console
$ wrangler kv:key delete --binding=KV "key"
Are you sure you want to delete key "key"? [y/n]
yes
🌀 Deleting key "key"
✨ Success
```

## `kv:bulk`

### `put`

Requires `--binding` or `--namespace-id` argument.

Writes a file full of key/value pairs to the given namespace. Takes as an argument a JSON file with a list of key-value pairs to upload (see JSON spec above). An example of JSON input:

```json
[
{
"key": "test_key",
"value": "test_value",
"expiration_ttl": 3600
}
]
```

The schema below is the full schema for key-value entries uploaded via the bulk API:

| **Name** | **Description** | Optional |
| ------------------------------ | ------------------------------------------------------------ | -------- |
| `key`<br />(String) | A key's name. The name may be at most 512 bytes. All printable, non-whitespace characters are valid. | no |
| `value`<br />(String) | A UTF-8 encoded string to be stored, up to 2 MB in length. | no |
| `expiration`<br />(Number) | The time, measured in number of seconds since the UNIX epoch, at which the key should expire. | yes |
| `expiration_ttl`<br />(Number) | The number of seconds for which the key should be visible before it expires. At least 60. | yes |
| `base64`<br />(Boolean) | Whether or not the server should base64 decode the value before storing it. Useful for writing values that wouldn't otherwise be valid JSON strings, such as images. Defaults to `false` | yes |

If both `expiration` and `expiration_ttl` are specified for a given key, the API will prefer `expiration_ttl`.

The `put` command also takes an optional `--env` [environment](./environments.md) argument.

#### Usage

```console
$ wrangler kv:bulk put --binding=KV allthethingsupload.json
✨ Success
```

### `delete`

Requires `--binding` or `--namespace-id` argument.

Deletes all specified keys within a given namespace.
Takes as an argument a JSON file with a list of keys to delete; for example:

```json
[
"key1",
"key2"
]
```

The `delete` command also takes an optional `--env` [environment](./environments.md) argument.

#### Usage

```console
$ wrangler kv:bulk delete --binding=KV allthethingsdelete.json
Are you sure you want to delete all keys in allthethingsdelete.json? [y/n]
yes
✨ Success
```
Loading