Skip to content

Commit

Permalink
Update routing rules doc (#1344)
Browse files Browse the repository at this point in the history
  • Loading branch information
@mattlord
mattlord authored Jan 15, 2023
1 parent 4c86ccb commit 39dba63
Show file tree
Hide file tree
Showing 2 changed files with 159 additions and 100 deletions.
195 changes: 159 additions & 36 deletions content/en/docs/16.0/reference/features/schema-routing-rules.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,52 +4,175 @@ weight: 15
aliases: ['/docs/schema-management/routing-rules/','/docs/reference/schema-routing-rules/']
---

The Vitess routing rules feature is a powerful mechanism for directing traffic to the right keyspaces, shards or tablet types.
It fulfils the following use cases:
The Vitess routing rules feature is a powerful mechanism for directing query traffic to the right keyspaces, shards, and tablet types in
[Vitess Gateways](../../../concepts/vtgate/) (`vtgate`). Their primary usage today is for the following use case:

* **Routing traffic during resharding**: During resharding, you can specify rules that decide where to send reads and writes. For example,
you can move traffic from the source shard to the destination shards, but only for the `rdonly` or `replica` types. This gives you
the option to try out the new shards and make sure they will work as intended before committing to move the rest of the traffic.
* **Routing traffic during data migrations**: during e.g. [`MoveTables`](../../vreplication/movetables/) and
[`Reshard`](../../vreplication/reshard/) operations, routing rules dictate where to send reads and writes. These routing rules are managed
automatically by [VReplication](../../vreplication/vreplication/). You can see an example of their usage in the
[MoveTables](../../../user-guides/migration/move-tables/) user guide.

## ApplyRoutingRules
Understanding the routing rules can help you debug migration related issues as well as provide you with another powerful tool as
you operate Vitess.

You can use the vtctlclient command to apply routing rules:
## Viewing Routing Rules

```
ApplyRoutingRules -- {--rules=<rules> || --rules_file=<rules_json_file>} [--cells=c1,c2,...] [--skip_rebuild] [--dry-run]
```
The routing rules are global and can be viewed using the [`GetRoutingRules` client command](../../programs/vtctldclient/vtctldclient_getroutingrules/).

## Updating Routing Rules

You can update the routing rules using the [`ApplyRoutingRules` client command](../../programs/vtctldclient/vtctldclient_applyroutingrules/).

## Syntax

### Resharding

Routing rules can be specified using JSON format. Here's an example:

``` json
{"rules": [
{
"from_table": "t@rdonly",
"to_tables": ["target.t"]
}, {
"from_table": "target.t",
"to_tables": ["source.t"]
}, {
"from_table": "t",
"to_tables": ["source.t"]
}
]}
Routing rules are managed using the JSON format. Here's an example, using the routing rules that are put in place by `MoveTables`
in the [local examples](../../../get-started/local/) where the `customer` and `corder` tables are being moved from the `commerce`
keyspace to the `customer` keyspace and we have not yet switched traffic from the `commerce` keyspace to the `customer` keyspace — so all
traffic, regardless of which keyspace a client uses, are sent to the `commerce` keyspace:
```json
$ vtctldclient --server=localhost:15999 GetRoutingRules
{
"rules": [
{
"from_table": "customer.customer",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "commerce.corder@replica",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "customer.customer@rdonly",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "commerce.corder@rdonly",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "corder@replica",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "commerce.customer@replica",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "commerce.customer@rdonly",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "customer.corder@replica",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "customer.corder@rdonly",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "customer.customer@replica",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "customer.corder",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "corder@rdonly",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "customer@replica",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "customer@rdonly",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "corder",
"to_tables": [
"commerce.corder"
]
},
{
"from_table": "customer",
"to_tables": [
"commerce.customer"
]
}
]
}
```

The above JSON specifies the following rules:
## When Routing Rules Are Applied

In the above example, we send all query traffic for the `customer` and `corder` tables to the `commerce` keyspace regardless of how
the client specifies the database/schema and table qualifiers. There is, however, one important exception and that is when the client
explicitly requests the usage of a specific shard, also known as "shard targeting". For example, if the client specifies the database
as `customer:0` or `customer:0@replica` then the query will get run against that shard in the customer keyspace.

{{< warning >}}
You should exercise _extreme_ caution when executing ad-hoc *write* queries during this time as you may think that you're deleting data
from the target keyspace, that is as of yet unused, when in reality you're deleting it from the source keyspace that is currently
serving production traffic.
{{</ warning >}}

{{< info >}}
You can leverage shard targeting to perform ad-hoc *read-only* queries against the target and source keyspace/shards to perform any
additional data validation or checks that you want (beyond [`VDiff`](../../vreplication/vdiff/)). You can also use this shard targeting
to see how your data is distributed across the keyspace's shards.
{{</ info >}}

## Additional Details

There are some key details to keep in mind if you will be creating and managing your own custom routing rules.
- The `to_tables` field must contain only one entry and the table name must be fully qualified.

* If you sent a query accessing `t` for an `rdonly` instance, then it would be sent to table `t` in the `target` keyspace.
* If you sent a query accessing `target.t` for anything other than `rdonly`, it would be sent `t` in the `source` keyspace.
* If you sent a query accessing `t` without any qualification, it would be sent to `t` in the `source` keyspace.
- If the `from_table` is qualified by a keyspace, then a query that references that table will get redirected to the corresponding target table. The reference need not be explicit. For example, if you are connected to the `customer` keyspace, then an unqualified reference to the `customer` table is interpreted as a qualified reference to `customer.customer`.

These rules are an example of how they can be used to shift traffic for a table during a vertical resharding process.
In this case, the assumption is that we are moving `t` from `source` to `target`, and so far, we've shifted traffic
for just the `rdonly` tablet types.
- You may further add a tablet type to the `from_table` field using the `@<type>` syntax seen in the example above. If so, only queries that target that tablet type will get redirected. Although you can qualify a table by its keyspace in a query, there is no equivalent syntax for specifying the tablet type. The only way to choose a tablet type is through the `use` statement, like `use customer@replica`, or by specifying it in the connection string.

By updating these rules, you can eventually move all traffic to `target.t`
- The more specific rules supercede the less specific one. For example, `customer.customer@replica` is chosen over `customer.customer` if the current tablet type is a `replica`.

The rules are applied only once. The resulting targets need to specify fully qualified table names.
- If the `to_tables` have special characters that need escaping, you can use the mysql backtick syntax to do so. As for the `from_tables`, the table name should *not* be escaped. Instead, you should just concatenate the table with the keyspace without the backticks. In the following example, we are redirecting the `b.c` table to the `c.b` table in keyspace `a`:
``` json
{
"rules": [
{
"from_table": "a.b.c",
"to_tables": [
"a.`c.b`"
]
}
]
}
```
64 changes: 0 additions & 64 deletions content/en/docs/16.0/reference/features/vschema.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -309,67 +309,3 @@ The examples/demo also shows more tricks you can perform:
* `music_extra` shares `music_user_idx` with `music`, and uses it as Primary Vindex.
* `music_extra` defines an additional Functional Vindex called `keyspace_id` which the demo auto-populates using the reverse mapping capability.
* There is also a `name_info` table that showcases a case-insensitive Vindex `unicode_loose_md5`.

## Routing Rules

The `RoutingRules` section of the VSchema can be used to dynamically route traffic of a tablet to a different table than originally referenced in the query. This feature is used by the `MoveTables` workflow allowing you to change the application independently of when the actual traffic is moved from the old source table to the new target table.

Here is an example of `RoutingRules`

``` json
{
"rules": [
{
"from_table": "customer",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "customer.customer",
"to_tables": [
"commerce.customer"
]
},
{
"from_table": "customer.customer@replica",
"to_tables": [
"commerce.customer"
]
}
]
}
```

In the above JSON data structure, each rule maps an input table to a target.
If the input table name is unqualified, then any unqualified reference to that table gets redirected to the fully qualified `to_tables`.

The `to_tables` field must contain only one entry and the table name must be fully qualified.

If the `from_table` is qualified by a keyspace, then a query that references that table will get redirected to the corresponding target table. The reference need not be explicit. For example, if you are connected to the `customer` keyspace, then an unqualified reference to the `customer` table is interpreted as a qualified reference to `customer.customer`.

You may further add a tablet type to the `from_table` field. If so, only queries that target that tablet type will get redirected. Although you can qualify a table by its keyspace in a query, there is no equivalent syntax for specifying the tablet type. The only way to choose a tablet type is through the `use` statement, like `use @replica`, or by specifying it in the connection string.

The more specific rules supercede the less specific one. For example, `customer.customer@replica` is chosen over `customer.customer` if the current tablet type is a `replica`.

If the `to_tables` have special characters that need escaping, you can use the mysql backtick syntax to do so. As for the `from_tables`, the table name should not be escaped. Instead, you should just concatenate the table with the keyspace without the backticks. In the following example, we are redirecting the `b.c` table to the `c.b` table in keyspace `a`:

``` json
{
"rules": [
{
"from_table": "a.b.c",
"to_tables": [
"a.`c.b`"
]
}
]
}
```

### Commands

You can use the following commands to maintain routing rules:

* `GetRoutingRules`
* `ApplyRoutingRules -- {--rules=<rules> || --rules_file=<rules_file>} [--cells=c1,c2,...] [--skip_rebuild] [--dry-run]`

0 comments on commit 39dba63

Please sign in to comment.