Skip to content

Commit

Permalink
[Docs] Minor fixes to docs pages around transactions (#23926)
Browse files Browse the repository at this point in the history
1. Remove remaining usage of the words "optimistic" and "pessimistic"
for concurrency control modes in distributed-transactions-ysql.md.
2. Fix information around serialization errors and deadlock errors in
transactions-errorcodes-ysql.md and transactions-retries-ysql.md.
3. Replace usage of ysql_max_write_restart_attempts with
yb_max_query_layer_retries in concurrency-control.md.

---------

Co-authored-by: Dwight Hodge <[email protected]>
  • Loading branch information
pkj415 and ddhodge authored Sep 18, 2024
1 parent 2a40433 commit f957dda
Show file tree
Hide file tree
Showing 10 changed files with 72 additions and 42 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -329,12 +329,12 @@ After a transaction T1 (that was waiting for other transactions) unblocks, it co

### Examples

The following examples describe different use cases detailing the Wait-on-Conflict behavior.
The following examples describe different use cases detailing the Wait-on-Conflict behavior. To run the examples, you need to do the following:

1. Note that the examples require you to set the YB-TServer flag `enable_wait_queues=true`.
1. Also, set the YB-TServer flag `ysql_max_write_restart_attempts=0` to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production.
1. Set the YB-TServer flag `enable_wait_queues=true`.
1. Set the per-session `yb_max_query_layer_retries=0` YSQL configuration parameter to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production. To set it at the cluster level, use the `ysql_pg_conf_csv` YB-TServer flag.

A restart is necessary for these flags to take effect.
A restart is necessary for the flags to take effect.

Start by setting up the table you'll use in all of the examples in this section.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,25 @@ The client can reconnect to the server and retry the transaction.
This error occurs when a transaction cannot be applied or progress further because of other conflicting transactions. For example, when multiple transactions are modifying the same key.

```output
ERROR: 40001: Operation expired: Transaction XXXX expired or aborted by a conflict
ERROR: could not serialize access due to concurrent update (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Serialization failure errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 40P01: Deadlock detected

This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted and they fail with the following error.

```output
ERROR: deadlock detected (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Deadlock detected errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 2D000: Invalid transaction termination

This error occurs when a transaction is terminated either by a `COMMIT` or a `ROLLBACK` in an invalid location. For example, when a `COMMIT` is issued inside a stored procedure that is called from inside a transaction.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.
Expand Down Expand Up @@ -76,19 +76,11 @@ If the `COMMIT` is successful, the program exits the loop. `attempt < max_attemp
##### 40001 - SerializationFailure
SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict) or when transactions are waiting on each other (deadlock). The error messages could be one of the following types:
SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict). During a conflict, certain transactions are retried.
- During a conflict, certain transactions are retried. However, after the retry limit is reached, an error occurs as follows:
```output
ERROR: 40001: All transparent retries exhausted.
```
- All transactions are given a dynamic priority. When a deadlock is detected, the transaction with lower priority is automatically killed. For this scenario, the client might receive a message similar to the following:
```output
ERROR: 40001: Operation expired: Heartbeat: Transaction XXXX expired or aborted by a conflict
```
```output
ERROR: could not serialize access due to concurrent update (...)
```
The correct way to handle this error is with a retry loop with exponential backoff, as described in [Client-side retry](#client-side-retry). When the [UPDATE](../../../../api/ysql/the-sql-language/statements/dml_update/) or [COMMIT](../../../../api/ysql/the-sql-language/statements/txn_commit/) fails because of `SerializationFailure`, the code retries after waiting for `sleep_time` seconds, up to `max_attempts`.
Expand All @@ -98,6 +90,17 @@ In read committed isolation level, as the server retries internally, the client
Another way to handle these failures is would be to rollback to a checkpoint before the failed statement and proceed further as described in [Savepoints](#savepoints).
##### 40001 - Deadlock detected
This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted
and they fail with the following error:
```output
ERROR: deadlock detected (...)
```
Retries to handle this error are similar to serialization errors (40001).
## Savepoints
[Savepoints](../../../../api/ysql/the-sql-language/statements/savepoint_create/) are named checkpoints that can be used to rollback just a few statements, and then proceed with the transaction, rather than aborting the entire transaction when there is an error.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,9 @@ Each update performed as a part of the transaction is replicated across multiple

### Concurrency control

[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. The two primary mechanisms to achieve concurrency control are optimistic and pessimistic.
[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. YugabyteDB currently supports two concurrency control mechanisms: Wait-on-Conflict and Fail-on-Conflict.

YugabyteDB currently supports optimistic concurrency control, with pessimistic concurrency control being worked on actively.
These are orthogonal to isolation levels (except for Read Committed isolation which has the same behaviour in both modes barring some limitations and performance penalty in Fail-on-Conflict mode). Also note that the Wait-on-Conflict concurrency control mode exactly matches PostgreSQL semantics and also gives better performance.

## Transaction options

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -329,12 +329,12 @@ After a transaction T1 (that was waiting for other transactions) unblocks, it co

### Examples

The following examples describe different use cases detailing the Wait-on-Conflict behavior.
The following examples describe different use cases detailing the Wait-on-Conflict behavior. To run the examples, you need to do the following:

1. Note that the examples require you to set the YB-TServer flag `enable_wait_queues=true`.
1. Also, set the YB-TServer flag `ysql_max_write_restart_attempts=0` to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production.
1. Set the YB-TServer flag `enable_wait_queues=true`.
1. Set the per-session `yb_max_query_layer_retries=0` YSQL configuration parameter to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production. To set it at the cluster level, use the `ysql_pg_conf_csv` YB-TServer flag.

A restart is necessary for these flags to take effect.
A restart is necessary for the flags to take effect.

Start by setting up the table you'll use in all of the examples in this section.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,25 @@ The client can reconnect to the server and retry the transaction.
This error occurs when a transaction cannot be applied or progress further because of other conflicting transactions. For example, when multiple transactions are modifying the same key.

```output
ERROR: 40001: Operation expired: Transaction XXXX expired or aborted by a conflict
ERROR: could not serialize access due to concurrent update (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Serialization failure errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 40P01: Deadlock detected

This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted and they fail with the following error.

```output
ERROR: deadlock detected (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Deadlock detected errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 2D000: Invalid transaction termination

This error occurs when a transaction is terminated either by a `COMMIT` or a `ROLLBACK` in an invalid location. For example, when a `COMMIT` is issued inside a stored procedure that is called from inside a transaction.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.
Expand Down Expand Up @@ -74,19 +74,11 @@ If the `COMMIT` is successful, the program exits the loop. `attempt < max_attemp
##### 40001 - SerializationFailure
SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict) or when transactions are waiting on each other (deadlock). The error messages could be one of the following types:
SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict). During a conflict, certain transactions are retried.
- During a conflict, certain transactions are retried. However, after the retry limit is reached, an error occurs as follows:
```output
ERROR: 40001: All transparent retries exhausted.
```
- All transactions are given a dynamic priority. When a deadlock is detected, the transaction with lower priority is automatically killed. For this scenario, the client might receive a message similar to the following:
```output
ERROR: 40001: Operation expired: Heartbeat: Transaction XXXX expired or aborted by a conflict
```
```output
ERROR: could not serialize access due to concurrent update (...)
```
The correct way to handle this error is with a retry loop with exponential backoff, as described in [Client-side retry](#client-side-retry). When the [UPDATE](../../../../api/ysql/the-sql-language/statements/dml_update/) or [COMMIT](../../../../api/ysql/the-sql-language/statements/txn_commit/) fails because of `SerializationFailure`, the code retries after waiting for `sleep_time` seconds, up to `max_attempts`.
Expand All @@ -96,6 +88,17 @@ In read committed isolation level, as the server retries internally, the client
Another way to handle these failures is would be to rollback to a checkpoint before the failed statement and proceed further as described in [Savepoints](#savepoints).
##### 40001 - Deadlock detected
This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted
and they fail with the following error:
```output
ERROR: deadlock detected (...)
```
Retries to handle this error are similar to serialization errors (40001).
## Savepoints
[Savepoints](../../../../api/ysql/the-sql-language/statements/savepoint_create/) are named checkpoints that can be used to rollback just a few statements, and then proceed with the transaction, rather than aborting the entire transaction when there is an error.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,9 @@ Each update performed as a part of the transaction is replicated across multiple

### Concurrency control

[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. The two primary mechanisms to achieve concurrency control are optimistic and pessimistic.
[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. YugabyteDB currently supports two concurrency control mechanisms: Wait-on-Conflict and Fail-on-Conflict.

YugabyteDB currently supports optimistic concurrency control, with pessimistic concurrency control being worked on actively.
These are orthogonal to isolation levels (except for Read Committed isolation which has the same behaviour in both modes barring some limitations and performance penalty in Fail-on-Conflict mode). Also note that the Wait-on-Conflict concurrency control mode exactly matches PostgreSQL semantics and also gives better performance.

## Transaction options

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.
Expand Down

0 comments on commit f957dda

Please sign in to comment.