Skip to content

Commit

Permalink
Merge a6ea2b6 into 26c9911
Browse files Browse the repository at this point in the history
  • Loading branch information
blinkov authored Sep 10, 2024
2 parents 26c9911 + a6ea2b6 commit cff8080
Show file tree
Hide file tree
Showing 516 changed files with 3,478 additions and 2,607 deletions.
18 changes: 9 additions & 9 deletions ydb/docs/.yfmlint
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@ log-levels:
MD002: 'error' # First heading should be a top-level heading
MD003: 'disabled' # Heading style
MD004: 'disabled' # Unordered list style
MD005: 'error' # Inconsistent indentation for list items at the same level
MD005: 'disabled' # Inconsistent indentation for list items at the same level
MD006: 'error' # Consider starting bulleted lists at the beginning of the line
MD007: 'error' # Unordered list indentation
MD007: 'disabled' # Unordered list indentation
MD009: 'error' # Trailing spaces
MD010: 'error' # Hard tabs
MD011: 'error' # Reversed link syntax
MD012: 'disabled' # Multiple consecutive blank lines
MD013: 'disabled' # Line length
MD014: 'disabled' # Dollar signs used before commands without showing output
MD018: 'disabled' # No space after hash on atx style heading
MD019: 'disabled' # Multiple spaces after hash on atx style heading
MD020: 'disabled' # No space inside hashes on closed atx style heading
MD021: 'disabled' # Multiple spaces inside hashes on closed atx style heading
MD019: 'error' # Multiple spaces after hash on atx style heading
MD020: 'error' # No space inside hashes on closed atx style heading
MD021: 'error' # Multiple spaces inside hashes on closed atx style heading
MD022: 'disabled' # Headings should be surrounded by blank lines
MD023: 'disabled' # Headings must start at the beginning of the line
MD024: 'disabled' # Multiple headings with the same content
Expand All @@ -29,11 +29,11 @@ log-levels:
MD032: 'disabled' # Lists should be surrounded by blank lines
MD033: 'disabled' # Inline HTML
MD034: 'disabled' # Bare URL used
MD035: 'disabled' # Horizontal rule style
MD035: 'error' # Horizontal rule style
MD036: 'disabled' # Emphasis used instead of a heading
MD037: 'error' # Spaces inside emphasis markers
MD037: 'disabled' # Spaces inside emphasis markers
MD038: 'error' # Spaces inside code span elements
MD039: 'error' # Spaces inside link text
MD039: 'disabled' # Spaces inside link text
MD040: 'disabled' # Fenced code blocks should have a language specified
MD041: 'disabled' # First line in a file should be a top-level heading
MD042: 'error' # No empty links
Expand Down Expand Up @@ -64,4 +64,4 @@ log-levels:

# Inline code length
YFM001:
maximum: 140
maximum: 140
78 changes: 23 additions & 55 deletions ydb/docs/en/core/changelog-cli.md

Large diffs are not rendered by default.

32 changes: 16 additions & 16 deletions ydb/docs/en/core/changelog-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@

Release date: May 14, 2024.

**Performance:**
### Performance

* [Fixed](https://github.com/ydb-platform/ydb/pull/3638) an issue of increased CPU consumption by a topic actor `PERSQUEUE_PARTITION_ACTOR`.
* [Optimized](https://github.com/ydb-platform/ydb/pull/2083) resource usage by SchemeBoard replicas. The greatest effect is noticeable when modifying the metadata of tables with a large number of partitions.

**Bug fixes:**
### Bug fixes

* [Fixed a bug](https://github.com/ydb-platform/ydb/pull/2169) of possible partial commit of accumulated changes when using persistent distributed transactions. This error occurs in an extremely rare combination of events, including restarting tablets that service the table partitions involved in the transaction.
* [Fixed a bug](https://github.com/ydb-platform/ydb/pull/3165) involving a race condition between the table merge and garbage collection processes, which could result in garbage collection ending with an invariant violation error, leading to an abnormal termination of the `ydbd` server process.
Expand All @@ -24,7 +24,7 @@ Release date: May 14, 2024.

Release date: October 12, 2023.

**Functionality:**
### Functionality

* Implemented visibility of own changes. With this feature enabled you can read changed values from the current transaction, which has not been committed yet. This functionality also allows multiple modifying operations in one transaction on a table with secondary indexes.
* Added support for [column tables](concepts/datamodel/table.md#column-tables). It is now possible to create analytical reports based on stored data in YDB with performance comparable to specialized analytical DBMS.
Expand All @@ -38,21 +38,21 @@ Release date: October 12, 2023.
* **_(Experimental)_** Added support for PostgreSQL compatibility. This involves executing SQL queries in PostgreSQL dialect on the YDB infrastructure using the PostgreSQL network protocol. With this capability, familiar PostgreSQL tools such as psql and drivers (e.g., pq for Golang and psycopg2 for Python) can be used. Queries can be developed using the familiar PostgreSQL syntax and take advantage of YDB's benefits such as horizontal scalability and fault tolerance.
* **_(Experimental)_** Added support for federated queries. This enables retrieving information from various data sources without the need to move the data into YDB. Federated queries support interaction with ClickHouse and PostgreSQL databases, as well as S3 class data stores (Object Storage). YQL queries can be used to access these databases without duplicating data between systems.

**Embedded UI:**
### Embedded UI

* A new option `PostgreSQL` has been added to the query type selector settings, which is available when the `Enable additional query modes` parameter is enabled. Also, the query history now takes into account the syntax used when executing the query.
* The YQL query template for creating a table has been updated. Added a description of the available parameters.
* Now sorting and filtering for Storage and Nodes tables takes place on the server. To use this functionality, you need to enable the parameter `Offload tables filters and sorting to backend` in the experiments section.
* Buttons for creating, changing and deleting [topics](concepts/topic.md) have been added to the context menu.
* Added sorting by criticality for all issues in the tree in `Healthcheck`.

**Performance:**
### Performance

* Implemented read iterators. This feature allows to separate reads and computations. Read iterators allow datashards to increase read queries throughput.
* The performance of writing to YDB topics has been optimized.
* Improved tablet balancing during node overload.

**Bug fixes:**
### Bug fixes

* Fixed an error regarding potential blocking of reading iterators of snapshots, of which the coordinators were unaware.
* Memory leak when closing the connection in Kafka proxy has been fixed.
Expand Down Expand Up @@ -130,12 +130,12 @@ Release date: October 12, 2023.

Release date: August 14, 2023.

**Functionality:**
### Functionality

* **_(Experimental)_** Implemented visibility of own changes. With this feature enabled you can read changed values from the current transaction, which has not been committed yet. This functionality also allows multiple modifying operations in one transaction on a table with secondary indexes. To enable this feature add `enable_kqp_immediate_effects: true` under `table_service_config` section into [configuration file](deploy/configuration/config.md).
* **_(Experimental)_** Implemented read iterators. This feature allows to separate reads and computations. Read iterators allow datashards to increase read queries throughput. To enable this feature add `enable_kqp_data_query_source_read: true` under `table_service_config` section into [configuration file](deploy/configuration/config.md).

**Embedded UI:**
### Embedded UI

* Navigation improvements:
* Diagnostics and Development mode switches are moved to the left panel.
Expand All @@ -145,13 +145,13 @@ Release date: August 14, 2023.
* Info tab for scheme objects displays parameters using terms from `CREATE` or `ALTER` statements.
* Added [column tables](concepts/datamodel/table.md#column-tables) support.

**Performance:**
### Performance

* For scan queries, you can now effectively search for individual rows using a primary key or secondary indexes. This can bring you a substantial performance gain in many cases. Similarly to regular queries, you need to explicitly specify its name in the query text using the `VIEW` keyword to use a secondary index.

* **_(Experimental)_** Added an option to give control of the system tablets of the database (SchemeShard, Coordinators, Mediators, SysViewProcessor) to its own Hive instead of the root Hive, and do so immediately upon creating a new database. Without this flag, the system tablets of the new database are created in the root Hive, which can negatively impact its load. Enabling this flag makes databases completely isolated in terms of load, that may be particularly relevant for installations, consisting from a roughly hundred or more databases. To enable this feature add `alter_database_create_hive_first: true` under `feature_flags` section into [configuration file](deploy/configuration/config.md).

**Bug fixes:**
### Bug fixes

* Fixed a bug in the autoconfiguration of the actor system, resulting in all the load being placed on the system pool.
* Fixed a bug that caused full scanning when searching by prefix of the primary key using `LIKE`.
Expand All @@ -171,21 +171,21 @@ Release date: August 14, 2023.

Release date: May 5, 2023. To update to version 23.1, select the [Downloads](downloads/index.md#ydb-server) section.

**Functionality:**
### Functionality

* Added [initial table scan](concepts/cdc.md#initial-scan) when creating a CDC changefeed. Now, you can export all the data existing at the time of changefeed creation.
* Added [atomic index replacement](dev/secondary-indexes.md#atomic-index-replacement). Now, you can atomically replace one pre-defined index with another. This operation is absolutely transparent for your application. Indexes are replaced seamlessly, with no downtime.
* Added the [audit log](security/audit-log.md): Event stream including data about all the operations on {{ ydb-short-name }} objects.

**Performance:**
### Performance

* Improved formats of data exchanged between query stages. As a result, we accelerated SELECTs by 10% on parameterized queries and by up to 30% on write operations.
* Added [autoconfiguring](deploy/configuration/config.md#autoconfig) for the actor system pools based on the workload against them. This improves performance through more effective CPU sharing.
* Optimized the predicate logic: Processing of parameterized OR or IN constraints is automatically delegated to DataShard.
* (Experimental) For scan queries, you can now effectively search for individual rows using a primary key or secondary indexes. This can bring you a substantial gain in performance in many cases. Similarly to regular queries, to use a secondary index, you need to explicitly specify its name in the query text using the `VIEW` keyword.
* The query's computational graph is now cached at query runtime, reducing the CPU resources needed to build the graph.

**Bug fixes:**
### Bug fixes

* Fixed bugs in the distributed data warehouse implementation. We strongly recommend all our users to upgrade to the latest version.
* Fixed the error that occurred on building an index on NOT NULL columns.
Expand All @@ -197,13 +197,13 @@ Release date: May 5, 2023. To update to version 23.1, select the [Downloads](dow

Release date: March 7, 2023. To update to version **22.5**, select the [Downloads](downloads/index.md#ydb-server) section.

**What's new:**
### What's new

* Added [changefeed configuration parameters](yql/reference/syntax/alter_table.md#changefeed-options) to transfer additional information about changes to a topic.
* You can now [rename tables](concepts/datamodel/table.md#rename) that have TTL enabled.
* You can now [manage the record retention period](concepts/cdc.md#retention-period).

**Bug fixes and improvements:**
### Bug fixes and improvements

* Fixed an error inserting 0 rows with a BulkUpsert.
* Fixed an error importing Date/DateTime columns from CSV.
Expand All @@ -216,7 +216,7 @@ Release date: March 7, 2023. To update to version **22.5**, select the [Download

Release date: October 12, 2022. To update to version **22.4**, select the [Downloads](downloads/index.md#ydb-server) section.

**What's new:**
### What's new

* {{ ydb-short-name }} Topics and Change Data Capture (CDC):

Expand Down
2 changes: 2 additions & 0 deletions ydb/docs/en/core/concepts/_includes/transactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,10 @@ By default, {{ ydb-short-name }} transactions are executed in *Serializable* mod
If consistency or freshness requirement for data read by a transaction can be relaxed, a user can take advantage of execution modes with lower guarantees:

* *Online Read-Only*: Each read operation in the transaction is reading the data that is most recent at execution time. The consistency of retrieved data depends on the *allow_inconsistent_reads* setting:

* *false* (consistent reads): Each individual read operation returns consistent data, but no consistency is guaranteed between reads. Reading the same table range twice may return different results.
* *true* (inconsistent reads): Even the data fetched by a particular read operation may contain inconsistent results.

* *Stale Read-Only*: Read operations within a transaction may return results that are slightly out-of-date (lagging by fractions of a second). Each individual read returns consistent data, but no consistency between different reads is guaranteed.
* *Snapshot Read-Only*: All the read operations within a transaction access the database snapshot. All the data reads are consistent. The snapshot is taken when the transaction begins, meaning the transaction sees all changes committed before it began.

Expand Down
10 changes: 3 additions & 7 deletions ydb/docs/en/core/concepts/auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,21 +44,17 @@ Authentication by username and password includes the following steps:
1. The client accesses the database and presents their username and password to the {{ ydb-short-name }} authentication service.

You can only use lower case Latin letters and digits in usernames. No restrictions apply to passwords (empty passwords can be used).

1. The authentication service passes authentication data to the {{ ydb-short-name }} authentication component.
1. The component validates authentication data. If the data matches, it generates a token and returns it to the authentication service.

Tokens accelerate authentication and strengthen security. Tokens have a default lifetime of 12 hours. YDB SDK rotates tokens by accessing the authentication service.

The username and hashed password are stored in the table inside the authentication component. The password is hashed by the [Argon2]{% if lang == "en" %}(https://en.wikipedia.org/wiki/Argon2){% endif %}{% if lang == "ru" %}(https://ru.wikipedia.org/wiki/Argon2){% endif %} method. In authentication mode, only the system administrator can use a username/password pair to access the table.

1. The authentication system returns the token to the client.
1. The client accesses the database, presenting their token as authentication data.

To enable username/password authentication, use `true` in the `enforce_user_token_requirement` key of the cluster's [configuration file](../deploy/configuration/config.md#auth).

To learn how to manage roles and users, see [{#T}](../security/access-management.md).

<!-- ### API получения токенов IAM {#token-refresh-api}
Для ротации токенов {{ ydb-short-name }} SDK и CLI используют gRPC-запрос к {{ yandex-cloud }} IAM API [IamToken - create]{% if lang == "en" %}(https://cloud.yandex.com/en/docs/iam/api-ref/grpc/iam_token_service#Create){% endif %}{% if lang == "ru" %}(https://cloud.yandex.ru/docs/iam/api-ref/grpc/iam_token_service#Create){% endif %}. В режиме **Refresh Token** заданный параметром OAuth токен передается в атрибуте `yandex_passport_oauth_token`. В режиме **Service Account Key** на основании заданных атрибутов сервисной учетной записи и ключа шифрования формируется короткоживущий JWT-токен и передается в атрибуте `jwt`. Исходный код формирования JWT-токена доступен в составе SDK (например, метод `get_jwt()` в [коде на Python](https://github.com/ydb-platform/ydb-python-sdk/blob/main/ydb/iam/auth.py)).
{{ ydb-short-name }} SDK и CLI позволяют подменить хост для обращения к API получения токенов, что дает возможность реализовать аналогичный API в корпоративных контекстах. -->
To learn how to manage roles and users, see [{#T}](../security/access-management.md).
Loading

0 comments on commit cff8080

Please sign in to comment.