Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge/stable 24 2/query float test asan #8196

Conversation

StekPerepolnen
Copy link
Collaborator

Changelog entry

...

Changelog category

  • New feature
  • Experimental feature
  • Improvement
  • Performance improvement
  • Bugfix
  • Backward incompatible change
  • Documentation (changelog entry is not required)
  • Not for changelog (changelog entry is not required)

Additional information

...

adameat and others added 30 commits February 21, 2024 10:16
Co-authored-by: nsofya <[email protected]>
Co-authored-by: nsofya <[email protected]>
…2391)

Cherry-pick 3819aed from main (ydb-platform#2083).

Make schemeboard replicas consume less CPU, especially when processing rapid updates for tables with huge amount of partitions.

Schema information on a path exist in the form of `DescribeSchemeResult` object: schemeshard generates those objects and publishes them to the schemeboard, schemeboard notifies scheme-caches on the nodes about path info changes. So schemeshard generates `DescribeSchemeResult`, scheme-cache serves `DescribeSchemeResult` to the consumers. But schemeboard components in-between do not require the full content of a TEvDescribeSchemeResult to operate efficiently.

This update enables the schemeboard to transmit `DescribeSchemeResult` through as an opaque payload rather than as a fully detailed protobuf object. Thus reducing the unnecessary memory management and serialization/deserialization overhead.
…latform#1960) (ydb-platform#2479)

KIKIMR-21002

In this PR we add the following algorithm for invalidating cache entries for outdated VIEWs:

1. Store path ids and schema versions of the views that were used in the query in the cache entries, so they can be accessed later.
2. Whenever we retrieve a compilation result from cache, send a request for SchemeCache to check if the schema version of the views used in this query (if any) has not changed since we compiled this query.
3. Send a recompilation request if any view is outdated.

There are two important things to note about this solution:

- We make a SchemeCache request for each repeated query and there is a lot of these in an OLTP-focused database like YDB. However, we have already been sending these request for preliminary (this is not the last check of schema version mismatch (at least for tables)) cache invalidation for tables, so views should not incur an additional performance impact here.
- This solution does not guarantee strong consistency for queries using views, because query cache invalidation will not happen instantly after the view definition is updated. The node should get an update from the SchemeCache, which takes some time.
maximyurchuk and others added 22 commits July 29, 2024 20:56
…ls-features

Merge mTLS features to stable-24-2
Copy link

🔴 Unable to merge your PR into the main branch. Please rebase or merge it with the main branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet