Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YQ-3345 support load cpu threshold #6790

Conversation

GrigoriyPA
Copy link
Collaborator

@GrigoriyPA GrigoriyPA commented Jul 17, 2024

Changelog entry

Support load cpu threshold

Changelog category

  • New feature

Additional information

Эти изменения нужны в пределах задачи YQ-3222: workload manager v1 (YDB DWH к скейлу), все фичи которые должны быть к скейлу согласовали с @fomichev3000 в этой дельте:
https://delta.yandex-team.ru/page/epad/page/oq95gk1890527ptp

Ещё нужно будет доделать изменения по этим тикетам:

  • YQ-3442: Оптимизировать default pool
  • YQ-3444: Пробросить pool id из connection string
  • YQ-3446: Добавить время ожидания в очереди в статистику

@GrigoriyPA GrigoriyPA requested a review from a team as a code owner July 17, 2024 14:04
@GrigoriyPA GrigoriyPA changed the title YQ-3345 support load cpu threshold (#6360) YQ-3345 support load cpu threshold Jul 17, 2024
@GrigoriyPA GrigoriyPA requested a review from spuchin July 17, 2024 14:06
Copy link

github-actions bot commented Jul 17, 2024

2024-07-17 14:07:52 UTC Pre-commit check for 8512c14 has started.
2024-07-17 14:10:21 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-07-17 14:17:16 UTC Build failed. see the build logs.

Copy link

github-actions bot commented Jul 17, 2024

2024-07-17 14:08:13 UTC Pre-commit check for 8512c14 has started.
2024-07-17 14:10:44 UTC Build linux-x86_64-release-asan is running...
🔴 2024-07-17 14:16:37 UTC Build failed. see the build logs.
🔴 2024-07-17 14:16:38 UTC Tests run skipped.

Copy link

github-actions bot commented Jul 17, 2024

2024-07-17 14:09:42 UTC Pre-commit check for 8512c14 has started.
2024-07-17 14:12:13 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-07-17 14:18:24 UTC Build failed. see the build logs.
🔴 2024-07-17 14:18:25 UTC Tests run skipped.

Copy link
Collaborator

@maximyurchuk maximyurchuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Напишите плз, зачем это нужно в 24-3

Copy link

github-actions bot commented Jul 18, 2024

2024-07-18 03:43:48 UTC Pre-commit check for 4b6db08 has started.
2024-07-18 03:46:18 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-18 04:25:34 UTC Build successful.
2024-07-18 04:25:50 UTC Tests are running...
🔴 2024-07-18 06:05:16 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39614 34380 0 2 5221 11

🟡 2024-07-18 06:05:57 UTC ydbd size 8.1 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: df15684 merge: 4b6db08 diff diff %
ydbd size 8 727 367 992 Bytes 8 728 687 456 Bytes +1.3 MiB +0.015%
ydbd stripped size 477 191 944 Bytes 477 237 384 Bytes +44.4 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 18, 2024

2024-07-18 03:43:49 UTC Pre-commit check for 4b6db08 has started.
2024-07-18 03:46:19 UTC Build linux-x86_64-release-clang14 is running...

Copy link

github-actions bot commented Jul 18, 2024

2024-07-18 03:43:53 UTC Pre-commit check for 4b6db08 has started.
2024-07-18 03:46:31 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-18 04:10:49 UTC Build successful.
2024-07-18 04:11:13 UTC Tests are running...
🔴 2024-07-18 06:16:20 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11110 10532 0 86 317 175

🟡 2024-07-18 06:17:09 UTC ydbd size 5.2 GiB changed* by +730.7 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: df15684 merge: 4b6db08 diff diff %
ydbd size 5 616 613 272 Bytes 5 617 361 536 Bytes +730.7 KiB +0.013%
ydbd stripped size 1 211 263 016 Bytes 1 211 398 248 Bytes +132.1 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA
Copy link
Collaborator Author

GrigoriyPA commented Jul 18, 2024

Напишите плз, зачем это нужно в 24-3

Эти изменения нужны в пределах задачи YQ-3222: workload manager v1 (YDB DWH к скейлу), все фичи которые должны быть к скейлу согласовали с @fomichev3000 в этой дельте:
https://delta.yandex-team.ru/page/epad/page/oq95gk1890527ptp

Ещё нужно будет доделать изменения по этим тикетам:

  • YQ-3442: Оптимизировать default pool
  • YQ-3444: Пробросить pool id из connection string
  • YQ-3446: Добавить время ожидания в очереди в статистику

@GrigoriyPA GrigoriyPA added the rebase-and-check Rebase PR with the current main and check label Jul 19, 2024
@github-actions github-actions bot removed the rebase-and-check Rebase PR with the current main and check label Jul 19, 2024
Copy link

github-actions bot commented Jul 19, 2024

2024-07-19 04:24:53 UTC Pre-commit check for 4b6db08 has started.
2024-07-19 04:27:33 UTC Build linux-x86_64-release-clang14 is running...

Copy link

github-actions bot commented Jul 19, 2024

2024-07-19 04:25:15 UTC Pre-commit check for 4b6db08 has started.
2024-07-19 04:27:55 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-19 04:51:08 UTC Build successful.
2024-07-19 04:51:25 UTC Tests are running...
🔴 2024-07-19 06:53:18 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11109 10533 0 84 320 172

🟡 2024-07-19 06:54:03 UTC ydbd size 5.2 GiB changed* by +730.7 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: df15684 merge: 4b6db08 diff diff %
ydbd size 5 616 613 272 Bytes 5 617 361 536 Bytes +730.7 KiB +0.013%
ydbd stripped size 1 211 263 016 Bytes 1 211 398 248 Bytes +132.1 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 19, 2024

2024-07-19 04:25:28 UTC Pre-commit check for 4b6db08 has started.
2024-07-19 04:28:02 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-19 05:05:57 UTC Build successful.
2024-07-19 05:06:14 UTC Tests are running...
🟢 2024-07-19 06:56:08 UTC Tests successful.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39613 34381 0 0 5219 13

🟡 2024-07-19 06:56:51 UTC ydbd size 8.1 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: df15684 merge: 4b6db08 diff diff %
ydbd size 8 727 367 992 Bytes 8 728 687 456 Bytes +1.3 MiB +0.015%
ydbd stripped size 477 191 944 Bytes 477 237 384 Bytes +44.4 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 22, 2024

2024-07-22 12:35:05 UTC Pre-commit check for d606f15 has started.
2024-07-22 12:37:56 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-22 13:16:41 UTC Build successful.

Copy link

github-actions bot commented Jul 22, 2024

2024-07-22 12:37:05 UTC Pre-commit check for d606f15 has started.
2024-07-22 12:39:51 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-22 13:23:58 UTC Build successful.
2024-07-22 13:24:14 UTC Tests are running...
🔴 2024-07-22 15:21:30 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11247 10673 0 84 317 173

🟡 2024-07-22 15:22:12 UTC ydbd size 5.2 GiB changed* by +820.2 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: 56c68e1 merge: d606f15 diff diff %
ydbd size 5 616 622 944 Bytes 5 617 462 824 Bytes +820.2 KiB +0.015%
ydbd stripped size 1 211 263 400 Bytes 1 211 414 536 Bytes +147.6 KiB +0.012%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 22, 2024

2024-07-22 12:37:37 UTC Pre-commit check for d606f15 has started.
2024-07-22 12:40:23 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-22 13:39:14 UTC Build successful.
2024-07-22 13:39:28 UTC Tests are running...
🔴 2024-07-22 15:21:28 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39762 34528 0 1 5221 12

🟡 2024-07-22 15:22:16 UTC ydbd size 8.1 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: 56c68e1 merge: d606f15 diff diff %
ydbd size 8 727 377 488 Bytes 8 728 883 096 Bytes +1.4 MiB +0.017%
ydbd stripped size 477 192 200 Bytes 477 240 200 Bytes +46.9 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 22, 2024

2024-07-22 16:56:18 UTC Pre-commit check for 3168f97 has started.
2024-07-22 16:59:05 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-22 17:23:53 UTC Build successful.
2024-07-22 17:24:09 UTC Tests are running...
🔴 2024-07-22 19:24:32 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11248 10672 0 86 319 171

🟡 2024-07-22 19:25:24 UTC ydbd size 5.2 GiB changed* by +820.8 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: 4515805 merge: 3168f97 diff diff %
ydbd size 5 616 779 216 Bytes 5 617 619 720 Bytes +820.8 KiB +0.015%
ydbd stripped size 1 211 284 392 Bytes 1 211 435 528 Bytes +147.6 KiB +0.012%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 22, 2024

2024-07-22 16:56:35 UTC Pre-commit check for 3168f97 has started.
2024-07-22 16:59:25 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-22 17:40:46 UTC Build successful.
2024-07-22 17:41:03 UTC Tests are running...
🔴 2024-07-22 19:25:38 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39766 34532 0 1 5219 14

🟡 2024-07-22 19:26:21 UTC ydbd size 8.1 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: 4515805 merge: 3168f97 diff diff %
ydbd size 8 727 463 664 Bytes 8 728 965 472 Bytes +1.4 MiB +0.017%
ydbd stripped size 477 202 184 Bytes 477 246 088 Bytes +42.9 KiB +0.009%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 22, 2024

2024-07-22 16:57:57 UTC Pre-commit check for 3168f97 has started.
2024-07-22 17:00:51 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-22 17:09:46 UTC Build successful.

Copy link

github-actions bot commented Jul 23, 2024

2024-07-23 04:19:29 UTC Pre-commit check for 2c36f2b has started.
2024-07-23 04:22:25 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-23 05:12:52 UTC Build successful.
2024-07-23 05:13:04 UTC Tests are running...
🔴 2024-07-23 07:17:21 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15688 15083 0 92 341 172

🟡 2024-07-23 07:18:16 UTC ydbd size 5.2 GiB changed* by +821.0 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: b87fc88 merge: 2c36f2b diff diff %
ydbd size 5 616 779 200 Bytes 5 617 619 896 Bytes +821.0 KiB +0.015%
ydbd stripped size 1 211 284 392 Bytes 1 211 435 688 Bytes +147.8 KiB +0.012%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 23, 2024

2024-07-23 04:19:42 UTC Pre-commit check for 2c36f2b has started.
2024-07-23 04:22:44 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-23 05:30:12 UTC Build successful.
2024-07-23 05:30:29 UTC Tests are running...
🔴 2024-07-23 07:37:09 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76739 63297 0 1 13428 13

🟡 2024-07-23 07:38:00 UTC ydbd size 8.1 GiB changed* by +1.4 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: b87fc88 merge: 2c36f2b diff diff %
ydbd size 8 727 463 480 Bytes 8 728 963 112 Bytes +1.4 MiB +0.017%
ydbd stripped size 477 202 184 Bytes 477 246 152 Bytes +42.9 KiB +0.009%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 23, 2024

2024-07-23 04:21:40 UTC Pre-commit check for 2c36f2b has started.
2024-07-23 04:24:46 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-23 05:13:53 UTC Build successful.

Copy link

github-actions bot commented Jul 24, 2024

2024-07-24 08:41:42 UTC Pre-commit check for b7845db has started.
2024-07-24 08:44:39 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-24 08:53:23 UTC Build successful.

Copy link

github-actions bot commented Jul 24, 2024

2024-07-24 08:41:49 UTC Pre-commit check for b7845db has started.
2024-07-24 08:44:46 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-24 09:26:56 UTC Build successful.
2024-07-24 09:27:08 UTC Tests are running...
🔴 2024-07-24 11:30:03 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76739 63300 0 3 13425 11

🟡 2024-07-24 11:30:36 UTC ydbd size 8.1 GiB changed* by +1.5 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: bf5f3d7 merge: b7845db diff diff %
ydbd size 8 727 584 672 Bytes 8 729 202 200 Bytes +1.5 MiB +0.019%
ydbd stripped size 477 316 552 Bytes 477 370 056 Bytes +52.2 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 24, 2024

2024-07-24 08:42:05 UTC Pre-commit check for b7845db has started.
2024-07-24 08:45:05 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-24 09:12:46 UTC Build successful.
2024-07-24 09:13:01 UTC Tests are running...
🔴 2024-07-24 11:27:15 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15689 15056 0 98 361 174

🟡 2024-07-24 11:28:06 UTC ydbd size 5.2 GiB changed* by +924.4 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: bf5f3d7 merge: b7845db diff diff %
ydbd size 5 616 909 560 Bytes 5 617 856 104 Bytes +924.4 KiB +0.017%
ydbd stripped size 1 211 406 792 Bytes 1 211 576 008 Bytes +165.2 KiB +0.014%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

2024-07-24 15:29:54 UTC Pre-commit check for 9b2b1ba has started.

Copy link

github-actions bot commented Jul 24, 2024

2024-07-24 15:30:18 UTC Pre-commit check for 9b2b1ba has started.
2024-07-24 15:33:31 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-24 16:12:12 UTC Build successful.
2024-07-24 16:12:25 UTC Tests are running...
🟢 2024-07-24 18:13:01 UTC Tests successful.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76738 63299 0 0 13427 12

🟡 2024-07-24 18:13:33 UTC ydbd size 8.1 GiB changed* by +1.5 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: bf5f3d7 merge: 9b2b1ba diff diff %
ydbd size 8 727 584 672 Bytes 8 729 203 120 Bytes +1.5 MiB +0.019%
ydbd stripped size 477 316 552 Bytes 477 370 056 Bytes +52.2 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 24, 2024

2024-07-24 15:30:26 UTC Pre-commit check for 9b2b1ba has started.
2024-07-24 15:33:26 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-24 15:58:25 UTC Build successful.
2024-07-24 15:58:39 UTC Tests are running...
🔴 2024-07-24 18:13:15 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15689 15069 0 100 347 173

🟡 2024-07-24 18:14:11 UTC ydbd size 5.2 GiB changed* by +934.9 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: bf5f3d7 merge: 9b2b1ba diff diff %
ydbd size 5 616 909 560 Bytes 5 617 866 928 Bytes +934.9 KiB +0.017%
ydbd stripped size 1 211 406 792 Bytes 1 211 580 104 Bytes +169.2 KiB +0.014%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA added the rebase-and-check Rebase PR with the current main and check label Jul 25, 2024
@github-actions github-actions bot removed the rebase-and-check Rebase PR with the current main and check label Jul 25, 2024
Copy link

github-actions bot commented Jul 25, 2024

2024-07-25 10:55:26 UTC Pre-commit check for 9b2b1ba has started.
2024-07-25 10:58:30 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-25 11:05:07 UTC Build successful.

Copy link

github-actions bot commented Jul 25, 2024

2024-07-25 10:56:42 UTC Pre-commit check for 9b2b1ba has started.
2024-07-25 11:00:09 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-25 11:39:45 UTC Build successful.
2024-07-25 11:39:56 UTC Tests are running...
🔴 2024-07-25 13:49:16 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76739 63298 0 1 13427 13

🟡 2024-07-25 13:49:55 UTC ydbd size 8.1 GiB changed* by +1.5 MiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: bf5f3d7 merge: 9b2b1ba diff diff %
ydbd size 8 727 584 672 Bytes 8 729 203 120 Bytes +1.5 MiB +0.019%
ydbd stripped size 477 316 552 Bytes 477 370 056 Bytes +52.2 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 25, 2024

2024-07-25 10:56:46 UTC Pre-commit check for 9b2b1ba has started.
2024-07-25 11:00:13 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-25 11:24:17 UTC Build successful.
2024-07-25 11:24:32 UTC Tests are running...
🔴 2024-07-25 13:41:01 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15689 15064 0 107 346 172

🟡 2024-07-25 13:41:46 UTC ydbd size 5.2 GiB changed* by +934.9 KiB, which is >= 100.0 KiB vs stable-24-3: Warning

ydbd size dash stable-24-3: bf5f3d7 merge: 9b2b1ba diff diff %
ydbd size 5 616 909 560 Bytes 5 617 866 928 Bytes +934.9 KiB +0.017%
ydbd stripped size 1 211 406 792 Bytes 1 211 580 104 Bytes +169.2 KiB +0.014%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA merged commit 90805a4 into ydb-platform:stable-24-3 Jul 25, 2024
11 of 17 checks passed
@GrigoriyPA GrigoriyPA deleted the merge-to-stable-WM-load-cpu-threshold branch July 25, 2024 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants