Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YQ-3617: fix GROUP BY HOP + AS_TABLE #9370

Merged
merged 15 commits into from
Oct 7, 2024

Conversation

APozdniakov
Copy link
Collaborator

Changelog entry

GROUP BY HOP now supports static inputs like AS_TABLE

Changelog category

  • New feature

@APozdniakov APozdniakov self-assigned this Sep 17, 2024
Copy link

github-actions bot commented Sep 17, 2024

2024-09-17 11:43:56 UTC Pre-commit check linux-x86_64-release-clang14 for 4603e46 has started.
2024-09-17 11:44:05 UTC Artifacts will be uploaded here
2024-09-17 11:46:59 UTC ya make is running...
🟢 2024-09-17 11:55:40 UTC Build successful.

Copy link

github-actions bot commented Sep 17, 2024

2024-09-17 11:43:59 UTC Pre-commit check linux-x86_64-release-asan for 4603e46 has started.
2024-09-17 11:44:10 UTC Artifacts will be uploaded here
2024-09-17 11:47:02 UTC ya make is running...
🔴 2024-09-17 13:34:20 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13563 13350 0 47 57 109

🟢 2024-09-17 13:35:36 UTC Build successful.
🔴 2024-09-17 13:36:12 UTC ydbd size 5.6 GiB changed* by +3.3 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 238a753 merge: 4603e46 diff diff %
ydbd size 6 045 711 792 Bytes 6 049 142 776 Bytes +3.3 MiB +0.057%
ydbd stripped size 1 513 156 368 Bytes 1 513 504 464 Bytes +339.9 KiB +0.023%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 17, 2024

2024-09-17 11:44:41 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4603e46 has started.
2024-09-17 11:44:51 UTC Artifacts will be uploaded here
2024-09-17 11:47:51 UTC ya make is running...
🟡 2024-09-17 13:09:01 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
75943 61873 0 10 14024 36

2024-09-17 13:16:49 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-17 13:24:39 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
282 (only retried tests) 188 0 0 7 87

🟢 2024-09-17 13:24:48 UTC Build successful.
🔴 2024-09-17 13:25:28 UTC ydbd size 8.4 GiB changed* by +3.0 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 238a753 merge: 4603e46 diff diff %
ydbd size 9 031 030 152 Bytes 9 034 154 432 Bytes +3.0 MiB +0.035%
ydbd stripped size 488 766 248 Bytes 488 854 120 Bytes +85.8 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 08:30:27 UTC Pre-commit check linux-x86_64-release-asan for 7783891 has started.
2024-09-19 08:31:10 UTC Artifacts will be uploaded here
2024-09-19 08:34:14 UTC Check cancelled

Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 08:31:19 UTC Pre-commit check linux-x86_64-release-clang14 for 7783891 has started.
2024-09-19 08:31:29 UTC Artifacts will be uploaded here
2024-09-19 08:34:14 UTC Check cancelled

Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 08:31:27 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7783891 has started.
2024-09-19 08:31:36 UTC Artifacts will be uploaded here
2024-09-19 08:34:15 UTC Check cancelled

Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 08:35:40 UTC Pre-commit check linux-x86_64-release-clang14 for d5247f0 has started.
2024-09-19 08:35:43 UTC Artifacts will be uploaded here
2024-09-19 08:38:43 UTC ya make is running...
🟢 2024-09-19 08:49:19 UTC Build successful.

Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 08:36:56 UTC Pre-commit check linux-x86_64-relwithdebinfo for d5247f0 has started.
2024-09-19 08:37:41 UTC Artifacts will be uploaded here
2024-09-19 08:41:08 UTC ya make is running...
🟡 2024-09-19 09:55:39 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76013 61907 0 31 14042 33

2024-09-19 10:02:54 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-09-19 10:10:23 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
301 (only retried tests) 177 0 23 7 94

2024-09-19 10:10:35 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-09-19 10:17:46 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
122 (only retried tests) 5 0 23 0 94

🟢 2024-09-19 10:18:00 UTC Build successful.
🟡 2024-09-19 10:18:42 UTC ydbd size 8.4 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: a430757 merge: d5247f0 diff diff %
ydbd size 9 037 972 232 Bytes 9 039 157 592 Bytes +1.1 MiB +0.013%
ydbd stripped size 489 049 640 Bytes 489 058 920 Bytes +9.1 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 08:38:10 UTC Pre-commit check linux-x86_64-release-asan for d5247f0 has started.
2024-09-19 08:38:19 UTC Artifacts will be uploaded here
2024-09-19 08:41:14 UTC ya make is running...
🔴 2024-09-19 10:28:50 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13553 13352 0 41 46 114

🟢 2024-09-19 10:30:00 UTC Build successful.
🟡 2024-09-19 10:30:38 UTC ydbd size 5.6 GiB changed* by +1.0 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: a430757 merge: d5247f0 diff diff %
ydbd size 6 054 203 864 Bytes 6 055 292 912 Bytes +1.0 MiB +0.018%
ydbd stripped size 1 514 633 424 Bytes 1 514 697 648 Bytes +62.7 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 20, 2024

2024-09-20 13:59:49 UTC Pre-commit check linux-x86_64-release-asan for dd71fc9 has started.
2024-09-20 14:00:04 UTC Artifacts will be uploaded here
2024-09-20 14:03:39 UTC ya make is running...
🔴 2024-09-20 15:53:15 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13611 13378 0 56 60 117

🟢 2024-09-20 15:54:27 UTC Build successful.
🟡 2024-09-20 15:55:02 UTC ydbd size 5.6 GiB changed* by +1.0 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 89871e4 merge: dd71fc9 diff diff %
ydbd size 6 062 009 512 Bytes 6 063 102 168 Bytes +1.0 MiB +0.018%
ydbd stripped size 1 516 450 096 Bytes 1 516 515 056 Bytes +63.4 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 20, 2024

2024-09-20 13:59:52 UTC Pre-commit check linux-x86_64-relwithdebinfo for dd71fc9 has started.
2024-09-20 14:00:02 UTC Artifacts will be uploaded here
2024-09-20 14:02:57 UTC ya make is running...
🟡 2024-09-20 15:19:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76063 61965 0 6 14052 40

2024-09-20 15:27:15 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-20 15:38:48 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
283 (only retried tests) 176 0 0 7 100

🟢 2024-09-20 15:38:56 UTC Build successful.
🟡 2024-09-20 15:39:33 UTC ydbd size 8.4 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 89871e4 merge: dd71fc9 diff diff %
ydbd size 9 043 468 784 Bytes 9 044 656 680 Bytes +1.1 MiB +0.013%
ydbd stripped size 489 388 296 Bytes 489 397 832 Bytes +9.3 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 20, 2024

2024-09-20 14:00:01 UTC Pre-commit check linux-x86_64-release-clang14 for dd71fc9 has started.
2024-09-20 14:00:55 UTC Artifacts will be uploaded here
2024-09-20 14:04:48 UTC ya make is running...
🟢 2024-09-20 14:13:18 UTC Build successful.

@APozdniakov APozdniakov marked this pull request as ready for review September 23, 2024 07:57
@APozdniakov APozdniakov requested review from a team as code owners September 23, 2024 07:57
uzhastik
uzhastik previously approved these changes Sep 23, 2024
@@ -5080,6 +5186,11 @@ void RegisterCoSimpleCallables1(TCallableOptimizerMap& map) {
return clean;
}

if (auto hopping = RewriteAsHoppingWindow(node, ctx); hopping) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

здесь не нужна ;, упрощай условие

@@ -5080,6 +5186,11 @@ void RegisterCoSimpleCallables1(TCallableOptimizerMap& map) {
return clean;
}

if (auto hopping = RewriteAsHoppingWindow(node, ctx); hopping) {
YQL_CLOG(DEBUG, Core) << "RewriteAggregate";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

тут можно вывести что-то более релевантное? при чем тут RewriteAggregate ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Печатаю название примененной оптимизации с названием как у dq-шной. Мне кажется очень релевантно + строка лога легко ищется по коду

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RewriteAsHoppingWindow тут более уместно

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Я тоже за более релевантное сообщение. Добавь туда что-нибудь про hopping

Copy link

github-actions bot commented Oct 1, 2024

2024-10-01 13:58:13 UTC Pre-commit check linux-x86_64-relwithdebinfo for 2969cf6 has started.
2024-10-01 13:58:23 UTC Artifacts will be uploaded here
2024-10-01 14:01:18 UTC ya make is running...
🟡 2024-10-01 15:00:25 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76205 62230 0 4 13838 133

2024-10-01 15:07:42 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-01 15:19:04 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
130 (only retried tests) 30 0 0 0 100

🟢 2024-10-01 15:19:11 UTC Build successful.
🟡 2024-10-01 15:19:33 UTC ydbd size 2.8 GiB changed* by +372.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: a5e7134 merge: 2969cf6 diff diff %
ydbd size 3 002 118 304 Bytes 3 002 499 896 Bytes +372.6 KiB +0.013%
ydbd stripped size 477 387 784 Bytes 477 401 480 Bytes +13.4 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 3, 2024

2024-10-03 14:15:07 UTC Pre-commit check linux-x86_64-release-asan for 0e21834 has started.
2024-10-03 14:15:18 UTC Artifacts will be uploaded here
2024-10-03 14:18:19 UTC ya make is running...
🟡 2024-10-03 15:50:16 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13446 13213 0 71 56 106

🟢 2024-10-03 15:51:23 UTC Build successful.
🟡 2024-10-03 15:51:56 UTC ydbd size 5.7 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 60d8681 merge: 0e21834 diff diff %
ydbd size 6 086 042 752 Bytes 6 087 163 944 Bytes +1.1 MiB +0.018%
ydbd stripped size 1 520 516 272 Bytes 1 520 584 304 Bytes +66.4 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 3, 2024

2024-10-03 14:16:18 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0e21834 has started.
2024-10-03 14:16:29 UTC Artifacts will be uploaded here
2024-10-03 14:19:30 UTC ya make is running...
🟡 2024-10-03 15:22:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76235 62286 0 5 13836 108

2024-10-03 15:30:31 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-10-03 15:41:43 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
111 (only retried tests) 6 0 4 0 101

2024-10-03 15:41:53 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-10-03 15:53:18 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
105 (only retried tests) 0 0 4 0 101

🟢 2024-10-03 15:53:25 UTC Build successful.
🟡 2024-10-03 15:53:46 UTC ydbd size 2.8 GiB changed* by +388.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 4f9fd4f merge: 0e21834 diff diff %
ydbd size 3 001 749 784 Bytes 3 002 147 752 Bytes +388.6 KiB +0.013%
ydbd stripped size 477 341 432 Bytes 477 356 536 Bytes +14.8 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 4, 2024

2024-10-04 12:50:26 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4b9a079 has started.
2024-10-04 12:50:37 UTC Artifacts will be uploaded here
2024-10-04 12:53:32 UTC ya make is running...
🟡 2024-10-04 13:52:53 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76258 62290 0 6 13836 126

2024-10-04 14:00:15 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-10-04 14:11:50 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
130 (only retried tests) 6 0 4 0 120

2024-10-04 14:11:58 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-10-04 14:23:08 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
124 (only retried tests) 21 0 4 0 99

🟢 2024-10-04 14:23:15 UTC Build successful.
🟡 2024-10-04 14:23:35 UTC ydbd size 2.8 GiB changed* by +388.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ce8c6eb merge: 4b9a079 diff diff %
ydbd size 3 001 253 376 Bytes 3 001 651 360 Bytes +388.7 KiB +0.013%
ydbd stripped size 477 265 272 Bytes 477 279 352 Bytes +13.8 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 4, 2024

2024-10-04 12:50:56 UTC Pre-commit check linux-x86_64-release-asan for 4b9a079 has started.
2024-10-04 12:51:07 UTC Artifacts will be uploaded here
2024-10-04 12:54:02 UTC ya make is running...
🟡 2024-10-04 14:23:52 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13460 13200 0 77 78 105

🟢 2024-10-04 14:25:00 UTC Build successful.
🟡 2024-10-04 14:25:34 UTC ydbd size 5.7 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ce8c6eb merge: 4b9a079 diff diff %
ydbd size 6 082 132 296 Bytes 6 083 260 200 Bytes +1.1 MiB +0.019%
ydbd stripped size 1 519 808 368 Bytes 1 519 881 328 Bytes +71.2 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 4, 2024

2024-10-04 15:27:44 UTC Pre-commit check linux-x86_64-release-asan for 306cfda has started.
2024-10-04 15:27:55 UTC Artifacts will be uploaded here
2024-10-04 15:31:02 UTC ya make is running...
🟡 2024-10-04 17:13:29 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13516 13103 0 133 157 123

🟢 2024-10-04 17:14:39 UTC Build successful.
🟡 2024-10-04 17:15:18 UTC ydbd size 5.7 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 631b912 merge: 306cfda diff diff %
ydbd size 6 084 304 280 Bytes 6 085 431 992 Bytes +1.1 MiB +0.019%
ydbd stripped size 1 520 424 272 Bytes 1 520 497 264 Bytes +71.3 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 4, 2024

2024-10-04 15:28:15 UTC Pre-commit check linux-x86_64-relwithdebinfo for 306cfda has started.
2024-10-04 15:28:20 UTC Artifacts will be uploaded here
2024-10-04 15:31:12 UTC ya make is running...
🟡 2024-10-04 16:31:49 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76275 62323 0 2 13848 102

2024-10-04 16:39:25 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-04 16:48:20 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
102 (only retried tests) 5 0 0 0 97

🟢 2024-10-04 16:48:27 UTC Build successful.
🟡 2024-10-04 16:48:46 UTC ydbd size 2.8 GiB changed* by +392.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 631b912 merge: 306cfda diff diff %
ydbd size 3 002 209 456 Bytes 3 002 611 400 Bytes +392.5 KiB +0.013%
ydbd stripped size 477 419 384 Bytes 477 437 624 Bytes +17.8 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@APozdniakov APozdniakov merged commit 7a861b9 into ydb-platform:main Oct 7, 2024
10 checks passed
APozdniakov added a commit to APozdniakov/ydb that referenced this pull request Oct 8, 2024
APozdniakov added a commit to APozdniakov/ydb that referenced this pull request Oct 9, 2024
APozdniakov added a commit to APozdniakov/ydb that referenced this pull request Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants