Fix max parallel streams for joined pipelines

[vdimir]

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

• Bug Fix

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fix wrong thread estimation for right subquery join in some cases. Close #24075

[vdimir]

Failed tests:

AST fuzzer (debug) — Assertion `frame_start <= frame_end' failed

Not related to changes, see #26072

Performance — 4 errors, 2 too long, 2 faster

Added cases was too long on current master

Stress test (thread) — Test script failed

Seems like exception in memory tracking

Log/StackTrace

2021.07.07 23:07:23.023218 [ 447 ] {} <Fatal> BaseDaemon: (version 21.8.1.7387, build id: D886AE6AEAADD3F273D67F5CF862222EF6620B52) (from thread 59067) Terminate called for uncaught exception:
Code: 241, e.displayText() = DB::Exception: Memory limit (total) exceeded: would use 63.05 GiB (attempt to allocate chunk of 4195678 bytes), maximum: 62.94 GiB, Stack trace (when copying this message, always include the lines below):

0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x1834f59b in /usr/bin/clickhouse
1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:59: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x951849c in /usr/bin/clickhouse
2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0x94f792b in /usr/bin/clickhouse
3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:226: MemoryTracker::allocImpl(long, bool) @ 0x94f67f8 in /usr/bin/clickhouse
4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool) @ 0x94f656b in /usr/bin/clickhouse
5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool) @ 0x94f656b in /usr/bin/clickhouse
6. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool) @ 0x94f656b in /usr/bin/clickhouse
7. ./obj-x86_64-linux-gnu/../src/Common/CurrentMemoryTracker.cpp:45: CurrentMemoryTracker::(anonymous namespace)::allocImpl(long, bool) @ 0x9538db9 in /usr/bin/clickhouse
8. ./obj-x86_64-linux-gnu/../src/Common/CurrentMemoryTracker.cpp:61: CurrentMemoryTracker::alloc(long) @ 0x9538c5f in /usr/bin/clickhouse
9. ./obj-x86_64-linux-gnu/../src/Common/Allocator.h:96: void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 0ul, 0ul>::resize<>(unsigned long) @ 0x1388750d in /usr/bin/clickhouse
10. ./obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:0: DB::CompressedWriteBuffer::nextImpl() @ 0x13892a33 in /usr/bin/clickhouse
11. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0x13892d4f in /usr/bin/clickhouse
12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::unique_ptr<DB::TinyLogBlockOutputStream::Stream, std::__1::default_delete<DB::TinyLogBlockOutputStream::Stream> > >, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::unique_ptr<DB::TinyLogBlockOutputStream::Stream, std::__1::default_delete<DB::TinyLogBlockOutputStream::Stream> > >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::unique_ptr<DB::TinyLogBlockOutputStream::Stream, std::__1::default_delete<DB::TinyLogBlockOutputStream::Stream> > > > >::destroy(std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::unique_ptr<DB::TinyLogBlockOutputStream::Stream, std::__1::default_delete<DB::TinyLogBlockOutputStream::Stream> > >, void*>*) @ 0x14af447d in /usr/bin/clickhouse
13. ./obj-x86_64-linux-gnu/.
2021.07.07 23:07:23.024213 [ 447 ] {} <Trace> BaseDaemon: Received signal 6
2021.07.07 23:07:23.025673 [ 94565 ] {} <Fatal> BaseDaemon: ########################################
2021.07.07 23:07:23.026226 [ 94565 ] {} <Fatal> BaseDaemon: (version 21.8.1.7387, build id: D886AE6AEAADD3F273D67F5CF862222EF6620B52) (from thread 59067) (query_id: 53b234f8-b604-4b79-ad87-696241d5bcdc) Received signal Aborted (6)
2021.07.07 23:07:23.026507 [ 94565 ] {} <Fatal> BaseDaemon: 
2021.07.07 23:07:23.027339 [ 94565 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f80533e918b 0x7f80533c8859 0x942e4ae 0x135049fd 0x1a9468b4 0x1a9467a7 0x94baebb 0x13892ed1 0x14af447d 0x14af4436 0x14af3a87 0x14af663e 0x140973be 0x1408cb29 0x140909e5 0x1408cd69 0x1408e845 0x1408ce89 0x13ca0ae6 0x13c6e669 0x1520262a 0x13771129 0x14fc91a9 0x1395c235 0x1395c2f1 0x14f785b2 0x14f66242 0x14f75e88 0x1826d2a3 0x1826da30 0x183e4082 0x183e2510 0x183e0c88 0x9428cad 0x7f805359e609 0x7f80534c5293
2021.07.07 23:07:23.027940 [ 94565 ] {} <Fatal> BaseDaemon: 5. gsignal @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.07.07 23:07:23.028442 [ 94565 ] {} <Fatal> BaseDaemon: 6. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.07.07 23:07:39.680715 [ 94565 ] {} <Fatal> BaseDaemon: 7. abort @ 0x942e4ae in /usr/bin/clickhouse
2021.07.07 23:07:40.455393 [ 94565 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../base/daemon/BaseDaemon.cpp:0: terminate_handler() @ 0x135049fd in /usr/bin/clickhouse
2021.07.07 23:07:40.487388 [ 94565 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:61: std::__terminate(void (*)()) @ 0x1a9468b4 in /usr/bin/clickhouse
2021.07.07 23:07:40.515578 [ 94565 ] {} <Fatal> BaseDaemon: 10.1. inlined from ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/include/atomic_support.h:78: void (*std::__1::(anonymous namespace)::__libcpp_atomic_load<void (*)()>(void (* const*)(), int))()
2021.07.07 23:07:40.515923 [ 94565 ] {} <Fatal> BaseDaemon: 10.2. inlined from ../contrib/libcxxabi/src/cxa_handlers.cpp:49: std::get_terminate()
2021.07.07 23:07:40.516059 [ 94565 ] {} <Fatal> BaseDaemon: 10. ../contrib/libcxxabi/src/cxa_handlers.cpp:92: std::terminate() @ 0x1a9467a7 in /usr/bin/clickhouse
2021.07.07 23:08:40.729089 [ 442 ] {} <Fatal> Application: Child process was terminated by signal 6.

---

Looks like flaky tests, not related:

Integration tests (asan) — fail: 7, passed: 1532, flaky: 0

test_cluster_copier, test_prometheus_endpoint

Integration tests (release) — fail: 11, passed: 1528, flaky: 3

test_cluster_copier, test_MemoryTracking, test_grpc_protocol, test_jdbc_bridge

Integration tests (thread) — fail: 6, passed: 1529, flaky: 1

test_replicated_mutations, test_rename_column, test_cluster_copier, test_prometheus_endpoint