Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FULL OUTER JOIN: Logical error: 'Bad cast from type DB::ColumnString to DB::ColumnNullable'. #19362

Closed
qoega opened this issue Jan 21, 2021 · 3 comments · Fixed by #21248
Closed

FULL OUTER JOIN: Logical error: 'Bad cast from type DB::ColumnString to DB::ColumnNullable'. #19362

qoega opened this issue Jan 21, 2021 · 3 comments · Fixed by #21248
Labels
fuzz Problem found by one of the fuzzers

Comments

@qoega
Copy link
Member

qoega commented Jan 21, 2021

Describe the bug
https://clickhouse-test-reports.s3.yandex.net/19316/9ae3628b662f5f9b34695a3f4f7b07307323fc8b/fuzzer_tsan/report.html#fail1

How to reproduce
Tried to reproduce locally, but it worked without data

CREATE TABLE foo
(
    `server_date` Date,
    `dimension_1` String,
    `metric_1` UInt32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(server_date)
ORDER BY server_date

CREATE TABLE bar
(
    `server_date` Date,
    `dimension_1` String,
    `metric_2` UInt32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(server_date)
ORDER BY server_date

SELECT dimension_1, sum_metric_1, sum_metric_2 FROM (SELECT dimension_1, sum(metric_1) AS sum_metric_1 FROM foo GROUP BY dimension_1 WITH TOTALS) AS subquery_1 ALL FULL OUTER JOIN (SELECT dimension_1, sum(metric_2) AS sum_metric_2 FROM bar GROUP BY dimension_1 WITH TOTALS) AS subquery_2 USING (dimension_1) WHERE sum_metric_2 < NULL ORDER BY dimension_1 ASC
2021.01.21 14:38:20.107827 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Fatal> : Logical error: 'Bad cast from type DB::ColumnString to DB::ColumnNullable'.
2021.01.21 14:38:20.108891 [ 147 ] {} <Fatal> BaseDaemon: ########################################
2021.01.21 14:38:20.109223 [ 147 ] {} <Fatal> BaseDaemon: (version 21.1.1.5748, build id: A13A5A1D2B64279205A7F80714E67C76ED64DBC7) (from thread 77) (query_id: ff7d1781-d7e2-4596-ac76-b8b721f05908) Received signal Aborted (6)
2021.01.21 14:38:20.109468 [ 147 ] {} <Fatal> BaseDaemon:
2021.01.21 14:38:20.109756 [ 147 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f349fe6318b 0x7f349fe42859 0x108a9b46 0x108a9bf1 0x10b09c04 0x196db4d7 0x19712f80 0x1a01eb8c 0x1a01efd7 0x1abb6bb3 0x1abb4892 0x1abae7d6 0x1abb9fc8 0x1e3f10fc 0x1e3f1900 0x1e530963 0x1e52d7ed 0x1e52c678 0x7f34a0018609 0x7f349ff3f293
2021.01.21 14:38:20.110203 [ 147 ] {} <Fatal> BaseDaemon: 4. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.01.21 14:38:20.110363 [ 147 ] {} <Fatal> BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.01.21 14:38:20.110890 [ 147 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:50: DB::handle_error_code(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x108a9b46 in /workspace/clickhouse
2021.01.21 14:38:20.111316 [ 147 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:56: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x108a9bf1 in /workspace/clickhouse
2021.01.21 14:38:20.115652 [ 147 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Common/assert_cast.h:47: DB::ColumnNullable const& assert_cast<DB::ColumnNullable const&, DB::IColumn const&>(DB::IColumn const&) @ 0x10b09c04 in /workspace/clickhouse
2021.01.21 14:38:20.150828 [ 147 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/DataTypes/DataTypeNullable.cpp:92: DB::DataTypeNullable::serializeBinaryBulkWithMultipleStreamsImpl(DB::IColumn const&, unsigned long, unsigned long, DB::IDataType::SerializeBinaryBulkSettings&, std::__1::shared_ptr<DB::IDataType::SerializeBinaryBulkState>&) const @ 0x196db4d7 in /workspace/clickhouse
2021.01.21 14:38:20.184167 [ 147 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/DataTypes/IDataType.cpp:287: DB::IDataType::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::IDataType::SerializeBinaryBulkSettings&, std::__1::shared_ptr<DB::IDataType::SerializeBinaryBulkState>&) const @ 0x19712f80 in /workspace/clickhouse
2021.01.21 14:38:20.220331 [ 147 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/DataStreams/NativeBlockOutputStream.cpp:58: DB::NativeBlockOutputStream::writeData(DB::IDataType const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::WriteBuffer&, unsigned long, unsigned long) @ 0x1a01eb8c in /workspace/clickhouse
2021.01.21 14:38:20.256528 [ 147 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../src/DataStreams/NativeBlockOutputStream.cpp:124: DB::NativeBlockOutputStream::write(DB::Block const&) @ 0x1a01efd7 in /workspace/clickhouse
2021.01.21 14:38:20.301459 [ 147 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:713: DB::TCPHandler::sendTotals(DB::Block const&) @ 0x1abb6bb3 in /workspace/clickhouse
2021.01.21 14:38:20.346480 [ 147 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:642: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x1abb4892 in /workspace/clickhouse
2021.01.21 14:38:20.390877 [ 147 ] {} <Fatal> BaseDaemon: 15. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:277: DB::TCPHandler::runImpl() @ 0x1abae7d6 in /workspace/clickhouse
2021.01.21 14:38:20.435869 [ 147 ] {} <Fatal> BaseDaemon: 16. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417: DB::TCPHandler::run() @ 0x1abb9fc8 in /workspace/clickhouse
2021.01.21 14:38:20.487235 [ 147 ] {} <Fatal> BaseDaemon: 17. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x1e3f10fc in /workspace/clickhouse
2021.01.21 14:38:20.538425 [ 147 ] {} <Fatal> BaseDaemon: 18. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:112: Poco::Net::TCPServerDispatcher::run() @ 0x1e3f1900 in /workspace/clickhouse
2021.01.21 14:38:20.589951 [ 147 ] {} <Fatal> BaseDaemon: 19. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199: Poco::PooledThread::run() @ 0x1e530963 in /workspace/clickhouse
2021.01.21 14:38:20.642154 [ 147 ] {} <Fatal> BaseDaemon: 20. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:56: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x1e52d7ed in /workspace/clickhouse
2021.01.21 14:38:20.694410 [ 147 ] {} <Fatal> BaseDaemon: 21. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345: Poco::ThreadImpl::runnableEntry(void*) @ 0x1e52c678 in /workspace/clickhouse
2021.01.21 14:38:20.694872 [ 147 ] {} <Fatal> BaseDaemon: 22. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
2021.01.21 14:38:20.695241 [ 147 ] {} <Fatal> BaseDaemon: 23. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.01.21 14:38:21.645514 [ 147 ] {} <Fatal> BaseDaemon: Calculated checksum of the binary: 21CF627C42A57CBD36677FCEAAE2085B. There is no information about the reference checksum.
2021.01.21 14:38:31.622759 [ 68 ] {} <Fatal> Application: Child process was terminated by signal 6.

2021.01.21 14:38:20.065603 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> executeQuery: (from [::1]:49620, using production parser) SELECT dimension_1, sum_metric_1, sum_metric_2 FROM (SELECT dimension_1, sum(metric_1) AS sum_metric_1 FROM foo GROUP BY dimension_1 WITH TOTALS) AS subquery_1 ALL FULL OUTER JOIN (SELECT dimension_1, sum(metric_2) AS sum_metric_2 FROM bar GROUP BY dimension_1 WITH TOTALS) AS subquery_2 USING (dimension_1) WHERE sum_metric_2 < NULL ORDER BY dimension_1 ASC
2021.01.21 14:38:20.068542 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> ContextAccess (default): Access granted: SELECT(dimension_1, metric_1) ON default.foo
2021.01.21 14:38:20.070805 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> ContextAccess (default): Access granted: SELECT(dimension_1, metric_2) ON default.bar
2021.01.21 14:38:20.073383 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> ContextAccess (default): Access granted: SELECT(dimension_1, metric_1) ON default.foo
2021.01.21 14:38:20.076820 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> ContextAccess (default): Access granted: SELECT(dimension_1, metric_2) ON default.bar
2021.01.21 14:38:20.078239 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> default.bar (1c923af5-c234-4905-b24e-79cc8abce11f) (SelectExecutor): Key condition: unknown
2021.01.21 14:38:20.078357 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> default.bar (1c923af5-c234-4905-b24e-79cc8abce11f) (SelectExecutor): MinMax index condition: unknown
2021.01.21 14:38:20.078461 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> default.bar (1c923af5-c234-4905-b24e-79cc8abce11f) (SelectExecutor): Not using primary index on part 202001_1_1_0
2021.01.21 14:38:20.078563 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> default.bar (1c923af5-c234-4905-b24e-79cc8abce11f) (SelectExecutor): Selected 1 parts by partition key, 1 parts by primary key, 1 marks by primary key, 1 marks to read from 1 ranges
2021.01.21 14:38:20.079029 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> default.bar (1c923af5-c234-4905-b24e-79cc8abce11f) (SelectExecutor): Reading approx. 2 rows with 1 streams
2021.01.21 14:38:20.079529 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2021.01.21 14:38:20.081010 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> HashJoin: Right sample block: subquery_2.dimension_1 String String(size = 0), sum_metric_2 UInt64 UInt64(size = 0)
2021.01.21 14:38:20.087090 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> default.foo (440b87bd-055c-4e04-9d78-e2b6e316e086) (SelectExecutor): Key condition: unknown
2021.01.21 14:38:20.087211 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> default.foo (440b87bd-055c-4e04-9d78-e2b6e316e086) (SelectExecutor): MinMax index condition: unknown
2021.01.21 14:38:20.087308 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> default.foo (440b87bd-055c-4e04-9d78-e2b6e316e086) (SelectExecutor): Not using primary index on part 202001_1_1_0
2021.01.21 14:38:20.087424 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> default.foo (440b87bd-055c-4e04-9d78-e2b6e316e086) (SelectExecutor): Selected 1 parts by partition key, 1 parts by primary key, 1 marks by primary key, 1 marks to read from 1 ranges
2021.01.21 14:38:20.087851 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> default.foo (440b87bd-055c-4e04-9d78-e2b6e316e086) (SelectExecutor): Reading approx. 2 rows with 1 streams
2021.01.21 14:38:20.088320 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2021.01.21 14:38:20.089351 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2021.01.21 14:38:20.103011 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> AggregatingTransform: Aggregating
2021.01.21 14:38:20.103114 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> Aggregator: Aggregation method: key_string
2021.01.21 14:38:20.103420 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> AggregatingTransform: Aggregated. 2 to 2 rows (from 36.00 B) in 0.006889853 sec. (290.2819552173319 rows/sec., 5.10 KiB/sec.)
2021.01.21 14:38:20.103509 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> Aggregator: Merging aggregated data
2021.01.21 14:38:20.104376 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> CreatingSetsTransform: Creating join.
2021.01.21 14:38:20.104810 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> CreatingSetsTransform: Created Join with 2 entries from 2 rows in 0.000419498 sec.
2021.01.21 14:38:20.105497 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Debug> MergingSortedTransform: Merge sorted 1 blocks, 0 rows in 0.008 sec., 0.0 rows/sec., 26.86 KiB/sec
2021.01.21 14:38:20.106979 [ 144 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> PipelineExecutor: Thread finished. Total time: 0.004590221 sec. Execution time: 0.002985379 sec. Processing time: 0.001532918 sec. Wait time: 7.1924e-05 sec.
2021.01.21 14:38:20.107047 [ 141 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> PipelineExecutor: Thread finished. Total time: 0.004615111 sec. Execution time: 0.000593121 sec. Processing time: 1.0846e-05 sec. Wait time: 0.004011144 sec.
2021.01.21 14:38:20.107069 [ 143 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Trace> PipelineExecutor: Thread finished. Total time: 0.004653067 sec. Execution time: 1.3484e-05 sec. Processing time: 5.579e-06 sec. Wait time: 0.004634004 sec.
2021.01.21 14:38:20.107827 [ 77 ] {ff7d1781-d7e2-4596-ac76-b8b721f05908} <Fatal> : Logical error: 'Bad cast from type DB::ColumnString to DB::ColumnNullable'.
2021.01.21 14:38:20.109223 [ 147 ] {} <Fatal> BaseDaemon: (version 21.1.1.5748, build id: A13A5A1D2B64279205A7F80714E67C76ED64DBC7) (from thread 77) (query_id: ff7d1781-d7e2-4596-ac76-b8b721f05908) Received signal Aborted (6)
@qoega qoega added the fuzz Problem found by one of the fuzzers label Jan 21, 2021
@qoega qoega mentioned this issue Jan 21, 2021
Merged
@alexey-milovidov alexey-milovidov mentioned this issue Jan 25, 2021
Closed
@alexey-milovidov
Copy link
Member

https://clickhouse-test-reports.s3.yandex.net/20543/0c34d1c565b964353284425c0b54a98f837098bc/fuzzer_ubsan/server.log

@alexey-milovidov alexey-milovidov mentioned this issue Feb 17, 2021
Closed
@alexey-milovidov
Copy link
Member

alexey-milovidov commented Feb 17, 2021 •
edited
Loading

Changed settings:

max_threads = '1',
max_distributed_connections = '1',
totals_mode = 'after_having_auto',
group_by_two_level_threshold = '1000',
distributed_aggregation_memory_efficient = '1',
optimize_skip_unused_shards = '1',
log_queries = '1',
log_queries_min_type = 'QUERY_FINISH',
distributed_product_mode = 'local',
join_use_nulls = '1',
joined_subquery_requires_alias = '0',
max_rows_to_group_by = '1000',
group_by_overflow_mode = 'any',
mutations_sync = '1',
optimize_move_functions_out_of_any = '0',
enable_global_with_statement = '1',
allow_experimental_map_type = '1',
input_format_null_as_default = '0',
input_format_values_interpret_expressions = '1',
input_format_values_accurate_types_of_literals = '0'

@alexey-milovidov alexey-milovidov changed the title Logical error: 'Bad cast from type DB::ColumnString to DB::ColumnNullable'. FULL OUTER JOIN: Logical error: 'Bad cast from type DB::ColumnString to DB::ColumnNullable'. Feb 20, 2021
@alexey-milovidov
Copy link
Member

https://clickhouse-test-reports.s3.yandex.net/20989/7474a7e3ca139f1a4e88e83af011b304ebdcaf3c/fuzzer_ubsan/server.log

@alexey-milovidov alexey-milovidov mentioned this issue Feb 20, 2021
Merged
@azat azat mentioned this issue Feb 26, 2021
Merged
@vdimir vdimir mentioned this issue Feb 27, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
fuzz Problem found by one of the fuzzers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Convert columns to nullable for totals in join vdimir/ClickHouse
2 participants
@qoega @alexey-milovidov