Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDL creation of a distributed table with empty cluster '' returns exception but with wrong table created, restart will fail. #27832

Closed
zhangjmruc opened this issue Aug 19, 2021 · 0 comments · Fixed by #27927
Assignees
Labels
potential bug To be reviewed by developers and confirmed/rejected.

Comments

@zhangjmruc
Copy link
Contributor

zhangjmruc commented Aug 19, 2021

When I tried to create a distributed table with empty cluster '', the create statement failed with exception "Code: 170. DB::Exception: Received from localhost:9000. DB::Exception: Requested cluster '' not found.". However, show tables can see the WRONG table.
Later when CH restarted, unable to start due to unable to attach this wrong table.

The question is: When we try to startup the new created table during creation, if error happens, this new table should be dropped? OR we should add some special check on the cluster for Distributed engine to avoid this kind of error? OR we can just ignore this error during start up to allow CK server to start, not failed?

CH version: 21.7.3.14 (official build).
How to reproduce:

  1. Create wrong table:
    create table t_empty(a int) engine=Distributed('','','');
  2. Force CH to restart, and check the status
    service clickhouse-server forcerestart
    service clickhouse-server status
    Now there is no clickhouse-server process.

##### output ###
node236 :) create table t_empty(a int) engine=Distributed('','','');

CREATE TABLE t_empty
(
a int
)
ENGINE = Distributed('', '', '')

Query id: c9eb24ea-f87d-4431-9bdc-18d21fbb5636

0 rows in set. Elapsed: 0.054 sec.

Received exception from server (version 21.7.3):
Code: 170. DB::Exception: Received from localhost:9000. DB::Exception: Requested cluster '' not found.

node236 :) show tables;

SHOW TABLES

Query id: 0d9e2398-26c5-46eb-baf2-a02111bc770e

┌─name────┐
│ people │
t_empty
└─────────┘

2 rows in set. Elapsed: 0.001 sec.

node236 :) show create table t_empty;

SHOW CREATE TABLE t_empty

Query id: 8c489c90-dfe0-4392-9742-17a43a075c12

┌─statement───────────────────────────────────────────────────────────────────┐
│ CREATE TABLE default.t_empty
(
a Int32
)
ENGINE = Distributed('', '', '')
└─────────────────────────────────────────────────────────────────────────────┘

==== clickhouse-server.err.log ===

backtrace for DDL

2021.08.19 12:12:03.150990 [ 19900 ] {c9eb24ea-f87d-4431-9bdc-18d21fbb5636} TCPHandler: Code: 170, e.displayText() = DB::Exception: Requested cluster '' n
ot found, Stack trace:

  1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, int, bool) @ 0x8d31b5a in /usr/bin/click
    house
  2. DB::Context::getCluster(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) const @ 0xf73b378 in /usr/bin/clickhouse
  3. DB::StorageDistributed::getCluster() const @ 0x100081f7 in /usr/bin/clickhouse
  4. DB::StorageDistributed::startup() @ 0x1000e5b2 in /usr/bin/clickhouse
  5. DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&) @ 0xf875226 in /usr/bin/clickhouse
  6. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0xf8711e3 in /usr/bin/clickhouse
  7. DB::InterpreterCreateQuery::execute() @ 0xf87735c in /usr/bin/clickhouse
  8. ? @ 0xfe22253 in /usr/bin/clickhouse
  9. DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::shared_ptrDB::Context, bool, DB::Que
    ryProcessingStage::Enum, bool) @ 0xfe208e3 in /usr/bin/clickhouse
  10. DB::TCPHandler::runImpl() @ 0x1069f6c2 in /usr/bin/clickhouse
  11. DB::TCPHandler::run() @ 0x106b25d9 in /usr/bin/clickhouse
  12. Poco::Net::TCPServerConnection::start() @ 0x1338b30f in /usr/bin/clickhouse
  13. Poco::Net::TCPServerDispatcher::run() @ 0x1338cd9a in /usr/bin/clickhouse
  14. Poco::PooledThread::run() @ 0x134bfc19 in /usr/bin/clickhouse
  15. Poco::ThreadImpl::runnableEntry(void*) @ 0x134bbeaa in /usr/bin/clickhouse
  16. start_thread @ 0x7ea5 in /usr/lib64/libpthread-2.17.so
  17. __clone @ 0xfe9fd in /usr/lib64/libc-2.17.so

backtrace for start up of CH

2021.08.19 12:22:47.254734 [ 25174 ] {} Application: Caught exception while loading metadata: Code: 170, e.displayText() = DB::Exception: Requested cluster '' not found: while loading database default from path /var/lib/clickhouse/metadata/default, Stack trace (when copying this message, always include the lines below):

  1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, int, bool) @ 0x8d31b5a in /usr/bin/clickhouse
  2. DB::Context::getCluster(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) const @ 0xf73b378 in /usr/bin/clickhouse
  3. DB::StorageDistributed::getCluster() const @ 0x100081f7 in /usr/bin/clickhouse
  4. DB::StorageDistributed::startup() @ 0x1000e5b2 in /usr/bin/clickhouse
  5. ? @ 0xf646a3b in /usr/bin/clickhouse
  6. ThreadPoolImpl::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0x8d75738 in /usr/bin/clickhouse
  7. ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl::scheduleImpl(std::__1::function<void ()>, int, std::__1::optional)::'lambda0'()>(void&&, void ThreadPoolImpl::scheduleImpl(std::__1::function<void ()>, int, std::__1::optional)::'lambda0'()&&...)::'lambda'()::operator()() @ 0x8d772df in /usr/bin/clickhouse
  8. ThreadPoolImplstd::__1::thread::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x8d72a1f in /usr/bin/clickhouse
  9. ? @ 0x8d76303 in /usr/bin/clickhouse
  10. start_thread @ 0x7ea5 in /usr/lib64/libpthread-2.17.so
  11. __clone @ 0xfe9fd in /usr/lib64/libc-2.17.so
    (version 21.7.3.14 (official build))
    2021.08.19 12:22:48.260957 [ 25174 ] {} Application: DB::Exception: Requested cluster '' not found: while loading database default from path /var/lib/clickhouse/metadata/default
@zhangjmruc zhangjmruc added the potential bug To be reviewed by developers and confirmed/rejected. label Aug 19, 2021
@tavplubix tavplubix self-assigned this Aug 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
potential bug To be reviewed by developers and confirmed/rejected.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants