Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

ASAN: heap-use-after-free in simple_lb_cure_test.cpp #364

Closed
foreverneverer opened this issue Dec 20, 2019 · 2 comments
Closed

ASAN: heap-use-after-free in simple_lb_cure_test.cpp #364

foreverneverer opened this issue Dec 20, 2019 · 2 comments

Comments

@foreverneverer
Copy link
Contributor 

==14511==ERROR: AddressSanitizer: heap-use-after-free on address 0x603006448220 at pc 0x000000649971 bp 0x7f962b5802a0 sp 0x7f962b580290
READ of size 8 at 0x603006448220 thread T31 (test_meta.THREA)
    #0 0x649970 in operator() /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:435
    #1 0x649970 in _M_invoke /usr/include/c++/5/functional:1857
    #2 0x66f112 in std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)>::operator()(dsn::rpc_address const&, dsn::message_ex*) const /usr/include/c++/5/functional:2267
    #3 0x66f112 in message_filter::send_message(dsn::rpc_address const&, dsn::message_ex*) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:75
    #4 0x84645f in dsn::replication::server_state::send_proposal(dsn::rpc_address, dsn::replication::configuration_update_request const&) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/meta_server/server_state.cpp:1307
    #5 0x846d76 in dsn::replication::server_state::send_proposal(dsn::replication::configuration_proposal_action const&, dsn::partition_configuration const&, dsn::replication::app_state const&) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/meta_server/server_state.cpp:1319
    #6 0x84857b in dsn::replication::server_state::check_all_partitions() /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/meta_server/server_state.cpp:2437
    #7 0xac12af in dsn::task::exec_internal() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task.cpp:180
    #8 0xaf4927 in dsn::task_worker::loop() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task_worker.cpp:211
    #9 0xaf50ab in dsn::task_worker::run_internal() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task_worker.cpp:191
    #10 0x7f963e7f4c7f  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb8c7f)
    #11 0x7f963eac56b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
    #12 0x7f963df5a41c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x10741c)

0x603006448220 is located 16 bytes inside of 24-byte region [0x603006448210,0x603006448228)
freed by thread T31 (test_meta.THREA) here:
    #0 0x7f96403aa132 in operator delete(void*, unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x9a132)
    #1 0x641177 in _M_destroy /usr/include/c++/5/functional:1726
    #2 0x641177 in _M_manager /usr/include/c++/5/functional:1750
    #3 0x66e53b in std::_Function_base::~_Function_base() /usr/include/c++/5/functional:1830
    #4 0x66e53b in std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)>::~function() /usr/include/c++/5/functional:1974
    #5 0x66e53b in std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)>::operator=(std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)> const&) /usr/include/c++/5/functional:2071
    #6 0x649168 in message_filter::set_filter(std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)> const&) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:70
    #7 0x649168 in operator() /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:434
    #8 0x649168 in _M_invoke /usr/include/c++/5/functional:1857
    #9 0x66f112 in std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)>::operator()(dsn::rpc_address const&, dsn::message_ex*) const /usr/include/c++/5/functional:2267
    #10 0x66f112 in message_filter::send_message(dsn::rpc_address const&, dsn::message_ex*) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:75
    #11 0x84645f in dsn::replication::server_state::send_proposal(dsn::rpc_address, dsn::replication::configuration_update_request const&) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/meta_server/server_state.cpp:1307
    #12 0x846d76 in dsn::replication::server_state::send_proposal(dsn::replication::configuration_proposal_action const&, dsn::partition_configuration const&, dsn::replication::app_state const&) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/meta_server/server_state.cpp:1319
    #13 0x84857b in dsn::replication::server_state::check_all_partitions() /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/meta_server/server_state.cpp:2437
    #14 0xac12af in dsn::task::exec_internal() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task.cpp:180
    #15 0xaf4927 in dsn::task_worker::loop() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task_worker.cpp:211
    #16 0xaf50ab in dsn::task_worker::run_internal() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task_worker.cpp:191
    #17 0x7f963e7f4c7f  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb8c7f)

previously allocated by thread T23 (test_meta.THREA) here:
    #0 0x7f96403a9532 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x99532)
    #1 0x6411a0 in _M_clone /usr/include/c++/5/functional:1710
    #2 0x6411a0 in _M_manager /usr/include/c++/5/functional:1746
    #3 0x66e464 in std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)>::function(std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)> const&) /usr/include/c++/5/functional:2238
    #4 0x66e464 in std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)>::operator=(std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)> const&) /usr/include/c++/5/functional:2071
    #5 0x66766b in message_filter::set_filter(std::function<std::shared_ptr<dsn::replication::configuration_update_request> (dsn::rpc_address const&, dsn::message_ex*)> const&) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:70
    #6 0x66766b in meta_service_test_app::simple_lb_cure_test() /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:437
    #7 0xcc6c62 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xcc6c62)
    #8 0xcc0682 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xcc0682)
    #9 0xca407d in testing::Test::Run() (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xca407d)
    #10 0xca4a15 in testing::TestInfo::Run() (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xca4a15)
    #11 0xca5108 in testing::TestCase::Run() (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xca5108)
    #12 0xcac24f in testing::internal::UnitTestImpl::RunAllTests() (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xcac24f)
    #13 0xcc822c in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xcc822c)
    #14 0xcc1456 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xcc1456)
    #15 0xcaaceb in testing::UnitTest::Run() (/home/mi/work/PegasusDB/pegasus/rdsn/builder/src/dist/replication/test/meta_test/unit_test/dsn.meta.test+0xcaaceb)
    #16 0x5c4135 in RUN_ALL_TESTS() /home/mi/work/PegasusDB/pegasus/rdsn/thirdparty/output/include/gtest/gtest.h:2233
    #17 0x5c4135 in meta_service_test_app::start(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/main.cpp:83
    #18 0xaa2090 in dsn::service_node::start_app() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/service_engine.cpp:99
    #19 0xafd015 in dsn::service_control_task::exec() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/tool_api.cpp:60
    #20 0xac12af in dsn::task::exec_internal() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task.cpp:180
    #21 0xaf4927 in dsn::task_worker::loop() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task_worker.cpp:211
    #22 0xaf50ab in dsn::task_worker::run_internal() /home/mi/work/PegasusDB/pegasus/rdsn/src/core/core/task_worker.cpp:191
    #23 0x7f963e7f4c7f  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb8c7f)

Thread T31 (test_meta.THREA) created by T0 here:
    #0 0x7f9640346253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f963e7f4dc2 in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>, void (*)()) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb8dc2)

Thread T23 (test_meta.THREA) created by T0 here:
    #0 0x7f9640346253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f963e7f4dc2 in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>, void (*)()) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb8dc2)

SUMMARY: AddressSanitizer: heap-use-after-free /home/mi/work/PegasusDB/pegasus/rdsn/src/dist/replication/test/meta_test/unit_test/simple_lb_cure_test.cpp:435 operator()
Shadow bytes around the buggy address:
  0x0c0680c80ff0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0680c81000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0680c81010: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0680c81020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0680c81030: fa fa fa fa fa fa fa fa fa fa fa fa 00 00 00 fa
=>0x0c0680c81040: fa fa fd fd[fd]fa fa fa fd fd fd fa fa fa fd fd
  0x0c0680c81050: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
  0x0c0680c81060: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fa
  0x0c0680c81070: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
  0x0c0680c81080: fd fa fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c0680c81090: fd fd fd fa fa fa fd fd fd fd fa fa fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==14511==ABORTING
tail: 无法打开'data/log/log.1.txt' 读取数据: 没有那个文件或目录
ERROR: run dsn.meta.test failed, return_code = 1
@weekly-digest weekly-digest bot mentioned this issue Dec 22, 2019
Closed
@neverchanje
Copy link
Contributor 

[ RUN      ] meta_load_balance_test.simple_lb_cure_test
Case: upgrade secondary to primary, and message lost
Case: upgrade secondary to primary, and the candidate died
Case: add secondary, and the message lost
Case: add secondary, but the primary is removing another
Case: add secondary, and the added secondary is dead
Case: add secondary, and the primary is dead
Case: recover from DDD state, nodes[1] isn't alive
Case: recover from DDD state, nodes[2] is not in dropped
Case: recover from DDD state, haven't collect nodes[2]'s info from replica, and nodes[2]'s info haven't updated
Case: recover from DDD state, haven't collect nodes[2]'s info from replica, and nodes[2]'s info have updated
Case: recover from DDD, haven't collect nodes[1/2]'s info from replica, and nodes[1/2]'s info both have updated
Case: recover from DDD state, larger ballot not match with larger decree
Case: recover from DDD state, committed decree less than meta's
Case: recover from DDD state, select primary from config_context::dropped
Case: recover from DDD state, only one primary
F2019-12-26 10:50:41.827 (1577357441827834734 41d5) test_meta.THREAD_POOL_META_STATE0.01030000000002af: rpc_engine.cpp:657:call_ip(): assertion expression: addr.port() > MAX_CLIENT_PORT
F2019-12-26 10:50:41.827 (1577357441827980865 41d5) test_meta.THREAD_POOL_META_STATE0.01030000000002af: rpc_engine.cpp:657:call_ip(): only server address can be called

@foreverneverer foreverneverer mentioned this issue Jan 8, 2020
Merged
@foreverneverer
Copy link
Contributor Author

#375 solved

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@neverchanje @foreverneverer