Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: ASAN use-after-poison in TimeZoneInfo #1677

Open
julianoes opened this issue May 22, 2024 · 9 comments
Open

[Bug]: ASAN use-after-poison in TimeZoneInfo #1677

julianoes opened this issue May 22, 2024 · 9 comments

Comments

@julianoes
Copy link

Describe the issue

While running our gRPC server with ASAN, we often see a use-after-poison error occur. Valgrind does not show a problem though.

I0521 09:09:13.214993342 1918165 config.cc:239]                        gRPC experiments: work_serializer_dispatch:off; default-enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pending_queue_cap, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0521 09:09:13.215412561 1918165 ev_epoll1_linux.cc:124]               grpc epoll fd: 4
D0521 09:09:13.215436331 1918165 ev_posix.cc:112]                      Using polling engine: epoll1
I0521 09:09:13.215873570 1918165 server_builder.cc:354]                Synchronous server. Num CQs: 1, Min pollers: 1, Max Pollers: 2, CQ timeout (msec): 10000
D0521 09:09:13.216397119 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "priority_experimental"
D0521 09:09:13.216448779 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "outlier_detection_experimental"
D0521 09:09:13.216461758 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "weighted_target_experimental"
D0521 09:09:13.216475978 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "pick_first"
D0521 09:09:13.216484668 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "round_robin"
D0521 09:09:13.216493578 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "weighted_round_robin"
D0521 09:09:13.216714098 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "grpclb"
D0521 09:09:13.216833398 1918165 dns_resolver_plugin.cc:42]            Using EventEngine dns resolver
D0521 09:09:13.216948097 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "rls_experimental"
D0521 09:09:13.217175977 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "xds_cluster_manager_experimental"
D0521 09:09:13.217195397 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "xds_cluster_impl_experimental"
D0521 09:09:13.217204257 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "cds_experimental"
D0521 09:09:13.217215457 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "xds_override_host_experimental"
D0521 09:09:13.217226597 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "xds_wrr_locality_experimental"
D0521 09:09:13.217236227 1918165 lb_policy_registry.cc:46]             registering LB policy factory for "ring_hash_experimental"
D0521 09:09:13.217268777 1918165 certificate_provider_registry.cc:34]  registering certificate provider factory for "file_watcher"
I0521 09:09:13.219810742 1918165 ev_epoll1_linux.cc:359]               grpc epoll fd: 6
I0521 09:09:13.222750916 1918165 tcp_socket_utils.cc:636]              TCP_USER_TIMEOUT is available. TCP_USER_TIMEOUT will be used thereafter
[09:09:13|Debug] MAVLink: info: GCS connection regained	 (system_impl.cpp:242)
[09:09:13|Info ] Server started (grpc_server.cpp:169)
[09:09:13|Info ] Server set to listen on 0.0.0.0:50051 (grpc_server.cpp:170)
=================================================================
==1918165==ERROR: AddressSanitizer: use-after-poison on address 0x7fe9fc7f5d18 at pc 0x7fea08de364a bp 0x7fe9fc7f5cd0 sp 0x7fe9fc7f5cc0
READ of size 8 at 0x7fe9fc7f5d18 thread T9
    #0 0x7fea08de3649 in unsigned long const& std::min<unsigned long>(unsigned long const&, unsigned long const&) /usr/include/c++/11/bits/stl_algobase.h:235
    #1 0x7fea0ac31ebc in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocator<absl::lts_20240116::time_internal::cctz::TransitionType> >::_S_max_size(std::allocator<absl::lts_20240116::time_internal::cctz::TransitionType> const&) /usr/include/c++/11/bits/stl_vector.h:1784
    #2 0x7fea0ac30567 in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocator<absl::lts_20240116::time_internal::cctz::TransitionType> >::max_size() const /usr/include/c++/11/bits/stl_vector.h:924
    #3 0x7fea0ac317b5 in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocator<absl::lts_20240116::time_internal::cctz::TransitionType> >::_M_check_len(unsigned long, char const*) const /usr/include/c++/11/bits/stl_vector.h:1758
    #4 0x7fea0ac2fdbf in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocator<absl::lts_20240116::time_internal::cctz::TransitionType> >::_M_default_append(unsigned long) /usr/include/c++/11/bits/vector.tcc:634
    #5 0x7fea0ac2ea28 in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocator<absl::lts_20240116::time_internal::cctz::TransitionType> >::resize(unsigned long) /usr/include/c++/11/bits/stl_vector.h:940
    #6 0x7fea0ac2a74c in absl::lts_20240116::time_internal::cctz::TimeZoneInfo::ResetToBuiltinUTC(std::chrono::duration<long, std::ratio<1l, 1l> > const&) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/internal/cctz/src/time_zone_info.cc:589
    #7 0x7fea0ac2bdf6 in absl::lts_20240116::time_internal::cctz::TimeZoneInfo::UTC() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/internal/cctz/src/time_zone_info.cc:831
    #8 0x7fea0ac276a6 in absl::lts_20240116::time_internal::cctz::TimeZoneIf::UTC() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/internal/cctz/src/time_zone_if.cc:26
    #9 0x7fea0ac24338 in absl::lts_20240116::time_internal::cctz::time_zone::Impl::Impl() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/internal/cctz/src/time_zone_impl.cc:102
    #10 0x7fea0ac2446d in absl::lts_20240116::time_internal::cctz::time_zone::Impl::UTCImpl() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/internal/cctz/src/time_zone_impl.cc:108
    #11 0x7fea0ac23dd3 in absl::lts_20240116::time_internal::cctz::time_zone::Impl::UTC() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/internal/cctz/src/time_zone_impl.cc:49
    #12 0x7fea0ac22b01 in absl::lts_20240116::time_internal::cctz::utc_time_zone() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/internal/cctz/src/time_zone_lookup.cc:213
    #13 0x7fea0a08c871 in absl::lts_20240116::UTCTimeZone() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/absl/absl/src/absl/absl/time/time.h:1252
    #14 0x7fea0a089ff9 in grpc_core::StatusSetTime(absl::lts_20240116::Status*, grpc_core::StatusTimeProperty, absl::lts_20240116::Time) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/gprpp/status_helper.cc:230
    #15 0x7fea0a0899d8 in grpc_core::StatusCreate(absl::lts_20240116::StatusCode, std::basic_string_view<char, std::char_traits<char> >, grpc_core::DebugLocation const&, std::vector<absl::lts_20240116::Status, std::allocator<absl::lts_20240116::Status> >) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/gprpp/status_helper.cc:178
    #16 0x7fea09ffb13d in grpc_chttp2_rst_stream_parser_parse(void*, grpc_chttp2_transport*, grpc_chttp2_stream*, grpc_slice const&, int) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/ext/transport/chttp2/transport/frame_rst_stream.cc:123
    #17 0x7fea0a02a74d in parse_frame_slice /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/ext/transport/chttp2/transport/parsing.cc:898
    #18 0x7fea0a0275d2 in grpc_chttp2_perform_read(grpc_chttp2_transport*, grpc_slice const&, unsigned long&) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/ext/transport/chttp2/transport/parsing.cc:385
    #19 0x7fea09fbb631 in read_action_parse_loop_locked /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:2651
    #20 0x7fea09fbc377 in read_action_locked /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:2749
    #21 0x7fea09fc17cd in operator() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:272
    #22 0x7fea09fc1854 in _FUN /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:272
    #23 0x7fea0a094477 in grpc_combiner_continue_exec_ctx() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/iomgr/combiner.cc:232
    #24 0x7fea0a09c910 in grpc_core::ExecCtx::Flush() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/iomgr/exec_ctx.cc:87
    #25 0x7fea0a0940e5 in operator() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/iomgr/combiner.cc:179
    #26 0x7fea0a094eaa in __invoke_impl<void, queue_offload(grpc_core::Combiner*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:61
    #27 0x7fea0a094e87 in __invoke<queue_offload(grpc_core::Combiner*)::<lambda()>&> /usr/include/c++/11/bits/invoke.h:96
    #28 0x7fea0a094e64 in invoke<queue_offload(grpc_core::Combiner*)::<lambda()>&> /usr/include/c++/11/functional:97
    #29 0x7fea0a094e25 in InvokeR<void, queue_offload(grpc_core::Combiner*)::<lambda()>&> /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/install/include/absl/functional/internal/any_invocable.h:132
    #30 0x7fea0a094dc6 in LocalInvoker<false, void, queue_offload(grpc_core::Combiner*)::<lambda()>&> /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/install/include/absl/functional/internal/any_invocable.h:310
    #31 0x7fea0a02deff in absl::lts_20240116::internal_any_invocable::Impl<void ()>::operator()() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/install/include/absl/functional/internal/any_invocable.h:868
    #32 0x7fea0a4e146f in grpc_event_engine::experimental::SelfDeletingClosure::Run() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/common_closures.h:55
    #33 0x7fea0a4e004b in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::Step() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:532
    #34 0x7fea0a4dfc53 in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::ThreadBody() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:495
    #35 0x7fea0a4de6f6 in operator() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:260
    #36 0x7fea0a4de73c in _FUN /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:262
    #37 0x7fea0a8bc695 in operator() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/gprpp/posix/thd.cc:148
    #38 0x7fea0a8bc6e2 in _FUN /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/gprpp/posix/thd.cc:153
    #39 0x7fea046cbac2 in start_thread nptl/pthread_create.c:442
    #40 0x7fea0475d84f  (/lib/x86_64-linux-gnu/libc.so.6+0x12684f)

Address 0x7fe9fc7f5d18 is located in stack of thread T9
SUMMARY: AddressSanitizer: use-after-poison /usr/include/c++/11/bits/stl_algobase.h:235 in unsigned long const& std::min<unsigned long>(unsigned long const&, unsigned long const&)
Shadow bytes around the buggy address:
  0x0ffdbf8f6b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6b90: 00 00 00 00 f7 f7 00 00 00 00 00 00 00 00 00 00
=>0x0ffdbf8f6ba0: 00 00 f7[f7]00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6bd0: 00 00 00 00 00 00 00 00 00 00 00 00 f7 f7 00 00
  0x0ffdbf8f6be0: f7 f7 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ffdbf8f6bf0: f7 f7 00 00 f7 f7 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
Thread T9 created by T0 here:
    #0 0x7fea0bea5685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
    #1 0x7fea0a8bcbfd in ThreadInternalsPosix /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/gprpp/posix/thd.cc:116
    #2 0x7fea0a8bd1d2 in grpc_core::Thread::Thread(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/gprpp/posix/thd.cc:224
    #3 0x7fea0a4de864 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:256
    #4 0x7fea0a4de4cf in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::Start() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:234
    #5 0x7fea0a4ddd8e in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPool(unsigned long) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:181
    #6 0x7fea0a4dd932 in void __gnu_cxx::new_allocator<grpc_event_engine::experimental::WorkStealingThreadPool>::construct<grpc_event_engine::experimental::WorkStealingThreadPool, unsigned long&>(grpc_event_engine::experimental::WorkStealingThreadPool*, unsigned long&) /usr/include/c++/11/ext/new_allocator.h:162
    #7 0x7fea0a4dd7c3 in void std::allocator_traits<std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool> >::construct<grpc_event_engine::experimental::WorkStealingThreadPool, unsigned long&>(std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool>&, grpc_event_engine::experimental::WorkStealingThreadPool*, unsigned long&) /usr/include/c++/11/bits/alloc_traits.h:516
    #8 0x7fea0a4dd5f1 in std::_Sp_counted_ptr_inplace<grpc_event_engine::experimental::WorkStealingThreadPool, std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<unsigned long&>(std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool>, unsigned long&) /usr/include/c++/11/bits/shared_ptr_base.h:519
    #9 0x7fea0a4dd319 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<grpc_event_engine::experimental::WorkStealingThreadPool, std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool>, unsigned long&>(grpc_event_engine::experimental::WorkStealingThreadPool*&, std::_Sp_alloc_shared_tag<std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool> >, unsigned long&) /usr/include/c++/11/bits/shared_ptr_base.h:650
    #10 0x7fea0a4dd24b in std::__shared_ptr<grpc_event_engine::experimental::WorkStealingThreadPool, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool>, unsigned long&>(std::_Sp_alloc_shared_tag<std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool> >, unsigned long&) /usr/include/c++/11/bits/shared_ptr_base.h:1342
    #11 0x7fea0a4dd1f6 in std::shared_ptr<grpc_event_engine::experimental::WorkStealingThreadPool>::shared_ptr<std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool>, unsigned long&>(std::_Sp_alloc_shared_tag<std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool> >, unsigned long&) /usr/include/c++/11/bits/shared_ptr.h:409
    #12 0x7fea0a4dd0e6 in std::shared_ptr<grpc_event_engine::experimental::WorkStealingThreadPool> std::allocate_shared<grpc_event_engine::experimental::WorkStealingThreadPool, std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool>, unsigned long&>(std::allocator<grpc_event_engine::experimental::WorkStealingThreadPool> const&, unsigned long&) /usr/include/c++/11/bits/shared_ptr.h:863
    #13 0x7fea0a4dcfbf in std::shared_ptr<grpc_event_engine::experimental::WorkStealingThreadPool> std::make_shared<grpc_event_engine::experimental::WorkStealingThreadPool, unsigned long&>(unsigned long&) /usr/include/c++/11/bits/shared_ptr.h:879
    #14 0x7fea0a4dcddd in grpc_event_engine::experimental::MakeThreadPool(unsigned long) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/thread_pool/thread_pool_factory.cc:40
    #15 0x7fea0a4aefee in grpc_event_engine::experimental::PosixEventEngine::PosixEventEngine() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/posix_engine/posix_engine.cc:388
    #16 0x7fea0a4abf6e in std::_MakeUniq<grpc_event_engine::experimental::PosixEventEngine>::__single_object std::make_unique<grpc_event_engine::experimental::PosixEventEngine>() /usr/include/c++/11/bits/unique_ptr.h:962
    #17 0x7fea0a4abe74 in grpc_event_engine::experimental::DefaultEventEngineFactory() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/default_event_engine_factory.cc:53
    #18 0x7fea0a4a9e04 in grpc_event_engine::experimental::CreateEventEngineInner() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/default_event_engine.cc:67
    #19 0x7fea0a4a9e49 in grpc_event_engine::experimental::CreateEventEngine() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/default_event_engine.cc:74
    #20 0x7fea0a4a9fbc in grpc_event_engine::experimental::GetDefaultEventEngine(grpc_core::SourceLocation) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/default_event_engine.cc:88
    #21 0x7fea0a4aa1a0 in EnsureEventEngineInChannelArgs /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/event_engine/default_event_engine.cc:99
    #22 0x7fea0a4ab656 in grpc_core::ChannelArgs std::__invoke_impl<grpc_core::ChannelArgs, grpc_core::ChannelArgs (*&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs>(std::__invoke_other, grpc_core::ChannelArgs (*&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs&&) /usr/include/c++/11/bits/invoke.h:61
    #23 0x7fea0a4ab2e2 in std::enable_if<is_invocable_r_v<grpc_core::ChannelArgs, grpc_core::ChannelArgs (*&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs>, grpc_core::ChannelArgs>::type std::__invoke_r<grpc_core::ChannelArgs, grpc_core::ChannelArgs (*&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs>(grpc_core::ChannelArgs (*&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs&&) /usr/include/c++/11/bits/invoke.h:116
    #24 0x7fea0a4aadd3 in std::_Function_handler<grpc_core::ChannelArgs (grpc_core::ChannelArgs), grpc_core::ChannelArgs (*)(grpc_core::ChannelArgs)>::_M_invoke(std::_Any_data const&, grpc_core::ChannelArgs&&) /usr/include/c++/11/bits/std_function.h:291
    #25 0x7fea0a071c6b in std::function<grpc_core::ChannelArgs (grpc_core::ChannelArgs)>::operator()(grpc_core::ChannelArgs) const (/home/julianoes/src/upstream/MAVSDK/build-asan/src/mavsdk_server/src/libmavsdk_server.so.2+0x3e81c6b)
    #26 0x7fea0a071765 in grpc_core::ChannelArgsPreconditioning::PreconditionChannelArgs(grpc_channel_args const*) const /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/lib/channel/channel_args_preconditioning.cc:37
    #27 0x7fea0a246c9f in grpc_server_create /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/core/server/server.cc:1908
    #28 0x7fea09f4ccfb in grpc::Server::Server(grpc::ChannelArguments*, std::shared_ptr<std::vector<std::unique_ptr<grpc::ServerCompletionQueue, std::default_delete<grpc::ServerCompletionQueue> >, std::allocator<std::unique_ptr<grpc::ServerCompletionQueue, std::default_delete<grpc::ServerCompletionQueue> > > > >, int, int, int, std::vector<std::shared_ptr<grpc::internal::ExternalConnectionAcceptorImpl>, std::allocator<std::shared_ptr<grpc::internal::ExternalConnectionAcceptorImpl> > >, grpc_server_config_fetcher*, grpc_resource_quota*, std::vector<std::unique_ptr<grpc::experimental::ServerInterceptorFactoryInterface, std::default_delete<grpc::experimental::ServerInterceptorFactoryInterface> >, std::allocator<std::unique_ptr<grpc::experimental::ServerInterceptorFactoryInterface, std::default_delete<grpc::experimental::ServerInterceptorFactoryInterface> > > >, grpc::experimental::ServerMetricRecorder*) /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/cpp/server/server_cc.cc:964
    #29 0x7fea09f3e718 in grpc::ServerBuilder::BuildAndStart() /home/julianoes/src/upstream/MAVSDK/build-asan/third_party/grpc/grpc/src/grpc/src/cpp/server/server_builder.cc:370
    #30 0x7fea08fd4392 in mavsdk::mavsdk_server::GrpcServer::run() /home/julianoes/src/upstream/MAVSDK/src/mavsdk_server/src/grpc_server.cpp:166
    #31 0x7fea08e2140d in MavsdkServer::Impl::startGrpcServer(int) (/home/julianoes/src/upstream/MAVSDK/build-asan/src/mavsdk_server/src/libmavsdk_server.so.2+0x2c3140d)
    #32 0x7fea08de0d72 in MavsdkServer::startGrpcServer(int) /home/julianoes/src/upstream/MAVSDK/src/mavsdk_server/src/mavsdk_server.cpp:61
    #33 0x7fea08de0379 in mavsdk_server_run /home/julianoes/src/upstream/MAVSDK/src/mavsdk_server/src/mavsdk_server_api.cpp:18
    #34 0x7fea08de045b in mavsdk_server_run_with_mavlink_ids /home/julianoes/src/upstream/MAVSDK/src/mavsdk_server/src/mavsdk_server_api.cpp:35
    #35 0x5588afb3d19d in main /home/julianoes/src/upstream/MAVSDK/src/mavsdk_server/src/mavsdk_server_bin.cpp:91
    #36 0x7fea04660d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

==1918165==ABORTING

Steps to reproduce the problem

I could not extract a minimal example to reproduce this yet. I'm hoping the backtrace is enough of a hint for you. If not, I will try to create a setup to reproduce.

This happens when used within a gRPC server.

I would be interested how to further debug this to get to the bottom of it.

What version of Abseil are you using?

20240116.2

What operating system and version are you using?

Linux Mint based on Ubuntu 22.04

What compiler and version are you using?

Default GCC 11.4.0

What build system are you using?

cmake version 3.22.1

Additional context

This happens for me in NZ time zone, so +12h in case that's relevant.

First raised in grpc/grpc#36668.
@derekmauro
Copy link
Member

We need a full reproduction before we can look into this. There are too many possible causes.

@julianoes
Copy link
Author

I appreciate the answer but that's very difficult because I'm much higher up the stack just using gRPC. I don't even know how to start to get to a small reproducible example.

It seems to happen here:

abseil-cpp/absl/time/internal/cctz/src/time_zone_info.cc

Line 589 in 1a31b81

transition_types_.resize(1);

@derekmauro
Copy link
Member

I can see where it happens from the stack trace, but as I said, there are too many possible causes (Abseil bug, gRPC bug, ASAN bug, ODR violation, etc). If I had to guess, I'd say ODR violation is most likely. We don't see this issue, and just because our code is at the top of the stack trace doesn't mean it is an error in our code.

@zlojvavan
Copy link

just updated grpc and got similar problem on app finalization:

==574266==ERROR: AddressSanitizer: use-after-poison on address 0x758548d6b498 at pc 0x5abad9b58a0e bp 0x758548d6b450 sp 0x758548d6b448 READ of size 8 at 0x758548d6b498 thread T56 #0 0x5abad9b58a0d in std::chrono::duration<long, std::ratio<1l, 1l>>::count() const /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/chrono.h:588:11 #1 0x5abadac314de in bool std::chrono::operator==<long, std::ratio<1l, 1l>, long, std::ratio<1l, 1l>>(std::chrono::duration<long, std::ratio<1l, 1l>> const&, std::chrono::duration<long, std::ratio<1l, 1l>> const&) /usr/include/c++/13/bits/chrono.h:814:26 #2 0x5abadbe3ae3d in absl::lts_20240116::time_internal::cctz::FixedOffsetToName[abi:cxx11](std::chrono::duration<long, std::ratio<1l, 1l>> const&) /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_fixed.cc:85:14 #3 0x5abadbe3b1f6 in absl::lts_20240116::time_internal::cctz::FixedOffsetToAbbr[abi:cxx11](std::chrono::duration<long, std::ratio<1l, 1l>> const&) /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_fixed.cc:121:46 #4 0x5abadbe3ea27 in absl::lts_20240116::time_internal::cctz::TimeZoneInfo::ResetToBuiltinUTC(std::chrono::duration<long, std::ratio<1l, 1l>> const&) /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_info.cc:622:37 #5 0x5abadbe3fe94 in absl::lts_20240116::time_internal::cctz::TimeZoneInfo::UTC() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_info.cc:831:24 #6 0x5abadbe3b728 in absl::lts_20240116::time_internal::cctz::TimeZoneIf::UTC() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_if.cc:26:73 #7 0x5abadbe37690 in absl::lts_20240116::time_internal::cctz::time_zone::Impl::Impl() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_impl.cc:102:41 #8 0x5abadbe37809 in absl::lts_20240116::time_internal::cctz::time_zone::Impl::UTCImpl() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_impl.cc:108:37 #9 0x5abadbe370bd in absl::lts_20240116::time_internal::cctz::time_zone::Impl::UTC() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_impl.cc:49:43 #10 0x5abadbe36a67 in absl::lts_20240116::time_internal::cctz::utc_time_zone() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_lookup.cc:213:30 #11 0x5abadb563b6f in absl::lts_20240116::UTCTimeZone() /opt/vcpkg/installed/x64-linux/include/absl/time/time.h:1252:10 #12 0x5abadb794aef in grpc_core::StatusSetTime(absl::lts_20240116::Status*, grpc_core::StatusTimeProperty, absl::lts_20240116::Time) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/status_helper.cc:231:69 #13 0x5abadb79446a in grpc_core::StatusCreate(absl::lts_20240116::StatusCode, std::basic_string_view<char, std::char_traits<char>>, grpc_core::DebugLocation const&, std::vector<absl::lts_20240116::Status, std::allocator<absl::lts_20240116::Status>>) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/status_helper.cc:179:16 #14 0x5abadb94d2b1 in grpc_core::Server::ShutdownAndNotify(grpc_completion_queue*, void*) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/surface/server.cc:1086:29 #15 0x5abadb9527a2 in grpc_server_shutdown_and_notify /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/surface/server.cc:1952:54 #16 0x5abadb28de86 in grpc::Server::ShutdownInternal(gpr_timespec) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/cpp/server/server_cc.cc:1257:34 #17 0x5abada8fa5d5 in grpc::ServerInterface::Shutdown() /opt/vcpkg/installed/x64-linux/include/grpcpp/server_interface.h:106:21

@zlojvavan
Copy link

zlojvavan commented Jul 2, 2024 •
edited
Loading

ouch it is much worse than that. not only it happens on finalization but it also crashes during the run

`

==597460==ERROR: AddressSanitizer: use-after-poison on address 0x7560a2b9ece0 at pc 0x589fcfe8f9e7 bp 0x7560a2b9ec70 sp 0x7560a2b9ec68
READ of size 8 at 0x7560a2b9ece0 thread T34
#0 0x589fcfe8f9e6 in unsigned long const& std::max(unsigned long const&, unsigned long const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/stl_algobase.h:262:17
#1 0x589fd2291aa4 in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocatorabsl::lts_20240116::time_internal::cctz::TransitionType>::_M_check_len(unsigned long, char const*) const /usr/include/c++/13/bits/stl_vector.h:1898:45
#2 0x589fd228ffa2 in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocatorabsl::lts_20240116::time_internal::cctz::TransitionType>::_M_default_append(unsigned long) /usr/include/c++/13/bits/vector.tcc:662:15
#3 0x589fd228ea88 in std::vector<absl::lts_20240116::time_internal::cctz::TransitionType, std::allocatorabsl::lts_20240116::time_internal::cctz::TransitionType>::resize(unsigned long) /usr/include/c++/13/bits/stl_vector.h:1013:21
#4 0x589fd228a7de in absl::lts_20240116::time_internal::cctz::TimeZoneInfo::ResetToBuiltinUTC(std::chrono::duration<long, std::ratio<1l, 1l>> const&) /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_info.cc:589:27
#5 0x589fd228beda in absl::lts_20240116::time_internal::cctz::TimeZoneInfo::UTC() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_info.cc:831:24
#6 0x589fd228776e in absl::lts_20240116::time_internal::cctz::TimeZoneIf::UTC() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_if.cc:26:73
#7 0x589fd22836d6 in absl::lts_20240116::time_internal::cctz::time_zone::Impl::Impl() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_impl.cc:102:41
#8 0x589fd228384f in absl::lts_20240116::time_internal::cctz::time_zone::Impl::UTCImpl() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_impl.cc:108:37
#9 0x589fd2283103 in absl::lts_20240116::time_internal::cctz::time_zone::Impl::UTC() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_impl.cc:49:43
#10 0x589fd2282aad in absl::lts_20240116::time_internal::cctz::utc_time_zone() /opt/vcpkg/buildtrees/abseil/src/20240116.2-ac9ed9836d.clean/absl/time/internal/cctz/src/time_zone_lookup.cc:213:30
#11 0x589fd19afbb5 in absl::lts_20240116::UTCTimeZone() /opt/vcpkg/installed/x64-linux/include/absl/time/time.h:1252:10
#12 0x589fd1be0b35 in grpc_core::StatusSetTime(absl::lts_20240116::Status*, grpc_core::StatusTimeProperty, absl::lts_20240116::Time) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/status_helper.cc:231:69
#13 0x589fd1be04b0 in grpc_core::StatusCreate(absl::lts_20240116::StatusCode, std::basic_string_view<char, std::char_traits>, grpc_core::DebugLocation const&, std::vector<absl::lts_20240116::Status, std::allocatorabsl::lts_20240116::Status>) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/status_helper.cc:179:16
#14 0x589fd1bf647d in grpc_status_create(absl::lts_20240116::StatusCode, std::basic_string_view<char, std::char_traits>, grpc_core::DebugLocation const&, unsigned long, absl::lts_20240116::Status*) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/iomgr/error.cc:50:56
#15 0x589fd1aacaa5 in read_action_locked(grpc_core::RefCountedPtr<grpc_chttp2_transport>, absl::lts_20240116::Status) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:2870:9
#16 0x589fd1ab2249 in grpc_closure* grpc_core::(anonymous namespace)::InitTransportClosure<&read_action_locked(grpc_core::RefCountedPtr<grpc_chttp2_transport>, absl::lts_20240116::Status)>(grpc_core::RefCountedPtr<grpc_chttp2_transport>, grpc_closure*)::'lambda'(void*, absl::lts_20240116::Status)::operator()(void*, absl::lts_20240116::Status) const /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:276:3
#17 0x589fd1ab22e4 in grpc_closure* grpc_core::(anonymous namespace)::InitTransportClosure<&read_action_locked(grpc_core::RefCountedPtr<grpc_chttp2_transport>, absl::lts_20240116::Status)>(grpc_core::RefCountedPtr<grpc_chttp2_transport>, grpc_closure*)::'lambda'(void*, absl::lts_20240116::Status)::_FUN(void*, absl::lts_20240116::Status) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:276:3
#18 0x589fd1bf533e in grpc_combiner_continue_exec_ctx() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/iomgr/combiner.cc:267:11
#19 0x589fd1c01cf6 in grpc_core::ExecCtx::Flush() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/iomgr/exec_ctx.cc:87:48
#20 0x589fd16b7634 in grpc_core::ExecCtx::~ExecCtx() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/iomgr/exec_ctx.h:130:10
#21 0x589fd1c155cf in grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::FinishPendingRead(absl::lts_20240116::Status) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/iomgr/event_engine_shims/endpoint.cc:134:5
#22 0x589fd1c15114 in grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)::operator()(absl::lts_20240116::Status) const /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/iomgr/event_engine_shims/endpoint.cc:105:56
#23 0x589fd1c17e28 in void std::__invoke_impl<void, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(std::__invoke_other, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /usr/include/c++/13/bits/invoke.h:61:36
#24 0x589fd1c17d49 in std::__invoke_result<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>::type std::__invoke<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /usr/include/c++/13/bits/invoke.h:96:40
#25 0x589fd1c17c57 in std::invoke_result<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>::type std::invoke<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /usr/include/c++/13/functional:113:27
#26 0x589fd1c17b32 in void absl::lts_20240116::internal_any_invocable::InvokeR<void, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status, void>(grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /opt/vcpkg/installed/x64-linux/include/absl/functional/internal/any_invocable.h:132:30
#27 0x589fd1c179c5 in void absl::lts_20240116::internal_any_invocable::LocalInvoker<false, void, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(absl::lts_20240116::internal_any_invocable::TypeErasedState*, absl::lts_20240116::internal_any_invocable::ForwardedParameterabsl::lts_20240116::Status::type) /opt/vcpkg/installed/x64-linux/include/absl/functional/internal/any_invocable.h:310:31
#28 0x589fd1b6d2aa in absl::lts_20240116::internal_any_invocable::Impl<void (absl::lts_20240116::Status)>::operator()(absl::lts_20240116::Status) /opt/vcpkg/installed/x64-linux/include/absl/functional/internal/any_invocable.h:868:1
#29 0x589fd1b81dca in grpc_event_engine::experimental::PosixEndpointImpl::HandleRead(absl::lts_20240116::Status) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/posix_engine/posix_endpoint.cc:588:5
#30 0x589fd1b84e5f in grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)::operator()(absl::lts_20240116::Status) const /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/posix_engine/posix_endpoint.cc:1311:47
#31 0x589fd1b87ea4 in void std::__invoke_impl<void, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(std::__invoke_other, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /usr/include/c++/13/bits/invoke.h:61:36
#32 0x589fd1b87d39 in std::__invoke_result<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>::type std::__invoke<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /usr/include/c++/13/bits/invoke.h:96:40
#33 0x589fd1b87bbb in std::invoke_result<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>::type std::invoke<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /usr/include/c++/13/functional:113:27
#34 0x589fd1b879dd in void absl::lts_20240116::internal_any_invocable::InvokeR<void, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status, void>(grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status&&) /opt/vcpkg/installed/x64-linux/include/absl/functional/internal/any_invocable.h:132:30
#35 0x589fd1b8761a in void absl::lts_20240116::internal_any_invocable::LocalInvoker<false, void, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::'lambda'(absl::lts_20240116::Status)&, absl::lts_20240116::Status>(absl::lts_20240116::internal_any_invocable::TypeErasedState*, absl::lts_20240116::internal_any_invocable::ForwardedParameterabsl::lts_20240116::Status::type) /opt/vcpkg/installed/x64-linux/include/absl/functional/internal/any_invocable.h:310:31
#36 0x589fd1b6d2aa in absl::lts_20240116::internal_any_invocable::Impl<void (absl::lts_20240116::Status)>::operator()(absl::lts_20240116::Status) /opt/vcpkg/installed/x64-linux/include/absl/functional/internal/any_invocable.h:868:1
#37 0x589fd1b6acc4 in grpc_event_engine::experimental::PosixEngineClosure::Run() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/posix_engine/posix_engine_closure.h:53:10
#38 0x589fd1bd4085 in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::Step() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:428:17
#39 0x589fd1bd3d31 in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::ThreadBody() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:394:14
#40 0x589fd1bd2de0 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread()::'lambda'(void*)::operator()(void*) const /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:207:27
#41 0x589fd1bd2e26 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread()::'lambda'(void*)::_FUN(void*) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:209:7
#42 0x589fd220df5d in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void ()(void), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::operator()(void*) const /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/posix/thd.cc:145:22
#43 0x589fd220dfaa in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void ()(void), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::_FUN(void*) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/posix/thd.cc:150:9
#44 0x589fcfe0319e in asan_thread_start(void*) crtstuff.c
#45 0x7560c8e97b59 in start_thread nptl/pthread_create.c:444:8
#46 0x7560c8f285fb in clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Address 0x7560a2b9ece0 is located in stack of thread T34
SUMMARY: AddressSanitizer: use-after-poison /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/stl_algobase.h:262:17 in unsigned long const& std::max(unsigned long const&, unsigned long const&)
Shadow bytes around the buggy address:
0x7560a2b9ea00: 00 00 00 00 00 00 00 00 f7 f7 00 00 00 00 00 00
0x7560a2b9ea80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7560a2b9eb00: 00 00 00 00 00 00 00 00 f7 f7 00 00 00 00 00 00
0x7560a2b9eb80: 00 00 00 00 00 00 00 00 00 00 00 00 f7 f7 00 00
0x7560a2b9ec00: 00 00 00 00 f7 f7 00 00 f7 f7 00 00 00 00 00 00
=>0x7560a2b9ec80: 00 00 f7 f7 00 00 00 00 f7 f7 00 00[f7]f7 00 00
0x7560a2b9ed00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7560a2b9ed80: 00 00 f7 f7 00 00 00 00 00 00 00 00 00 00 00 00
0x7560a2b9ee00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f7 f7
0x7560a2b9ee80: 00 00 f7 f7 00 00 00 00 00 00 00 00 00 00 00 00
0x7560a2b9ef00: 00 00 f7 f7 00 00 f7 f7 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Thread T34 created by T0 here:
#0 0x589fcfdeae41 in pthread_create (/home/vavan/.vs/LCS/out/install/linux-debug-clang/bin/lcs+0x3847e41) (BuildId: edc3c51e1c08e7c4)
#1 0x589fd220e2f3 in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void ()(void), void*, bool*, grpc_core::Thread::Options const&) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/posix/thd.cc:113:44
#2 0x589fd220e6de in grpc_core::Thread::Thread(char const*, void ()(void), void*, bool*, grpc_core::Thread::Options const&) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/gprpp/posix/thd.cc:199:78
#3 0x589fd1bd2fba in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:203:14
#4 0x589fd1bd2c4b in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::Start() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:181:16
#5 0x589fd1bd2542 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPool(unsigned long) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:128:15
#6 0x589fd1bd22ac in void std::_Construct<grpc_event_engine::experimental::WorkStealingThreadPool, unsigned long&>(grpc_event_engine::experimental::WorkStealingThreadPool*, unsigned long&) /usr/include/c++/13/bits/stl_construct.h:119:7
#7 0x589fd1bd208b in void std::allocator_traits<std::allocator>::construct<grpc_event_engine::experimental::WorkStealingThreadPool, unsigned long&>(std::allocator&, grpc_event_engine::experimental::WorkStealingThreadPool*, unsigned long&) /usr/include/c++/13/bits/alloc_traits.h:660:19
#8 0x589fd1bd208b in std::_Sp_counted_ptr_inplace<grpc_event_engine::experimental::WorkStealingThreadPool, std::allocator, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<unsigned long&>(std::allocator, unsigned long&) /usr/include/c++/13/bits/shared_ptr_base.h:604:39
#9 0x589fd1bd1e55 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<grpc_event_engine::experimental::WorkStealingThreadPool, std::allocator, unsigned long&>(grpc_event_engine::experimental::WorkStealingThreadPool*&, std::_Sp_alloc_shared_tag<std::allocator>, unsigned long&) /usr/include/c++/13/bits/shared_ptr_base.h:971:16
#10 0x589fd1bd1d93 in std::__shared_ptr<grpc_event_engine::experimental::WorkStealingThreadPool, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator, unsigned long&>(std::_Sp_alloc_shared_tag<std::allocator>, unsigned long&) /usr/include/c++/13/bits/shared_ptr_base.h:1712:14
#11 0x589fd1bd1c90 in std::shared_ptr<grpc_event_engine::experimental::WorkStealingThreadPool>::shared_ptr<std::allocator, unsigned long&>(std::_Sp_alloc_shared_tag<std::allocator>, unsigned long&) /usr/include/c++/13/bits/shared_ptr.h:464:59
#12 0x589fd1bd1bd8 in ZSt11make_sharedIN17grpc_event_engine12experimental22WorkStealingThreadPoolEJRmEESt10shared_ptrINSt9enable_ifIXntsrSt8is_arrayIT_E5valueES7_E4typeEEDpOT0 /usr/include/c++/13/bits/shared_ptr.h:1010:39
#13 0x589fd1bd1a45 in grpc_event_engine::experimental::MakeThreadPool(unsigned long) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/thread_pool/thread_pool_factory.cc:40:78
#14 0x589fd1b65985 in grpc_event_engine::experimental::PosixEventEngine::PosixEventEngine() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/posix_engine/posix_engine.cc:377:7
#15 0x589fd1b627b9 in std::__detail::_MakeUniq<grpc_event_engine::experimental::PosixEventEngine>::__single_object std::make_unique<grpc_event_engine::experimental::PosixEventEngine>() /usr/include/c++/13/bits/unique_ptr.h:1070:30
#16 0x589fd1b6270c in grpc_event_engine::experimental::DefaultEventEngineFactory() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/default_event_engine_factory.cc:54:44
#17 0x589fd1b60674 in grpc_event_engine::experimental::CreateEventEngineInner() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/default_event_engine.cc:68:36
#18 0x589fd1b606b9 in grpc_event_engine::experimental::CreateEventEngine() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/default_event_engine.cc:75:33
#19 0x589fd1b6082c in grpc_event_engine::experimental::GetDefaultEventEngine(grpc_core::SourceLocation) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/default_event_engine.cc:89:56
#20 0x589fd1b60a27 in grpc_event_engine::experimental::(anonymous namespace)::EnsureEventEngineInChannelArgs(grpc_core::ChannelArgs) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/event_engine/default_event_engine.cc:100:59
#21 0x589fd1b61fa4 in grpc_core::ChannelArgs std::__invoke_impl<grpc_core::ChannelArgs, grpc_core::ChannelArgs (&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs>(std::__invoke_other, grpc_core::ChannelArgs (&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs&&) /usr/include/c++/13/bits/invoke.h:61:67
#22 0x589fd1b61c92 in std::enable_if<is_invocable_r_v<grpc_core::ChannelArgs, grpc_core::ChannelArgs (&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs>, grpc_core::ChannelArgs>::type std::__invoke_r<grpc_core::ChannelArgs, grpc_core::ChannelArgs (&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs>(grpc_core::ChannelArgs (&)(grpc_core::ChannelArgs), grpc_core::ChannelArgs&&) /usr/include/c++/13/bits/invoke.h:116:38
#23 0x589fd1b616fd in std::_Function_handler<grpc_core::ChannelArgs (grpc_core::ChannelArgs), grpc_core::ChannelArgs ()(grpc_core::ChannelArgs)>::_M_invoke(std::_Any_data const&, grpc_core::ChannelArgs&&) /usr/include/c++/13/bits/std_function.h:291:44
#24 0x589fd1b3246f in std::function<grpc_core::ChannelArgs (grpc_core::ChannelArgs)>::operator()(grpc_core::ChannelArgs) const /usr/include/c++/13/bits/std_function.h:591:66
#25 0x589fd1b31f61 in grpc_core::ChannelArgsPreconditioning::PreconditionChannelArgs(grpc_channel_args const*) const /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/channel/channel_args_preconditioning.cc:37:25
#26 0x589fd1d9e39a in grpc_server_create /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/core/lib/surface/server.cc:1905:57
#27 0x589fd16d7fe0 in grpc::Server::Server(grpc::ChannelArguments*, std::shared_ptr<std::vector<std::unique_ptr<grpc::ServerCompletionQueue, std::default_deletegrpc::ServerCompletionQueue>, std::allocator<std::unique_ptr<grpc::ServerCompletionQueue, std::default_deletegrpc::ServerCompletionQueue>>>>, int, int, int, std::vector<std::shared_ptrgrpc::internal::ExternalConnectionAcceptorImpl, std::allocator<std::shared_ptrgrpc::internal::ExternalConnectionAcceptorImpl>>, grpc_server_config_fetcher*, grpc_resource_quota*, std::vector<std::unique_ptr<grpc::experimental::ServerInterceptorFactoryInterface, std::default_deletegrpc::experimental::ServerInterceptorFactoryInterface>, std::allocator<std::unique_ptr<grpc::experimental::ServerInterceptorFactoryInterface, std::default_deletegrpc::experimental::ServerInterceptorFactoryInterface>>>, grpc::experimental::ServerMetricRecorder*) /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/cpp/server/server_cc.cc:964:31
#28 0x589fd16c9cb8 in grpc::ServerBuilder::BuildAndStart() /opt/vcpkg/buildtrees/grpc/src/v1.60.0-2dcaca28c0.clean/src/cpp/server/server_builder.cc:368:64
#29 0x589fd0d36654 in lcs::grpc::StartService() /home/vavan/.vs/LCS/src/common/grpc.cpp:142:30
#30 0x589fd0a9ed8d in lcs::main_common(int, char const* const*, std::function<void ()>) /home/vavan/.vs/LCS/src/common/common.cpp:808:26
#31 0x589fcfe48373 in main /home/vavan/.vs/LCS/src/server/main.cpp:72:19
#32 0x7560c8e2814f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

==597460==ABORTING

`

lcs597460 2024-07-02 10-41-30.txt

@zlojvavan zlojvavan mentioned this issue Jul 2, 2024
Closed
@derekmauro
Copy link
Member

Repeating what I said again: Without reproduction instructions, we cannot look into this.

@julianoes
Copy link
Author

@zlojvavan do you happen to have minimal steps to reproduce it?

@zlojvavan
Copy link

@julianoes no, got it in real-world project that worked just fine with asan before updating grpc

@sburton84
Copy link

After an update to gRPC/Abseil I also started getting the same use-after-poison error as above. Though after updating again I'm now getting a different use-after-poison error:

76: ==415963==ERROR: AddressSanitizer: use-after-poison on address 0x7ffdc4b9c090 at pc 0x5608895af2e4 bp 0x7ffdc4b9bfa0 sp 0x7ffdc4b9bf98
76: READ of size 1 at 0x7ffdc4b9c090 thread T0
76:     #0 0x5608895af2e3 in std::_Optional_base_impl<unsigned long, std::_Optional_base<unsigned long, true, true>>::_M_is_engaged() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/optional:469:58
76:     #1 0x560889dcf4c4 in std::optional<unsigned long>::has_value() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/optional:986:22
76:     #2 0x56088ad9987c in absl::lts_20240116::status_internal::StatusRep::SetPayload(std::basic_string_view<char, std::char_traits<char>>, absl::lts_20240116::Cord) /home/sburton/.conan/data/abseil/20240116.2/_/_/build/94496ed8f083396518281a0e71b7ac7c92bc01d2/src/absl/status/internal/status_internal.cc:85:13
76:     #3 0x560889d5f023 in absl::lts_20240116::Status::SetPayload(std::basic_string_view<char, std::char_traits<char>>, absl::lts_20240116::Cord) /home/sburton/.conan/data/abseil/20240116.2/_/_/package/94496ed8f083396518281a0e71b7ac7c92bc01d2/include/absl/status/status.h:869:8
76:     #4 0x560889e18884 in grpc_core::StatusSetStr(absl::lts_20240116::Status*, grpc_core::StatusStrProperty, std::basic_string_view<char, std::char_traits<char>>) /home/sburton/.conan/data/grpc/1.65.0/_/_/build/22e809ff8e53971a8ae5dc120d9464e44340b561/src/src/core/lib/gprpp/status_helper.cc:184:11
76:     #5 0x560889e185e9 in grpc_core::StatusCreate(absl::lts_20240116::StatusCode, std::basic_string_view<char, std::char_traits<char>>, grpc_core::DebugLocation const&, std::vector<absl::lts_20240116::Status, std::allocator<absl::lts_20240116::Status>>) /home/sburton/.conan/data/grpc/1.65.0/_/_/build/22e809ff8e53971a8ae5dc120d9464e44340b561/src/src/core/lib/gprpp/status_helper.cc:143:5
76:     #6 0x560889ebb126 in grpc_core::LegacyChannel::Orphaned() /home/sburton/.conan/data/grpc/1.65.0/_/_/build/22e809ff8e53971a8ae5dc120d9464e44340b561/src/src/core/lib/surface/legacy_channel.cc:167:31
76:     #7 0x560889d2b564 in grpc_core::DualRefCounted<grpc_core::UnstartedCallDestination, grpc_core::PolymorphicRefCount, grpc_core::UnrefDelete>::Unref() /home/sburton/.conan/data/grpc/1.65.0/_/_/build/22e809ff8e53971a8ae5dc120d9464e44340b561/src/src/core/lib/gprpp/dual_ref_counted.h:103:7
76:     #8 0x560889e9269c in grpc_channel_destroy_internal(grpc_channel*) /home/sburton/.conan/data/grpc/1.65.0/_/_/build/22e809ff8e53971a8ae5dc120d9464e44340b561/src/src/core/lib/surface/channel.h:156:39
76:     #9 0x560889e90ef6 in grpc_channel_destroy /home/sburton/.conan/data/grpc/1.65.0/_/_/build/22e809ff8e53971a8ae5dc120d9464e44340b561/src/src/core/lib/surface/channel.cc:99:3
76:     #10 0x560889d0199c in grpc::Channel::~Channel() /home/sburton/.conan/data/grpc/1.65.0/_/_/build/22e809ff8e53971a8ae5dc120d9464e44340b561/src/src/cpp/client/channel_cc.cc:59:3

The fact that a different gRPC version exhibits a similar error but pointing to a completely different part of Abseil does suggest that the issue may be with gRPC rather than Abseil (though it's still hard to say for sure).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@derekmauro @julianoes @sburton84 @zlojvavan