[tsan] Add callbacks for futex syscalls and mark them as blocking on tsan #86537
Conversation
|
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write If you have received no comments on your PR for a week, you can request a review If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
llvmbot
added
compiler-rt
compiler-rt:tsan
|
@llvm/pr-subscribers-compiler-rt-sanitizer Author: Nazım Can Altınova (canova) ChangesFixes #83844. This PR adds callbacks to mark futex syscalls as blocking. Unfortunately we didn't have a mechanism before to mark syscalls as a blocking call, so I had to implement it, but it mostly reuses the The issue includes some information but this issue was discovered because Rust uses futexes directly. So most likely we need to update Rust as well to use these callbacks. Also see the latest comments in #85188 for some context. Full diff: https://github.com/llvm/llvm-project/pull/86537.diff 4 Files Affected:
diff --git a/compiler-rt/include/sanitizer/linux_syscall_hooks.h b/compiler-rt/include/sanitizer/linux_syscall_hooks.h
index 3f3f1e78dfb859..5f262455cb9466 100644
--- a/compiler-rt/include/sanitizer/linux_syscall_hooks.h
+++ b/compiler-rt/include/sanitizer/linux_syscall_hooks.h
@@ -1856,6 +1856,15 @@
__sanitizer_syscall_pre_impl_sigaltstack((long)ss, (long)oss)
#define __sanitizer_syscall_post_sigaltstack(res, ss, oss) \
__sanitizer_syscall_post_impl_sigaltstack(res, (long)ss, (long)oss)
+#define __sanitizer_syscall_pre_futex(uaddr, futex_op, val, timeout, uaddr2, \
+ val3) \
+ __sanitizer_syscall_pre_impl_futex((long)uaddr, (long)futex_op, (long)val, \
+ (long)timeout, (long)uaddr2, (long)val3)
+#define __sanitizer_syscall_post_futex(res, uaddr, futex_op, val, timeout, \
+ uaddr2, val3) \
+ __sanitizer_syscall_post_impl_futex(res, (long)uaddr, (long)futex_op, \
+ (long)val, (long)timeout, (long)uaddr2, \
+ (long)val3)
// And now a few syscalls we don't handle yet.
#define __sanitizer_syscall_pre_afs_syscall(...)
@@ -1875,7 +1884,6 @@
#define __sanitizer_syscall_pre_fchown32(...)
#define __sanitizer_syscall_pre_ftime(...)
#define __sanitizer_syscall_pre_ftruncate64(...)
-#define __sanitizer_syscall_pre_futex(...)
#define __sanitizer_syscall_pre_getegid32(...)
#define __sanitizer_syscall_pre_geteuid32(...)
#define __sanitizer_syscall_pre_getgid32(...)
@@ -1954,7 +1962,6 @@
#define __sanitizer_syscall_post_fchown32(res, ...)
#define __sanitizer_syscall_post_ftime(res, ...)
#define __sanitizer_syscall_post_ftruncate64(res, ...)
-#define __sanitizer_syscall_post_futex(res, ...)
#define __sanitizer_syscall_post_getegid32(res, ...)
#define __sanitizer_syscall_post_geteuid32(res, ...)
#define __sanitizer_syscall_post_getgid32(res, ...)
@@ -3093,6 +3100,11 @@ void __sanitizer_syscall_post_impl_rt_sigaction(long res, long signum, long act,
long oldact, long sz);
void __sanitizer_syscall_pre_impl_sigaltstack(long ss, long oss);
void __sanitizer_syscall_post_impl_sigaltstack(long res, long ss, long oss);
+void __sanitizer_syscall_pre_impl_futex(long uaddr, long futex_op, long val,
+ long timeout, long uaddr2, long val3);
+void __sanitizer_syscall_post_impl_futex(long res, long uaddr, long futex_op,
+ long val, long timeout, long uaddr2,
+ long val3);
#ifdef __cplusplus
} // extern "C"
#endif
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_common_syscalls.inc b/compiler-rt/lib/sanitizer_common/sanitizer_common_syscalls.inc
index b3161690f3ce8a..14615f9668dea6 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_common_syscalls.inc
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_common_syscalls.inc
@@ -38,6 +38,10 @@
// Called before fork syscall.
// COMMON_SYSCALL_POST_FORK(long res)
// Called after fork syscall.
+// COMMON_SYSCALL_BLOCKING_START()
+// Called before blocking syscall.
+// COMMON_SYSCALL_BLOCKING_END()
+// Called after blocking syscall.
//===----------------------------------------------------------------------===//
#include "sanitizer_platform.h"
@@ -85,6 +89,16 @@
{}
# endif
+# ifndef COMMON_SYSCALL_BLOCKING_START
+# define COMMON_SYSCALL_BLOCKING_START() \
+ {}
+# endif
+
+# ifndef COMMON_SYSCALL_BLOCKING_END
+# define COMMON_SYSCALL_BLOCKING_END() \
+ {}
+# endif
+
// FIXME: do some kind of PRE_READ for all syscall arguments (int(s) and such).
extern "C" {
@@ -3176,6 +3190,18 @@ POST_SYSCALL(sigaltstack)(long res, void *ss, void *oss) {
}
}
}
+
+PRE_SYSCALL(futex)
+(void *uaddr, long futex_op, long val, void *timeout, void *uaddr2, long val3) {
+ COMMON_SYSCALL_BLOCKING_START();
+}
+
+POST_SYSCALL(futex)
+(long res, void *uaddr, long futex_op, long val, void *timeout, void *uaddr2,
+ long val3) {
+ COMMON_SYSCALL_BLOCKING_END();
+}
+
} // extern "C"
# undef PRE_SYSCALL
diff --git a/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp b/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp
index 8ffc703b05eace..ef4796bbb8bd67 100644
--- a/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp
+++ b/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp
@@ -2673,6 +2673,25 @@ static USED void syscall_fd_release(uptr pc, int fd) {
FdRelease(thr, pc, fd);
}
+static USED void sycall_blocking_start() {
+ DPrintf("sycall_blocking_start()\n");
+ ThreadState *thr = cur_thread();
+ EnterBlockingFunc(thr);
+ // When we are in a "blocking call", we process signals asynchronously
+ // (right when they arrive). In this context we do not expect to be
+ // executing any user/runtime code. The known interceptor sequence when
+ // this is not true is: pthread_join -> munmap(stack). It's fine
+ // to ignore munmap in this case -- we handle stack shadow separately.
+ thr->ignore_interceptors++;
+}
+
+static USED void sycall_blocking_end() {
+ DPrintf("sycall_blocking_end()\n");
+ ThreadState *thr = cur_thread();
+ thr->ignore_interceptors--;
+ atomic_store(&thr->in_blocking_func, 0, memory_order_relaxed);
+}
+
static void syscall_pre_fork(uptr pc) { ForkBefore(cur_thread(), pc); }
static void syscall_post_fork(uptr pc, int pid) {
@@ -2727,6 +2746,9 @@ static void syscall_post_fork(uptr pc, int pid) {
#define COMMON_SYSCALL_POST_FORK(res) \
syscall_post_fork(GET_CALLER_PC(), res)
+#define COMMON_SYSCALL_BLOCKING_START() sycall_blocking_start()
+#define COMMON_SYSCALL_BLOCKING_END() sycall_blocking_end()
+
#include "sanitizer_common/sanitizer_common_syscalls.inc"
#include "sanitizer_common/sanitizer_syscalls_netbsd.inc"
diff --git a/compiler-rt/test/tsan/signal_in_futex_wait.cpp b/compiler-rt/test/tsan/signal_in_futex_wait.cpp
new file mode 100644
index 00000000000000..cf31e5467486ad
--- /dev/null
+++ b/compiler-rt/test/tsan/signal_in_futex_wait.cpp
@@ -0,0 +1,116 @@
+// RUN: %clang_tsan %s -lstdc++ -o %t && %run %t 2>&1 | FileCheck %s
+
+#include "test.h"
+#include <errno.h>
+#include <linux/futex.h>
+#include <pthread.h>
+#include <signal.h>
+#include <stdio.h>
+#include <sys/syscall.h>
+
+#include <cassert>
+#include <stdexcept>
+#include <thread>
+
+#include <sanitizer/linux_syscall_hooks.h>
+
+int futex(int *uaddr, int futex_op, int val, const struct timespec *timeout,
+ int *uaddr2, int val3) {
+ __sanitizer_syscall_pre_futex(uaddr, futex_op, val, timeout, uaddr2, val3);
+ int result = syscall(SYS_futex, uaddr, futex_op, val, timeout, uaddr2, val3);
+ __sanitizer_syscall_post_futex(result, uaddr, futex_op, val, timeout, uaddr2,
+ val3);
+ return result;
+}
+
+// Simple mutex implementation using futex.
+class Mutex {
+public:
+ Mutex() : value(0) {}
+
+ void lock() {
+ int c;
+ while ((c = __sync_val_compare_and_swap(&value, 0, 1)) != 0) {
+ if (c != 1)
+ continue;
+ int r = futex(&value, FUTEX_WAIT_PRIVATE, 1, nullptr, nullptr, 0);
+ if (r == -1 && errno != EAGAIN) {
+ fprintf(stderr, "futex wait error\n");
+ abort();
+ }
+ }
+ }
+
+ void unlock() {
+ value = 0;
+ int r = futex(&value, FUTEX_WAKE_PRIVATE, 1, nullptr, nullptr, 0);
+ if (r == -1) {
+ fprintf(stderr, "futex wake error\n");
+ abort();
+ }
+ }
+
+private:
+ int value;
+};
+
+Mutex mutex;
+
+void *Thread(void *x) {
+ // fprintf(stderr, "canova here thread 0\n");
+ // Waiting for the futex.
+ mutex.lock();
+ // fprintf(stderr, "canova here thread 1\n");
+ // Finished waiting.
+ return nullptr;
+}
+
+static void SigprofHandler(int signal, siginfo_t *info, void *context) {
+ // fprintf(stderr, "canova here sigprof handler\n");
+ // Unlock the futex.
+ mutex.unlock();
+}
+
+void InstallSignalHandler() {
+ struct sigaction sa;
+ sa.sa_sigaction = SigprofHandler;
+ sigemptyset(&sa.sa_mask);
+ sa.sa_flags = SA_RESTART | SA_SIGINFO;
+ if (sigaction(SIGPROF, &sa, 0) != 0) {
+ fprintf(stderr, "failed to install signal handler\n");
+ abort();
+ }
+}
+
+int main() {
+ alarm(60); // Kill the test if it hangs.
+
+ // Install the signal handler
+ InstallSignalHandler();
+
+ // Lock the futex at first so the other thread will wait for it.
+ mutex.lock();
+
+ // Create the thread to wait for the futex.
+ pthread_t thread;
+ pthread_create(&thread, NULL, Thread, NULL);
+
+ // Just waiting a bit to make sure the thead is at the FUTEX_WAIT_PRIVATE
+ // syscall.
+ std::this_thread::sleep_for(std::chrono::milliseconds(100));
+
+ // Send the signal to the other thread, which will send the futex wake
+ // syscall.
+ int r = pthread_kill(thread, SIGPROF);
+ assert(r == 0);
+
+ // Futex should be notified and the thread should be able to continue.
+ pthread_join(thread, NULL);
+
+ // Exiting successfully.
+ fprintf(stderr, "PASS\n");
+ return 0;
+}
+
+// CHECK-NOT: WARNING: ThreadSanitizer:
+// CHECK: PASS
|
|
@canova Congratulations on having your first Pull Request (PR) merged into the LLVM Project! Your changes will be combined with recent changes from other authors, then tested Please check whether problems have been caused by your change specifically, as How to do this, and the rest of the post-merge process, is covered in detail here. If your change does cause a problem, it may be reverted, or you can revert it yourself. If you don't get any reports, no action is required from you. Your changes are working as expected, well done! |
|
@canova, as Dan Liew (delcypher) mentioned above, the test added in the PR seems to be specific to Linux: I think the simplest way to only run this test on Linux is to move it into the |
Linux specific test was introduced by #86537
|
@delcypher @yln Sorry about the inconvenience! This is indeed a linux only test, moving this test to the Linux subdirectory in #86896. |
|
Oh nvm, looks like it's fixed already. |
Fixes #83844.
This PR adds callbacks to mark futex syscalls as blocking. Unfortunately we didn't have a mechanism before to mark syscalls as a blocking call, so I had to implement it, but it mostly reuses the
BlockingCallimplementation here.The issue includes some information but this issue was discovered because Rust uses futexes directly. So most likely we need to update Rust as well to use these callbacks.
Also see the latest comments in #85188 for some context.
I also sent another PR #84162 to mark
pthread_*_lockcalls as blocking.