[compiler-rt] r273260 - [asan] add primitives that allow coroutine implementations
Mike Aizatsky via llvm-commits
llvm-commits at lists.llvm.org
Tue Jun 21 11:28:03 PDT 2016
Dmitry,
I think this breaks sanitizer-windows build:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/24305/steps/run%20tests/logs/stdio
First build with the problem:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/24286
On Tue, Jun 21, 2016 at 5:36 AM Dmitry Vyukov via llvm-commits <
llvm-commits at lists.llvm.org> wrote:
> Author: dvyukov
> Date: Tue Jun 21 07:29:18 2016
> New Revision: 273260
>
> URL: http://llvm.org/viewvc/llvm-project?rev=273260&view=rev
> Log:
> [asan] add primitives that allow coroutine implementations
>
> This patch adds the __sanitizer_start_switch_fiber and
> __sanitizer_finish_switch_fiber methods inspired from what can be found
> here
>
> https://github.com/facebook/folly/commit/2ea64dd24946cbc9f3f4ac3f6c6b98a486c56e73
> .
>
> These methods are needed when the compiled software needs to implement
> coroutines, fibers or the like. Without a way to annotate them, when the
> program
> jumps to a stack that is not the thread stack, __asan_handle_no_return
> shows a
> warning about that, and the fake stack mechanism may free fake frames that
> are
> still in use.
>
> Author: blastrock (Philippe Daouadi)
> Reviewed in http://reviews.llvm.org/D20913
>
>
> Added:
> compiler-rt/trunk/test/asan/TestCases/Linux/swapcontext_annotation.cc
> Modified:
> compiler-rt/trunk/include/sanitizer/common_interface_defs.h
> compiler-rt/trunk/lib/asan/asan_thread.cc
> compiler-rt/trunk/lib/asan/asan_thread.h
>
> Modified: compiler-rt/trunk/include/sanitizer/common_interface_defs.h
> URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/include/sanitizer/common_interface_defs.h?rev=273260&r1=273259&r2=273260&view=diff
>
> ==============================================================================
> --- compiler-rt/trunk/include/sanitizer/common_interface_defs.h (original)
> +++ compiler-rt/trunk/include/sanitizer/common_interface_defs.h Tue Jun 21
> 07:29:18 2016
> @@ -139,6 +139,26 @@ extern "C" {
> // `top_percent` should be between 1 and 100.
> // Experimental feature currently available only with asan on
> Linux/x86_64.
> void __sanitizer_print_memory_profile(size_t top_percent);
> +
> + // Fiber annotation interface.
> + // Before switching to a different stack, one must call
> + // __sanitizer_start_switch_fiber with a pointer to the bottom of the
> + // destination stack and its size. When code starts running on the new
> stack,
> + // it must call __sanitizer_finish_switch_fiber to finalize the switch.
> + // The start_switch function takes a void** to store the current fake
> stack if
> + // there is one (it is needed when detect_stack_use_after_return is
> enabled).
> + // When restoring a stack, this pointer must be given to the
> finish_switch
> + // function. In most cases, this void* can be stored on the stack just
> before
> + // switching. When leaving a fiber definitely, null must be passed as
> first
> + // argument to the start_switch function so that the fake stack is
> destroyed.
> + // If you do not want support for stack use-after-return detection, you
> can
> + // always pass null to these two functions.
> + // Note that the fake stack mechanism is disabled during fiber switch,
> so if a
> + // signal callback runs during the switch, it will not benefit from the
> stack
> + // use-after-return detection.
> + void __sanitizer_start_switch_fiber(void **fake_stack_save,
> + const void *bottom, size_t size);
> + void __sanitizer_finish_switch_fiber(void *fake_stack_save);
> #ifdef __cplusplus
> } // extern "C"
> #endif
>
> Modified: compiler-rt/trunk/lib/asan/asan_thread.cc
> URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/asan/asan_thread.cc?rev=273260&r1=273259&r2=273260&view=diff
>
> ==============================================================================
> --- compiler-rt/trunk/lib/asan/asan_thread.cc (original)
> +++ compiler-rt/trunk/lib/asan/asan_thread.cc Tue Jun 21 07:29:18 2016
> @@ -120,6 +120,71 @@ void AsanThread::Destroy() {
> DTLS_Destroy();
> }
>
> +void AsanThread::StartSwitchFiber(FakeStack **fake_stack_save, uptr
> bottom,
> + uptr size) {
> + if (atomic_load(&stack_switching_, memory_order_relaxed)) {
> + Report("ERROR: starting fiber switch while in fiber switch\n");
> + Die();
> + }
> +
> + next_stack_bottom_ = bottom;
> + next_stack_top_ = bottom + size;
> + atomic_store(&stack_switching_, 1, memory_order_release);
> +
> + FakeStack *current_fake_stack = fake_stack_;
> + if (fake_stack_save)
> + *fake_stack_save = fake_stack_;
> + fake_stack_ = nullptr;
> + SetTLSFakeStack(nullptr);
> + // if fake_stack_save is null, the fiber will die, delete the fakestack
> + if (!fake_stack_save && current_fake_stack)
> + current_fake_stack->Destroy(this->tid());
> +}
> +
> +void AsanThread::FinishSwitchFiber(FakeStack *fake_stack_save) {
> + if (!atomic_load(&stack_switching_, memory_order_relaxed)) {
> + Report("ERROR: finishing a fiber switch that has not started\n");
> + Die();
> + }
> +
> + if (fake_stack_save) {
> + SetTLSFakeStack(fake_stack_save);
> + fake_stack_ = fake_stack_save;
> + }
> +
> + stack_bottom_ = next_stack_bottom_;
> + stack_top_ = next_stack_top_;
> + atomic_store(&stack_switching_, 0, memory_order_release);
> + next_stack_top_ = 0;
> + next_stack_bottom_ = 0;
> +}
> +
> +inline AsanThread::StackBounds AsanThread::GetStackBounds() const {
> + if (!atomic_load(&stack_switching_, memory_order_acquire))
> + return StackBounds{stack_bottom_, stack_top_}; // NOLINT
> + char local;
> + const uptr cur_stack = (uptr)&local;
> + // Note: need to check next stack first, because FinishSwitchFiber
> + // may be in process of overwriting stack_top_/bottom_. But in such case
> + // we are already on the next stack.
> + if (cur_stack >= next_stack_bottom_ && cur_stack < next_stack_top_)
> + return StackBounds{next_stack_bottom_, next_stack_top_}; // NOLINT
> + return StackBounds{stack_bottom_, stack_top_}; // NOLINT
> +}
> +
> +uptr AsanThread::stack_top() {
> + return GetStackBounds().top;
> +}
> +
> +uptr AsanThread::stack_bottom() {
> + return GetStackBounds().bottom;
> +}
> +
> +uptr AsanThread::stack_size() {
> + const auto bounds = GetStackBounds();
> + return bounds.top - bounds.bottom;
> +}
> +
> // We want to create the FakeStack lazyly on the first use, but not
> eralier
> // than the stack size is known and the procedure has to be async-signal
> safe.
> FakeStack *AsanThread::AsyncSignalSafeLazyInitFakeStack() {
> @@ -150,6 +215,8 @@ FakeStack *AsanThread::AsyncSignalSafeLa
> }
>
> void AsanThread::Init() {
> + next_stack_top_ = next_stack_bottom_ = 0;
> + atomic_store(&stack_switching_, false, memory_order_release);
> fake_stack_ = nullptr; // Will be initialized lazily if needed.
> CHECK_EQ(this->stack_size(), 0U);
> SetThreadStackAndTls();
> @@ -195,9 +262,10 @@ thread_return_t AsanThread::ThreadStart(
>
> void AsanThread::SetThreadStackAndTls() {
> uptr tls_size = 0;
> - GetThreadStackAndTls(tid() == 0, &stack_bottom_, &stack_size_,
> &tls_begin_,
> - &tls_size);
> - stack_top_ = stack_bottom_ + stack_size_;
> + uptr stack_size = 0;
> + GetThreadStackAndTls(tid() == 0, const_cast<uptr *>(&stack_bottom_),
> + const_cast<uptr *>(&stack_size), &tls_begin_,
> &tls_size);
> + stack_top_ = stack_bottom_ + stack_size;
> tls_end_ = tls_begin_ + tls_size;
> dtls_ = DTLS_Get();
>
> @@ -250,6 +318,11 @@ bool AsanThread::GetStackFrameAccessByAd
> return true;
> }
>
> +bool AsanThread::AddrIsInStack(uptr addr) {
> + const auto bounds = GetStackBounds();
> + return addr >= bounds.bottom && addr < bounds.top;
> +}
> +
> static bool ThreadStackContainsAddress(ThreadContextBase *tctx_base,
> void *addr) {
> AsanThreadContext *tctx = static_cast<AsanThreadContext*>(tctx_base);
> @@ -357,3 +430,29 @@ void EnsureMainThreadIDIsCorrect() {
> __asan::EnsureMainThreadIDIsCorrect();
> }
> } // namespace __lsan
> +
> +// ---------------------- Interface ---------------- {{{1
> +using namespace __asan; // NOLINT
> +
> +extern "C" {
> +SANITIZER_INTERFACE_ATTRIBUTE
> +void __sanitizer_start_switch_fiber(void **fakestacksave, const void
> *bottom,
> + uptr size) {
> + AsanThread *t = GetCurrentThread();
> + if (!t) {
> + VReport(1, "__asan_start_switch_fiber called from unknown thread\n");
> + return;
> + }
> + t->StartSwitchFiber((FakeStack**)fakestacksave, (uptr)bottom, size);
> +}
> +
> +SANITIZER_INTERFACE_ATTRIBUTE
> +void __sanitizer_finish_switch_fiber(void* fakestack) {
> + AsanThread *t = GetCurrentThread();
> + if (!t) {
> + VReport(1, "__asan_finish_switch_fiber called from unknown thread\n");
> + return;
> + }
> + t->FinishSwitchFiber((FakeStack*)fakestack);
> +}
> +}
>
> Modified: compiler-rt/trunk/lib/asan/asan_thread.h
> URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/asan/asan_thread.h?rev=273260&r1=273259&r2=273260&view=diff
>
> ==============================================================================
> --- compiler-rt/trunk/lib/asan/asan_thread.h (original)
> +++ compiler-rt/trunk/lib/asan/asan_thread.h Tue Jun 21 07:29:18 2016
> @@ -66,9 +66,9 @@ class AsanThread {
> thread_return_t ThreadStart(uptr os_id,
> atomic_uintptr_t
> *signal_thread_is_registered);
>
> - uptr stack_top() { return stack_top_; }
> - uptr stack_bottom() { return stack_bottom_; }
> - uptr stack_size() { return stack_size_; }
> + uptr stack_top();
> + uptr stack_bottom();
> + uptr stack_size();
> uptr tls_begin() { return tls_begin_; }
> uptr tls_end() { return tls_end_; }
> DTLS *dtls() { return dtls_; }
> @@ -83,9 +83,7 @@ class AsanThread {
> };
> bool GetStackFrameAccessByAddr(uptr addr, StackFrameAccess *access);
>
> - bool AddrIsInStack(uptr addr) {
> - return addr >= stack_bottom_ && addr < stack_top_;
> - }
> + bool AddrIsInStack(uptr addr);
>
> void DeleteFakeStack(int tid) {
> if (!fake_stack_) return;
> @@ -95,13 +93,19 @@ class AsanThread {
> t->Destroy(tid);
> }
>
> + void StartSwitchFiber(FakeStack **fake_stack_save, uptr bottom, uptr
> size);
> + void FinishSwitchFiber(FakeStack *fake_stack_save);
> +
> bool has_fake_stack() {
> - return (reinterpret_cast<uptr>(fake_stack_) > 1);
> + return !atomic_load(&stack_switching_, memory_order_relaxed) &&
> + (reinterpret_cast<uptr>(fake_stack_) > 1);
> }
>
> FakeStack *fake_stack() {
> if (!__asan_option_detect_stack_use_after_return)
> return nullptr;
> + if (atomic_load(&stack_switching_, memory_order_relaxed))
> + return nullptr;
> if (!has_fake_stack())
> return AsyncSignalSafeLazyInitFakeStack();
> return fake_stack_;
> @@ -127,14 +131,24 @@ class AsanThread {
> void ClearShadowForThreadStackAndTLS();
> FakeStack *AsyncSignalSafeLazyInitFakeStack();
>
> + struct StackBounds {
> + uptr bottom;
> + uptr top;
> + };
> + StackBounds GetStackBounds() const;
> +
> AsanThreadContext *context_;
> thread_callback_t start_routine_;
> void *arg_;
> +
> uptr stack_top_;
> uptr stack_bottom_;
> - // stack_size_ == stack_top_ - stack_bottom_;
> - // It needs to be set in a async-signal-safe manner.
> - uptr stack_size_;
> + // these variables are used when the thread is about to switch stack
> + uptr next_stack_top_;
> + uptr next_stack_bottom_;
> + // true if switching is in progress
> + atomic_uint8_t stack_switching_;
> +
> uptr tls_begin_;
> uptr tls_end_;
> DTLS *dtls_;
>
> Added:
> compiler-rt/trunk/test/asan/TestCases/Linux/swapcontext_annotation.cc
> URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/asan/TestCases/Linux/swapcontext_annotation.cc?rev=273260&view=auto
>
> ==============================================================================
> --- compiler-rt/trunk/test/asan/TestCases/Linux/swapcontext_annotation.cc
> (added)
> +++ compiler-rt/trunk/test/asan/TestCases/Linux/swapcontext_annotation.cc
> Tue Jun 21 07:29:18 2016
> @@ -0,0 +1,178 @@
> +// Check that ASan plays well with annotated makecontext/swapcontext.
> +
> +// RUN: %clangxx_asan -lpthread -O0 %s -o %t && %run %t 2>&1 | FileCheck
> %s
> +// RUN: %clangxx_asan -lpthread -O1 %s -o %t && %run %t 2>&1 | FileCheck
> %s
> +// RUN: %clangxx_asan -lpthread -O2 %s -o %t && %run %t 2>&1 | FileCheck
> %s
> +// RUN: %clangxx_asan -lpthread -O3 %s -o %t && %run %t 2>&1 | FileCheck
> %s
> +//
> +// This test is too subtle to try on non-x86 arch for now.
> +// REQUIRES: x86_64-supported-target,i386-supported-target
> +
> +#include <pthread.h>
> +#include <setjmp.h>
> +#include <stdio.h>
> +#include <sys/time.h>
> +#include <ucontext.h>
> +#include <unistd.h>
> +
> +#include <sanitizer/common_interface_defs.h>
> +
> +ucontext_t orig_context;
> +ucontext_t child_context;
> +ucontext_t next_child_context;
> +
> +char *next_child_stack;
> +
> +const int kStackSize = 1 << 20;
> +
> +void *main_thread_stack;
> +size_t main_thread_stacksize;
> +
> +__attribute__((noinline, noreturn)) void LongJump(jmp_buf env) {
> + longjmp(env, 1);
> + _exit(1);
> +}
> +
> +// Simulate __asan_handle_no_return().
> +__attribute__((noinline)) void CallNoReturn() {
> + jmp_buf env;
> + if (setjmp(env) != 0) return;
> +
> + LongJump(env);
> + _exit(1);
> +}
> +
> +void NextChild() {
> + CallNoReturn();
> + __sanitizer_finish_switch_fiber();
> +
> + char x[32] = {0}; // Stack gets poisoned.
> + printf("NextChild: %p\n", x);
> +
> + CallNoReturn();
> +
> + __sanitizer_start_switch_fiber(main_thread_stack,
> main_thread_stacksize);
> + CallNoReturn();
> + if (swapcontext(&next_child_context, &orig_context) < 0) {
> + perror("swapcontext");
> + _exit(1);
> + }
> +}
> +
> +void Child(int mode) {
> + CallNoReturn();
> + __sanitizer_finish_switch_fiber();
> + char x[32] = {0}; // Stack gets poisoned.
> + printf("Child: %p\n", x);
> + CallNoReturn();
> + // (a) Do nothing, just return to parent function.
> + // (b) Jump into the original function. Stack remains poisoned unless
> we do
> + // something.
> + // (c) Jump to another function which will then jump back to the main
> function
> + if (mode == 0) {
> + __sanitizer_start_switch_fiber(main_thread_stack,
> main_thread_stacksize);
> + CallNoReturn();
> + } else if (mode == 1) {
> + __sanitizer_start_switch_fiber(main_thread_stack,
> main_thread_stacksize);
> + CallNoReturn();
> + if (swapcontext(&child_context, &orig_context) < 0) {
> + perror("swapcontext");
> + _exit(1);
> + }
> + } else if (mode == 2) {
> + getcontext(&next_child_context);
> + next_child_context.uc_stack.ss_sp = next_child_stack;
> + next_child_context.uc_stack.ss_size = kStackSize / 2;
> + makecontext(&next_child_context, (void (*)())NextChild, 0);
> + __sanitizer_start_switch_fiber(next_child_context.uc_stack.ss_sp,
> + next_child_context.uc_stack.ss_size);
> + CallNoReturn();
> + if (swapcontext(&child_context, &next_child_context) < 0) {
> + perror("swapcontext");
> + _exit(1);
> + }
> + }
> +}
> +
> +int Run(int arg, int mode, char *child_stack) {
> + printf("Child stack: %p\n", child_stack);
> + // Setup child context.
> + getcontext(&child_context);
> + child_context.uc_stack.ss_sp = child_stack;
> + child_context.uc_stack.ss_size = kStackSize / 2;
> + if (mode == 0) {
> + child_context.uc_link = &orig_context;
> + }
> + makecontext(&child_context, (void (*)())Child, 1, mode);
> + CallNoReturn();
> + __sanitizer_start_switch_fiber(child_context.uc_stack.ss_sp,
> + child_context.uc_stack.ss_size);
> + CallNoReturn();
> + if (swapcontext(&orig_context, &child_context) < 0) {
> + perror("swapcontext");
> + _exit(1);
> + }
> + CallNoReturn();
> + __sanitizer_finish_switch_fiber();
> + CallNoReturn();
> +
> + // Touch childs's stack to make sure it's unpoisoned.
> + for (int i = 0; i < kStackSize; i++) {
> + child_stack[i] = i;
> + }
> + return child_stack[arg];
> +}
> +
> +void handler(int sig) { CallNoReturn(); }
> +
> +void InitStackBounds() {
> + pthread_attr_t attr;
> + pthread_attr_init(&attr);
> + pthread_getattr_np(pthread_self(), &attr);
> + pthread_attr_getstack(&attr, &main_thread_stack,
> &main_thread_stacksize);
> + pthread_attr_destroy(&attr);
> +}
> +
> +int main(int argc, char **argv) {
> + InitStackBounds();
> +
> + // set up a signal that will spam and trigger __asan_handle_no_return at
> + // tricky moments
> + struct sigaction act = {};
> + act.sa_handler = &handler;
> + if (sigaction(SIGPROF, &act, 0)) {
> + perror("sigaction");
> + _exit(1);
> + }
> +
> + itimerval t;
> + t.it_interval.tv_sec = 0;
> + t.it_interval.tv_usec = 10;
> + t.it_value = t.it_interval;
> + if (setitimer(ITIMER_PROF, &t, 0)) {
> + perror("setitimer");
> + _exit(1);
> + }
> +
> + char *heap = new char[kStackSize + 1];
> + next_child_stack = new char[kStackSize + 1];
> + char stack[kStackSize + 1];
> + // CHECK: WARNING: ASan doesn't fully support makecontext/swapcontext
> + int ret = 0;
> + // CHECK-NOT: ASan is ignoring requested __asan_handle_no_return
> + for (unsigned int i = 0; i < 30; ++i) {
> + ret += Run(argc - 1, 0, stack);
> + ret += Run(argc - 1, 1, stack);
> + ret += Run(argc - 1, 2, stack);
> + ret += Run(argc - 1, 0, heap);
> + ret += Run(argc - 1, 1, heap);
> + ret += Run(argc - 1, 2, heap);
> + }
> + // CHECK: Test passed
> + printf("Test passed\n");
> +
> + delete[] heap;
> + delete[] next_child_stack;
> +
> + return ret;
> +}
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
--
Mike
Sent from phone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160621/3f6f1169/attachment.html>
More information about the llvm-commits
mailing list