[PATCH] [TSan][MIPS] Adding support for MIPS64

[Alexey Samsonov]

================
Comment at: lib/tsan/rtl/tsan_interceptors.cc:661
@@ +660,3 @@
+#ifdef __mips__
+  // For mips memset gets called before a call to __cxa_guard_aquire
+  // which causes a crash in REAL(pthread_create) during initialization
----------------
sagar wrote:
> samsonov wrote:
> > sagar wrote:
> > > samsonov wrote:
> > > > sagar wrote:
> > > > > samsonov wrote:
> > > > > > I don't understand this comment. Why does REAL(pthread_create) crash?
> > > > > As per my observation, in x86_64 first __cxa_guard_aquired() is called and its interceptor calls Initialize().
> > > > > Initialize() creates background thread.
> > > > > But in mips64 __interceptor_memset gets called before __cxa_guard_aquire() interceptor. So Initialize() is called from within __intercerptor_memset and REAL(pthread_create) emits a SIGIOT.
> > > > > 
> > > > > @dvyukov I don't know why such behaviour in mips, can you help us with this?
> > > > Why does the call to REAL(pthread_create) succeeds if it is issued from `__cxa_guard_acquired` interceptor and fails if it is issued from `memset` interceptor? I think you'd have to figure it out.
> > > > 
> > > > I don't see the immediate problem from your description: memset interceptor calls Initialize(), which calls InitializeIntercepotrs() (that is supposed to initialize all REAL(foo) function pointers), and only then calls internal_start_thread.
> > > While debugging the code, I observed that __interceptor_memset and __cxa_guard_acquire are being called even before __start. So its reference is probably in LLVM/clang. I have no knowledge from where they are being called.
> > > 
> > > Through strace in x86_64, the sequence of calls observed:
> > >   rt_sigaction() -> rt_sigprocmask() -> __cxa_guard_acquired() -> Initialize()
> > > 
> > > In case of mips64, the sequence is:
> > >   __interceptor_memset() -> Initialize()
> > > 
> > > So for mips64 there is no signal handler registered during Initialize() because rt_sigaction syscall is not yet issued.
> > > 
> > > I will be very thankful if someone could advice me on how to solve this issue.
> > It's hard to predict which TSan interceptor will be called first (and thus will be first to call __tsan::Initialize()). For example, on my x86_64 host the stack trace is:
> >   __tsan::Initialize
> >   ScopedInterceptor::ScopedInterceptor
> >   __cxa_guard_acquire
> >   __future_category_instance
> >   __static_initialization_and_destruction_0
> >   _GLOBAL__sub_I_compatibility_thread_c__0x.cc
> >   call_init
> >   _dl_start_user
> > 
> > I don't see why signal handler is at all important. Can you just debug your TSan-instrumented program in gdb, reach the place where it fails with SIGIOT/SIGABRT and learn the reason why it's happening? E.g. it calls REAL(foo), but this function pointer is not yet initialized. Or it accesses mprotect()-ed memory. Or something else.
> > 
> > Sorry, I can't yet diagnose the issue from your descriptions.
> Hi,
> 
> The intercepted memset is called during pthread initialization function __pthread_initialize_minimal_internal ().
> 
> Sequence of calls for MIPS:
> 
> ```
>  __pthread_initialize_minimal_internal () -> __sigemptyset () -> __interceptor_memset () -> pthread_create () -> __GI_abort ()
> ```
> 
> The intercepted memset calls pthread_create() before the minimal initialization required for pthread_create is complete and this results in abort.
> On x86, __pthread_initialize_minimal_internal () does not call memset and thus can complete the minimal initialization.
> For MIPS, we need to avoid calling pthread_create () if memset is called during pthread initialization.
So, this has nothing to do with `__cxa_guard_acquired` or `memset`. You are just not allowed to call pthread_create() from TSan runtime before `__pthread_initialize_minimal_internal` exits. For example, you can try to intercept `__pthread_initialize_minimal_internal` and use `ScopedIgnoreInterceptors` to disable TSan interceptors for sigemptyset(), memset() or whatever.

http://reviews.llvm.org/D6291

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/