[PATCH] [TSan][MIPS] Adding support for MIPS64

[Alexey Samsonov]

================
Comment at: lib/tsan/rtl/tsan_interceptors.cc:661
@@ +660,3 @@
+#ifdef __mips__
+  // For mips memset gets called before a call to __cxa_guard_aquire
+  // which causes a crash in REAL(pthread_create) during initialization
----------------
sagar wrote:
> samsonov wrote:
> > sagar wrote:
> > > samsonov wrote:
> > > > sagar wrote:
> > > > > samsonov wrote:
> > > > > > sagar wrote:
> > > > > > > samsonov wrote:
> > > > > > > > I don't understand this comment. Why does REAL(pthread_create) crash?
> > > > > > > As per my observation, in x86_64 first __cxa_guard_aquired() is called and its interceptor calls Initialize().
> > > > > > > Initialize() creates background thread.
> > > > > > > But in mips64 __interceptor_memset gets called before __cxa_guard_aquire() interceptor. So Initialize() is called from within __intercerptor_memset and REAL(pthread_create) emits a SIGIOT.
> > > > > > > 
> > > > > > > @dvyukov I don't know why such behaviour in mips, can you help us with this?
> > > > > > Why does the call to REAL(pthread_create) succeeds if it is issued from `__cxa_guard_acquired` interceptor and fails if it is issued from `memset` interceptor? I think you'd have to figure it out.
> > > > > > 
> > > > > > I don't see the immediate problem from your description: memset interceptor calls Initialize(), which calls InitializeIntercepotrs() (that is supposed to initialize all REAL(foo) function pointers), and only then calls internal_start_thread.
> > > > > While debugging the code, I observed that __interceptor_memset and __cxa_guard_acquire are being called even before __start. So its reference is probably in LLVM/clang. I have no knowledge from where they are being called.
> > > > > 
> > > > > Through strace in x86_64, the sequence of calls observed:
> > > > >   rt_sigaction() -> rt_sigprocmask() -> __cxa_guard_acquired() -> Initialize()
> > > > > 
> > > > > In case of mips64, the sequence is:
> > > > >   __interceptor_memset() -> Initialize()
> > > > > 
> > > > > So for mips64 there is no signal handler registered during Initialize() because rt_sigaction syscall is not yet issued.
> > > > > 
> > > > > I will be very thankful if someone could advice me on how to solve this issue.
> > > > It's hard to predict which TSan interceptor will be called first (and thus will be first to call __tsan::Initialize()). For example, on my x86_64 host the stack trace is:
> > > >   __tsan::Initialize
> > > >   ScopedInterceptor::ScopedInterceptor
> > > >   __cxa_guard_acquire
> > > >   __future_category_instance
> > > >   __static_initialization_and_destruction_0
> > > >   _GLOBAL__sub_I_compatibility_thread_c__0x.cc
> > > >   call_init
> > > >   _dl_start_user
> > > > 
> > > > I don't see why signal handler is at all important. Can you just debug your TSan-instrumented program in gdb, reach the place where it fails with SIGIOT/SIGABRT and learn the reason why it's happening? E.g. it calls REAL(foo), but this function pointer is not yet initialized. Or it accesses mprotect()-ed memory. Or something else.
> > > > 
> > > > Sorry, I can't yet diagnose the issue from your descriptions.
> > > Hi,
> > > 
> > > The intercepted memset is called during pthread initialization function __pthread_initialize_minimal_internal ().
> > > 
> > > Sequence of calls for MIPS:
> > > 
> > > ```
> > >  __pthread_initialize_minimal_internal () -> __sigemptyset () -> __interceptor_memset () -> pthread_create () -> __GI_abort ()
> > > ```
> > > 
> > > The intercepted memset calls pthread_create() before the minimal initialization required for pthread_create is complete and this results in abort.
> > > On x86, __pthread_initialize_minimal_internal () does not call memset and thus can complete the minimal initialization.
> > > For MIPS, we need to avoid calling pthread_create () if memset is called during pthread initialization.
> > So, this has nothing to do with `__cxa_guard_acquired` or `memset`. You are just not allowed to call pthread_create() from TSan runtime before `__pthread_initialize_minimal_internal` exits. For example, you can try to intercept `__pthread_initialize_minimal_internal` and use `ScopedIgnoreInterceptors` to disable TSan interceptors for sigemptyset(), memset() or whatever.
> Hi,
> 
> I tried to intercept `__pthread_initialize_minimal_internal ()`, but it does not get intercepted. As far as I know, this is probably because references to `__pthread_initialize_minimal_internal ()` are resolved when glibc is compiled. I also tried to search for calls to functions occurring before call to `__pthread_initialize_minimal_internal ()` which would be resolved at runtime, so that we could intercept them and use `ScopedIgnoreInterceptors`, but I couldn’t see any such function call.
Who calls `__pthread_initialize_minimal_internal` at all in that case? Is it called from library constructor? As a quick workaround, you can disable background thread in MIPS port - IIRC it's not critical to TSan functioning.

http://reviews.llvm.org/D6291

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/