<div dir="ltr">So, the issue is that std::mutex and std::recursive_mutex simply cannot be used during an atexit() handler. It still seems like a good drop-in replacement for the hand-rolled mutex in all other cases. <div>
<br></div><div>This is not documented anywhere, but if you dig into the code you'll find that std::mutex and std::recursive_mutex are implemented using ConcRT (microsoft concurrency runtime), and it relies on a call to RegisterWaitForSingleObject(). That function in turn uses QueueUserAPC, and if you check the documentation you'll see that it says "When the thread is in the process of being terminated, calling QueueUserAPC to the thread's APC queue will fail with ERROR_GEN_FAILURE (31)".</div>
<div><br></div><div>So, TL;DR - can't use mutexes from atexit handler. This seems like a reasonable restriction, as atexit handlers are probably pretty rare anyway.</div><div><br></div><div>I will try to find a way to change the shutdown order to not rely on the atexit handler.<br>
<div><br></div><div><br><div><p style="margin:0px 0px 1em;padding:0px;border:0px;font-size:14px;vertical-align:baseline;clear:both;color:rgb(0,0,0);font-family:Arial,'Liberation Sans','DejaVu Sans',sans-serif;line-height:17.8048000335693px;background:transparent">
<span style="font-family:arial;font-size:small;line-height:normal;color:rgb(34,34,34)">On Thu, May 29, 2014 at 4:25 PM, Zachary Turner </span><span dir="ltr" style="font-family:arial;font-size:small;line-height:normal;color:rgb(34,34,34)"><<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>></span><span style="font-family:arial;font-size:small;line-height:normal;color:rgb(34,34,34)"> wrote:</span><span style="color:rgb(34,34,34);font-family:arial;font-size:small;line-height:normal;background-color:transparent"><br>
</span></p></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">It looks like the issue is that we're trying to acquire the mutex during process shutdown. Specifically, here's a partial callstack:<div>
<br></div><div><div> <span style="white-space:pre-wrap"> </span>libclang.dll!mtx_do_lock(_Mtx_internal_imp_t * * mtx=0x01b118e4, const xtime * target=0x00000000) Line 67<span style="white-space:pre-wrap"> </span>C++</div>
<div> <span style="white-space:pre-wrap"> </span>libclang.dll!_Mtx_lock(_Mtx_internal_imp_t * * mtx=0x01b118e4) Line 153<span style="white-space:pre-wrap"> </span>C++</div><div> <span style="white-space:pre-wrap"> </span>libclang.dll!std::_Mtx_lockX(_Mtx_internal_imp_t * * _Mtx=0x01b118e4) Line 68<span style="white-space:pre-wrap"> </span>C++</div>
<div> <span style="white-space:pre-wrap"> </span>libclang.dll!std::_Mutex_base::lock() Line 41<span style="white-space:pre-wrap"> </span>C++</div><div> <span style="white-space:pre-wrap"> </span>libclang.dll!llvm::sys::Mutex<0,1>::acquire() Line 94<span style="white-space:pre-wrap"> </span>C++</div>
<div> <span style="white-space:pre-wrap"> </span>libclang.dll!llvm::MutexGuard::MutexGuard(llvm::sys::MutexBase & m={...}) Line 33<span style="white-space:pre-wrap"> </span>C++</div><div>><span style="white-space:pre-wrap"> </span>libclang.dll!cleanupOnDiskMapAtExit() Line 124<span style="white-space:pre-wrap"> </span>C++</div>
<div> <span style="white-space:pre-wrap"> </span>libclang.dll!doexit(int code=0, int quick=0, int retcaller=1) Line 628<span style="white-space:pre-wrap"> </span>C</div><div> <span style="white-space:pre-wrap"> </span>libclang.dll!_cexit() Line 448<span style="white-space:pre-wrap"> </span>C</div>
<div> <span style="white-space:pre-wrap"> </span>libclang.dll!_CRT_INIT(void * hDllHandle=0x0f360000, unsigned long dwReason=0, void * lpreserved=0x00000001) Line 169<span style="white-space:pre-wrap"> </span>C</div>
<div> <span style="white-space:pre-wrap"> </span>libclang.dll!__DllMainCRTStartup(void * hDllHandle=0x0f360000, unsigned long dwReason=0, void * lpreserved=0x00000001) Line 399<span style="white-space:pre-wrap"> </span>C</div>
<div> <span style="white-space:pre-wrap"> </span>libclang.dll!_DllMainCRTStartup(void * hDllHandle=0x0f360000, unsigned long dwReason=0, void * lpreserved=0x00000001) Line 340<span style="white-space:pre-wrap"> </span>C</div>
<div> <span style="white-space:pre-wrap"> </span>ntdll.dll!_LdrxCallInitRoutine@16 ()<span style="white-space:pre-wrap"> </span>Unknown</div><div> <span style="white-space:pre-wrap"> </span>ntdll.dll!LdrpCallInitRoutine()<span style="white-space:pre-wrap"> </span>Unknown</div>
<div> <span style="white-space:pre-wrap"> </span>ntdll.dll!LdrShutdownProcess()<span style="white-space:pre-wrap"> </span>Unknown</div><div> <span style="white-space:pre-wrap"> </span>ntdll.dll!RtlExitUserProcess()<span style="white-space:pre-wrap"> </span>Unknown</div>
<div><br></div><div><br></div><div>And this is the mutex that is hung. If I go debug inside the CRT, the owning threadid is set to -1, so I'm not sure. Maybe something's been uninitialized by this point. I thought of using ManagedStatic<> to better control the uninitialization of the mutex, but I don't think that really solves the problem. The issue is that we're even relying on a mutex at all during CRT shutdown. </div>
<div><br></div></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 29, 2014 at 2:44 PM, Aaron Ballman <span dir="ltr"><<a href="mailto:aaron@aaronballman.com" target="_blank">aaron@aaronballman.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On Thu, May 29, 2014 at 5:41 PM, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
> What command are you using to run the tests? I'm still a new contributor,<br>
> and am not familiar with how to do everything, but I run ninja check-all and<br>
> while it doesn't seem to be finishing, I am not getting any output. How can<br>
> I get a command line of a single test that I can easily run independently on<br>
> the command line to reproduce a deadlock so I can look into this further?<br>
<br>
</div>I was running my tests from within MSVC by building the check-clang<br>
project; output pipes automatically into the output window as the<br>
tests run. You can use python and llvm-lit to run specific tests from<br>
the command line. Eg) python.exe ./bin/llvm-lit.py -v<br>
../tools/clang/test/Sema/warn-thread-safety-analysis.c<br>
<br>
HTH!<br>
<span><font color="#888888"><br>
~Aaron<br>
</font></span></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div></div>