[LLVMdev] multithreaded use of llvm::sys::RemoveFileOnSignal

[Vikas Bhargava]

Yes, there is a crash when lot of threads are used. I am using 60 threads
to compile 60 source files and the crash happens 90% of the time. Here is
the relevant stack trace:

(gdb) bt
#0  0x00007f30e11d9e4e in __libc_sigaction (sig=31, act=<optimized out>,
oact=0x2c5ffa0)
    at ../sysdeps/unix/sysv/linux/x86_64/sigaction.c:71
#1  0x0000000001c75636 in RegisterHandlers() ()
#2  0x0000000001c7579f in llvm::sys::RemoveFileOnSignal(llvm::StringRef,
std::__1::basic_string<char, std::__1::char_traits<char>,
std::__1::allocator<char> >*) ()
#3  0x0000000000ed3286 in
clang::CompilerInstance::createOutputFile(llvm::StringRef,
std::__1::basic_string<char, std::__1::char_traits<char>,
std::__1::allocator<char> >&, bool, bool, llvm::StringRef, llvm::StringRef,
bool, bool, std::__1::basic_string<char, std::__1::char_traits<char>,
std::__1::allocator<char> >*, std::__1::basic_string<char,
std::__1::char_traits<char>, std::__1::allocator<char> >*) ()
#4  0x0000000000ed20c8 in
clang::CompilerInstance::createOutputFile(llvm::StringRef, bool, bool,
llvm::StringRef, llvm::StringRef, bool, bool) ()
#5  0x0000000000ed200b in
clang::CompilerInstance::createDefaultOutputFile(bool, llvm::StringRef,
llvm::StringRef) ()
#6  0x0000000000d166b9 in
clang::CodeGenAction::CreateASTConsumer(clang::CompilerInstance&,
llvm::StringRef) ()
#7  0x0000000000ef87a4 in
clang::FrontendAction::CreateWrappedASTConsumer(clang::CompilerInstance&,
llvm::StringRef) ()
#8  0x0000000000ef943e in
clang::FrontendAction::BeginSourceFile(clang::CompilerInstance&,
clang::FrontendInputFile const&) ()
#9  0x0000000000ed3e01 in
clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) ()
#10 0x000000000140dad9 in
clang::tooling::FrontendActionFactory::runInvocation(clang::CompilerInvocation*,
clang::FileManager*, clang::DiagnosticConsumer*) ()
#11 0x000000000140d423 in clang::tooling::ToolInvocation::run() ()


Please let me know if you want me to create a working project to illustrate
the crash.

thx
Vikas.
=========


On Thu, Feb 27, 2014 at 7:31 AM, Argyrios Kyrtzidis <kyrtzidis at apple.com>wrote:

>
> On Feb 27, 2014, at 3:23 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
>
> On Thu, Feb 27, 2014 at 12:50 AM, Vikas Bhargava
> <vikasbhargava at gmail.com> wrote:
>
> Hi,
> I am using clang::ToolInvocation class to compile some code in-memory:
>
>   clang::tooling::ToolInvocation ti
>      (
>       compilerArgs,
>       new clang::EmitBCAction(),
>       new clang::FileManager(clang::FileSystemOptions())
>      );
>   //filename is the name of the source file, e.g. "Somefile.cpp"
>   //sourcecode contains the source of the file
>   ti.mapVirtualFile( filename, sourcecode );
>   bool ret = ti.run();
>
> In order to speed up compilation of several sources, I call the above code
> concurrently in separate threads, with each thread compiling its own source
> code. However, this does not work because clang::CompilerInstance calls
> llvm::Sys::RemoveFileOnSignal which in turn calls RegisterHandlers() which
> is not re-entrant.
>
> Is there a way to make RegisterHandlers() re-entrant? I do call
> llvm::start_multi_threaded() before any of this, hoping that it will make
> llvm routines thread-safe, but to no avail.
>
>
> Seems like it is a bug in either RemoveFileOnSignal().  For some
> reason, the first one releases the mutex before calling the
> RegisterHandlers().
>
> bool llvm::sys::RemoveFileOnSignal(StringRef Filename,
>                                   std::string* ErrMsg) {
>  SignalsMutex.acquire();
> ...
>  SignalsMutex.release();
>
>  RegisterHandlers();
>  return false;
> }
>
> CC'ing Argyrios because this is important for libclang.
>
> Argyrios: what do you think -- should we call RegisterHandlers() under a
> mutex?
>
>
> That, or get RegisterHandlers to use the mutex.
> Also access to CallBacksToRun is not thread-safe (the whole file probably
> needs reviewing..)
>
> But it's not so clear to me what is the exact symptom, is there a crash
> occurring ? Vika, could you explain ?
>
>
> Dmitri
>
> --
> main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
> (j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at gmail.com>*/
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140227/4360204b/attachment.html>