[cfe-dev] Locking mechanism for preventing concurrent reading & writing of the same PCH file

[Argyrios Kyrtzidis]

FYI, we will not use this, it was the second worst idea in the history of the world (the first one was chopsticks, gah!)

-Argiris

On Sep 15, 2010, at 3:41 PM, Argyrios Kyrtzidis wrote:

> There is an issue of clang crashing if there is concurrent reading & writing of the same PCH file (see rdar://8392711&8294781).
> This should probably be avoided by the client of clang in the first place, but nevertheless we would like to have clang being more defensive and prevent the crashes.
> 
> I was able to verify the issue by continuously spawning processes that either read or write the same PCH file, I saw crashes & failures like these:
> 
> ........
> DO: clang++ -cc1 t.cpp -include-pch t.ast
> error: malformed block record in PCH file: 't.ast'
> DO: clang++ -cc1 t.cpp -include-pch t.ast
> DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
> 0  clang++           0x000000010107c242 PrintStackTrace(void*) + 34
> 1  clang++           0x000000010107cd63 SignalHandler(int) + 531
> 2  libSystem.B.dylib 0x00007fff8077535a _sigtramp + 26
> 3  libSystem.B.dylib 0x0000000000000001 _sigtramp + 2139663553
> 4  clang++           0x0000000100050b94 ParsePreprocessorArgs(clang::PreprocessorOptions&, clang::driver::ArgList&, clang::Diagnostic&) + 1828
> 5  clang++           0x000000010005450c clang::CompilerInvocation::CreateFromArgs(clang::CompilerInvocation&, char const**, char const**, clang::Diagnostic&) + 3052
> 6  clang++           0x0000000100015103 cc1_main(char const**, char const**, char const*, void*) + 355
> 7  clang++           0x000000010001b493 main + 4691
> 8  clang++           0x0000000100013c38 start + 52
> Stack dump:
> 0.	Program arguments: /Users/argiris/proj/llvm/Release/bin/clang++ -cc1 t.cpp -include-pch t.ast
> ........
> 
> ........
> DO: clang++ -cc1 t.cpp -include-pch t.ast
> error: malformed block record in PCH file: 't.ast'
> DO: clang++ -cc1 t.cpp -include-pch t.ast
> DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
> 0  clang++           0x000000010107c242 PrintStackTrace(void*) + 34
> 1  clang++           0x000000010107cd63 SignalHandler(int) + 531
> 2  libSystem.B.dylib 0x00007fff8077535a _sigtramp + 26
> 3  libSystem.B.dylib 0x0000000102010f90 _sigtramp + 2173287504
> 4  clang++           0x00000001000dde2f clang::ASTReader::ReadAST(std::string const&) + 47
> 5  clang++           0x00000001000407f4 clang::CompilerInstance::createPCHExternalASTSource(llvm::StringRef, std::string const&, bool, clang::Preprocessor&, clang::ASTContext&, void*) + 164
> 6  clang++           0x0000000100040904 clang::CompilerInstance::createPCHExternalASTSource(llvm::StringRef, bool, void*) + 68
> 7  clang++           0x0000000100063973 clang::FrontendAction::BeginSourceFile(clang::CompilerInstance&, llvm::StringRef, clang::InputKind) + 1187
> 8  clang++           0x00000001000420bc clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 268
> 9  clang++           0x000000010001c747 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 1287
> 10 clang++           0x0000000100015164 cc1_main(char const**, char const**, char const*, void*) + 452
> 11 clang++           0x000000010001b493 main + 4691
> 12 clang++           0x0000000100013c38 start + 52
> 13 clang++           0x0000000000000005 start + 4294886401
> Stack dump:
> 0.	Program arguments: /Users/argiris/proj/llvm/Release/bin/clang++ -cc1 t.cpp -include-pch t.ast 
> ........
> 
> 
> In order to prevent the crashes I propose using flock (http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/flock.2.html) as file locking mechanism.
> The attached patches introduce the FileLock class in llvm::sys namespace which gets used for locking a PCH file.<filelock_llvm.diff><filelock_clang.diff>
> 
> Multiple concurrent reads are allowed but writing will block until the reads (or another write) release their locks, while on the other hand any read will block until a write is finished and releases its lock.
> 
> Apart from preventing crashes we also prevent failures since after the block is lifted we can merrily continue reading/writing.
> 
> After the patches are applied, when I try the multiple-spawned-processes test, the read/writes happen sequentially without a crash or failure:
> 
> ........
> DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
> DO: clang++ -cc1 t.cpp -include-pch t.ast
> DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
> DO: clang++ -cc1 t.cpp -include-pch t.ast
> ........
> 
> 
> Please review.
> 
> -Argiris_______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev