[cfe-dev] Locking mechanism for preventing concurrent reading & writing of the same PCH file

[Argyrios Kyrtzidis]

There is an issue of clang crashing if there is concurrent reading & writing of the same PCH file (see rdar://8392711&8294781).
This should probably be avoided by the client of clang in the first place, but nevertheless we would like to have clang being more defensive and prevent the crashes.

I was able to verify the issue by continuously spawning processes that either read or write the same PCH file, I saw crashes & failures like these:

........
DO: clang++ -cc1 t.cpp -include-pch t.ast
error: malformed block record in PCH file: 't.ast'
DO: clang++ -cc1 t.cpp -include-pch t.ast
DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
0  clang++           0x000000010107c242 PrintStackTrace(void*) + 34
1  clang++           0x000000010107cd63 SignalHandler(int) + 531
2  libSystem.B.dylib 0x00007fff8077535a _sigtramp + 26
3  libSystem.B.dylib 0x0000000000000001 _sigtramp + 2139663553
4  clang++           0x0000000100050b94 ParsePreprocessorArgs(clang::PreprocessorOptions&, clang::driver::ArgList&, clang::Diagnostic&) + 1828
5  clang++           0x000000010005450c clang::CompilerInvocation::CreateFromArgs(clang::CompilerInvocation&, char const**, char const**, clang::Diagnostic&) + 3052
6  clang++           0x0000000100015103 cc1_main(char const**, char const**, char const*, void*) + 355
7  clang++           0x000000010001b493 main + 4691
8  clang++           0x0000000100013c38 start + 52
Stack dump:
0.	Program arguments: /Users/argiris/proj/llvm/Release/bin/clang++ -cc1 t.cpp -include-pch t.ast
........

........
DO: clang++ -cc1 t.cpp -include-pch t.ast
error: malformed block record in PCH file: 't.ast'
DO: clang++ -cc1 t.cpp -include-pch t.ast
DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
0  clang++           0x000000010107c242 PrintStackTrace(void*) + 34
1  clang++           0x000000010107cd63 SignalHandler(int) + 531
2  libSystem.B.dylib 0x00007fff8077535a _sigtramp + 26
3  libSystem.B.dylib 0x0000000102010f90 _sigtramp + 2173287504
4  clang++           0x00000001000dde2f clang::ASTReader::ReadAST(std::string const&) + 47
5  clang++           0x00000001000407f4 clang::CompilerInstance::createPCHExternalASTSource(llvm::StringRef, std::string const&, bool, clang::Preprocessor&, clang::ASTContext&, void*) + 164
6  clang++           0x0000000100040904 clang::CompilerInstance::createPCHExternalASTSource(llvm::StringRef, bool, void*) + 68
7  clang++           0x0000000100063973 clang::FrontendAction::BeginSourceFile(clang::CompilerInstance&, llvm::StringRef, clang::InputKind) + 1187
8  clang++           0x00000001000420bc clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 268
9  clang++           0x000000010001c747 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 1287
10 clang++           0x0000000100015164 cc1_main(char const**, char const**, char const*, void*) + 452
11 clang++           0x000000010001b493 main + 4691
12 clang++           0x0000000100013c38 start + 52
13 clang++           0x0000000000000005 start + 4294886401
Stack dump:
0.	Program arguments: /Users/argiris/proj/llvm/Release/bin/clang++ -cc1 t.cpp -include-pch t.ast 
........


In order to prevent the crashes I propose using flock (http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/flock.2.html) as file locking mechanism.
The attached patches introduce the FileLock class in llvm::sys namespace which gets used for locking a PCH file.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: filelock_llvm.diff
Type: application/octet-stream
Size: 8878 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20100915/b0542ba4/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: filelock_clang.diff
Type: application/octet-stream
Size: 15352 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20100915/b0542ba4/attachment-0001.obj>
-------------- next part --------------


Multiple concurrent reads are allowed but writing will block until the reads (or another write) release their locks, while on the other hand any read will block until a write is finished and releases its lock.

Apart from preventing crashes we also prevent failures since after the block is lifted we can merrily continue reading/writing.

After the patches are applied, when I try the multiple-spawned-processes test, the read/writes happen sequentially without a crash or failure:

........
DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
DO: clang++ -cc1 t.cpp -include-pch t.ast
DO: clang++ -cc1 th.cpp -emit-pch -o t.ast
DO: clang++ -cc1 t.cpp -include-pch t.ast
........


Please review.

-Argiris