[cfe-dev] [windows libclang] libclang fails to delete obsolete temporary .pch files
Reid Kleckner
rnk at google.com
Fri Dec 20 10:43:51 PST 2013
Ultimately this boils down to the fact that you can unlink files you have
open on non-Windows platforms and the fd will keep it alive. Maybe we
should use FILE_SHARE_DELETE in llvm::sys::fs::openFileForRead(). Will
that make our behavior consistent across platforms?
On Fri, Dec 20, 2013 at 2:27 AM, William Ledoux <william.ledoux at gmail.com>wrote:
> Hi,
>
> I use clang_reparseTranslationUnit with CXTranslationUnit_PrecompiledPreamble
> flag on windows.
>
> I investigated why libclang.dll fails to delete old pch files on windows.
>
> Modules keep an active file mapping on file until they are destroyed.
> Later on, this still active mapping will cause DeleteFileW to fail
> (silently..) .
>
>
> There is at least two cases when it happens:
>
> The first one is when some preamble has changed and a new pch needs to
> replace the current one, CleanPreambleFile , which purpose is to destroy
> the now old pch file will be called *before *the corresponding module is
> destroyed when the mapping will be recycled in ASTUnit::Parse.
>
> The second one is when we destroy the translation unit with clang_
> disposeTranslationUnit, CleanPreambleFile which purpose is to destroy the
> pch file, is called *before* the corresponding module is destroyed in ~
> ModuleFile.
>
> Below are the corresponding callstacks.
>
>
> 1. libclang.dll!`anonymous namespace'::OnDiskData::
> *CleanPreambleFile*() Line 175 C++
> 2. libclang.dll!erasePreambleFile(const clang::ASTUnit * AU) Line
> 144 C++
> 3.
> libclang.dll!clang::ASTUnit::getMainBufferWithPrecompiledPreamble(const
> clang::CompilerInvocation & PreambleInvocationIn, bool AllowRebuild,
> unsigned int MaxLines) Line 1511 C++
> 4.
> libclang.dll!clang::ASTUnit::Reparse(std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char>
> >,llvm::PointerUnion<char const *,llvm::MemoryBuffer const *> > *
> RemappedFiles, unsigned int NumRemappedFiles) Line 2116 C++
> 5. libclang.dll!clang_reparseTranslationUnit_Impl(void * UserData)
> Line 3008 C++
>
>
> 1.
> libclang.dll!llvm::sys::fs::mapped_file_region::~mapped_file_region() Line
> 833 C++
> 2. libclang.dll!`anonymous
> namespace'::MemoryBufferMMapFile::~MemoryBufferMMapFile() C++
> 3. libclang.dll!`anonymous namespace'::MemoryBufferMMapFile::`scalar
> deleting destructor'(unsigned int) C++
> 4.
> libclang.dll!llvm::OwningPtr<llvm::MemoryBuffer>::~OwningPtr<llvm::MemoryBuffer>()
> Line 45 C++
> 5. libclang.dll!clang::serialization::ModuleFile::~ModuleFile() Line
> 58 C++
> 6. libclang.dll!clang::serialization::ModuleFile::`scalar deleting
> destructor'(unsigned int) C++
> 7.
> libclang.dll!clang::serialization::ModuleManager::~ModuleManager() Line 230
> C++
> 8. libclang.dll!clang::ASTReader::~ASTReader() Line 7638 C++
> 9. libclang.dll!clang::ASTReader::`scalar deleting
> destructor'(unsigned int) C++
> 10.
> libclang.dll!llvm::OwningPtr<clang::ExternalASTSource>::~OwningPtr<clang::ExternalASTSource>()
> Line 45 C++
> 11. libclang.dll!clang::ASTContext::~ASTContext() Line 790 C++
> 12. libclang.dll!clang::ASTContext::`scalar deleting
> destructor'(unsigned int) C++
> 13. libclang.dll!llvm::RefCountedBase<clang::ASTContext>::Release()
> Line 54 C++
> 14.
> libclang.dll!llvm::IntrusiveRefCntPtrInfo<clang::ASTContext>::release(clang::ASTContext
> * obj) Line 89 C++
> 15.
> libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::release() Line
> 178 C++
> 16.
> libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::~IntrusiveRefCntPtr<clang::ASTContext>()
> Line 148 C++
> 17.
> libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::operator=(llvm::IntrusiveRefCntPtr<clang::ASTContext>
> S) Line 145 C++
> 18. libclang.dll!clang::ASTUnit::Parse(llvm::MemoryBuffer *
> OverrideMainBuffer) Line 1169 C++
> 19.
> libclang.dll!clang::ASTUnit::Reparse(std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char>
> >,llvm::PointerUnion<char const *,llvm::MemoryBuffer const *> > *
> RemappedFiles, unsigned int NumRemappedFiles) Line 2125 C++
> 20. libclang.dll!clang_reparseTranslationUnit_Impl(void * UserData)
> Line 3008 C++
>
>
> 1. libclang.dll!`anonymous namespace'::OnDiskData::
> *CleanPreambleFile*() Line 175 C++
> 2. libclang.dll!`anonymous namespace'::OnDiskData::Cleanup() Line 183
> C++
> 3. libclang.dll!removeOnDiskEntry(const clang::ASTUnit * AU) Line 154
> C++
> 4. libclang.dll!clang::ASTUnit::~ASTUnit() Line 249 C++
> 5. libclang.dll!clang::ASTUnit::`scalar deleting
> destructor'(unsigned int) C++
> 6. libclang.dll!clang_disposeTranslationUnit(CXTranslationUnitImpl *
> CTUnit) Line 2945 C++
>
>
> 1. libclang.dll!llvm::sys::fs::mapped_file_region::~mapped_file_region()
> Line 833 C++
> 2. libclang.dll!`anonymous namespace'::MemoryBufferMMapFile::~MemoryBufferMMapFile()
> C++
> 3. libclang.dll!`anonymous namespace'::MemoryBufferMMapFile::`scalar
> deleting destructor'(unsigned int) C++
> 4. libclang.dll!llvm::OwningPtr<llvm::MemoryBuffer>::~
> OwningPtr<llvm::MemoryBuffer>() Line 45 C++
> 5. libclang.dll!clang::serialization::ModuleFile::~ModuleFile() Line
> 58 C++
> 6. libclang.dll!clang::serialization::ModuleFile::`scalar deleting
> destructor'(unsigned int) C++
> 7. libclang.dll!clang::serialization::ModuleManager::~ModuleManager()
> Line 230 C++
> 8. libclang.dll!clang::ASTReader::~ASTReader() Line 7638 C++
> 9. libclang.dll!clang::ASTReader::`scalar deleting
> destructor'(unsigned int) C++
> 10. libclang.dll!llvm::OwningPtr<clang::ExternalASTSource>::~
> OwningPtr<clang::ExternalASTSource>() Line 45 C++
> 11. libclang.dll!clang::ASTContext::~ASTContext() Line 790 C++
> 12. libclang.dll!clang::ASTContext::`scalar deleting
> destructor'(unsigned int) C++
> 13. libclang.dll!llvm::RefCountedBase<clang::ASTContext>::Release()
> Line 54 C++
> 14. libclang.dll!llvm::IntrusiveRefCntPtrInfo<clang::
> ASTContext>::release(clang::ASTContext * obj) Line 89 C++
> 15. libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::release()
> Line 178 C++
> 16. libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::~
> IntrusiveRefCntPtr<clang::ASTContext>() Line 148 C++
> 17. libclang.dll!clang::ASTUnit::~ASTUnit() Line 274 C++
> 18. libclang.dll!clang::ASTUnit::`scalar deleting
> destructor'(unsigned int) C++
> 19. libclang.dll!clang_disposeTranslationUnit(CXTranslationUnitImpl
> * CTUnit) Line 2945 C++
>
>
> To verify this, we made the following dirty fix that solved the problem
> for our limited usecase.
>
> ASTUnit.cpp
>
> static void unmapPreambleFile(const ASTUnit *AU, OnDiskData& D){
> if (!D.PreambleFile.empty()) {
> const ASTContext& Context = AU->getASTContext();
> ASTReader& Reader = (ASTReader&)*Context.ExternalSource;
> if (&Reader){
> serialization::ModuleManager& ModuleMgr = Reader.getModuleManager();
> ModuleMgr.removeModule(D.PreambleFile);
> }
> }
> }
>
> static void erasePreambleFile(const ASTUnit *AU) {
> OnDiskData& D = getOnDiskData(AU);
> unmapPreambleFile(AU, D);
> D.CleanPreambleFile();
> }
>
> static void removeOnDiskEntry(const ASTUnit *AU) {
> // We require the mutex since we are modifying the structure of the
> // DenseMap.
> llvm::MutexGuard Guard(getOnDiskMutex());
> OnDiskDataMap &M = getOnDiskDataMap();
> OnDiskDataMap::iterator I = M.find(AU);
> if (I != M.end()) {
> OnDiskData* pD = I->second;
> unmapPreambleFile(AU, *pD);
> pD->Cleanup();
> delete pD;
> M.erase(AU);
> }
> }
>
> ModuleManager.cpp
>
> void ModuleManager::removeModule(const std::string& FileName){
> for (ModuleIterator M = Chain.begin(), MEnd = Chain.end(); M != MEnd;
> ++M) {
> serialization::ModuleFile* pModuleFile = *M;
> if (pModuleFile->FileName.compare(FileName)==0){
> delete pModuleFile;
> Chain.erase(M);
> break;
> }
> }
> }
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20131220/85ae4fa5/attachment.html>
More information about the cfe-dev
mailing list