[cfe-dev] [windows libclang] libclang fails to delete obsolete temporary .pch files

[William Ledoux]

Hi,

I use clang_reparseTranslationUnit with CXTranslationUnit_PrecompiledPreamble
flag on windows.

I investigated why libclang.dll fails to delete old pch files on windows.

Modules keep an active file mapping on file until they are destroyed.
 Later on, this still active mapping will cause DeleteFileW to fail
(silently..) .


There is at least two cases when it happens:

The first one is when some preamble has changed and a new pch needs to
replace the current one,  CleanPreambleFile , which purpose is to destroy
the now old pch file will be called *before *the corresponding module is
destroyed when the mapping will be recycled in ASTUnit::Parse.

The second one is when we destroy the translation unit with clang_
disposeTranslationUnit, CleanPreambleFile which purpose is to destroy the
pch file, is called *before* the corresponding module is destroyed in ~
ModuleFile.

Below are the corresponding callstacks.


   1.   libclang.dll!`anonymous namespace'::OnDiskData::*CleanPreambleFile*()
   Line 175 C++
   2.   libclang.dll!erasePreambleFile(const clang::ASTUnit * AU) Line 144
   C++
   3.
   libclang.dll!clang::ASTUnit::getMainBufferWithPrecompiledPreamble(const
   clang::CompilerInvocation & PreambleInvocationIn, bool AllowRebuild,
   unsigned int MaxLines) Line 1511 C++
   4.
   libclang.dll!clang::ASTUnit::Reparse(std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char>
   >,llvm::PointerUnion<char const *,llvm::MemoryBuffer const *> > *
   RemappedFiles, unsigned int NumRemappedFiles) Line 2116 C++
   5.   libclang.dll!clang_reparseTranslationUnit_Impl(void * UserData)
   Line 3008 C++


   1.
   libclang.dll!llvm::sys::fs::mapped_file_region::~mapped_file_region() Line
   833 C++
   2.   libclang.dll!`anonymous
   namespace'::MemoryBufferMMapFile::~MemoryBufferMMapFile() C++
   3.   libclang.dll!`anonymous namespace'::MemoryBufferMMapFile::`scalar
   deleting destructor'(unsigned int) C++
   4.
   libclang.dll!llvm::OwningPtr<llvm::MemoryBuffer>::~OwningPtr<llvm::MemoryBuffer>()
   Line 45 C++
   5.   libclang.dll!clang::serialization::ModuleFile::~ModuleFile() Line
   58 C++
   6.   libclang.dll!clang::serialization::ModuleFile::`scalar deleting
   destructor'(unsigned int) C++
   7.   libclang.dll!clang::serialization::ModuleManager::~ModuleManager()
   Line 230 C++
   8.   libclang.dll!clang::ASTReader::~ASTReader() Line 7638 C++
   9.   libclang.dll!clang::ASTReader::`scalar deleting
   destructor'(unsigned int) C++
   10.
   libclang.dll!llvm::OwningPtr<clang::ExternalASTSource>::~OwningPtr<clang::ExternalASTSource>()
   Line 45 C++
   11.   libclang.dll!clang::ASTContext::~ASTContext() Line 790 C++
   12.   libclang.dll!clang::ASTContext::`scalar deleting
   destructor'(unsigned int) C++
   13.   libclang.dll!llvm::RefCountedBase<clang::ASTContext>::Release()
   Line 54 C++
   14.
   libclang.dll!llvm::IntrusiveRefCntPtrInfo<clang::ASTContext>::release(clang::ASTContext
   * obj) Line 89 C++
   15.
   libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::release() Line
   178 C++
   16.
   libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::~IntrusiveRefCntPtr<clang::ASTContext>()
   Line 148 C++
   17.
   libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::operator=(llvm::IntrusiveRefCntPtr<clang::ASTContext>
   S) Line 145 C++
   18.   libclang.dll!clang::ASTUnit::Parse(llvm::MemoryBuffer *
   OverrideMainBuffer) Line 1169 C++
   19.
   libclang.dll!clang::ASTUnit::Reparse(std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char>
   >,llvm::PointerUnion<char const *,llvm::MemoryBuffer const *> > *
   RemappedFiles, unsigned int NumRemappedFiles) Line 2125 C++
   20.   libclang.dll!clang_reparseTranslationUnit_Impl(void * UserData)
   Line 3008 C++


   1.   libclang.dll!`anonymous namespace'::OnDiskData::*CleanPreambleFile*()
   Line 175 C++
   2.   libclang.dll!`anonymous namespace'::OnDiskData::Cleanup() Line 183
   C++
   3.   libclang.dll!removeOnDiskEntry(const clang::ASTUnit * AU) Line 154
   C++
   4.   libclang.dll!clang::ASTUnit::~ASTUnit() Line 249 C++
   5.   libclang.dll!clang::ASTUnit::`scalar deleting destructor'(unsigned
   int) C++
   6.   libclang.dll!clang_disposeTranslationUnit(CXTranslationUnitImpl *
   CTUnit) Line 2945 C++


   1.   libclang.dll!llvm::sys::fs::mapped_file_region::~mapped_file_region()
   Line 833 C++
   2.   libclang.dll!`anonymous
namespace'::MemoryBufferMMapFile::~MemoryBufferMMapFile()
   C++
   3.   libclang.dll!`anonymous namespace'::MemoryBufferMMapFile::`scalar
   deleting destructor'(unsigned int) C++
   4.   libclang.dll!llvm::OwningPtr<llvm::MemoryBuffer>::~
   OwningPtr<llvm::MemoryBuffer>() Line 45 C++
   5.   libclang.dll!clang::serialization::ModuleFile::~ModuleFile() Line
   58 C++
   6.   libclang.dll!clang::serialization::ModuleFile::`scalar deleting
   destructor'(unsigned int) C++
   7.   libclang.dll!clang::serialization::ModuleManager::~ModuleManager()
   Line 230 C++
   8.   libclang.dll!clang::ASTReader::~ASTReader() Line 7638 C++
   9.   libclang.dll!clang::ASTReader::`scalar deleting
   destructor'(unsigned int) C++
   10.   libclang.dll!llvm::OwningPtr<clang::ExternalASTSource>::~
   OwningPtr<clang::ExternalASTSource>() Line 45 C++
   11.   libclang.dll!clang::ASTContext::~ASTContext() Line 790 C++
   12.   libclang.dll!clang::ASTContext::`scalar deleting
   destructor'(unsigned int) C++
   13.   libclang.dll!llvm::RefCountedBase<clang::ASTContext>::Release()
   Line 54 C++
   14.   libclang.dll!llvm::IntrusiveRefCntPtrInfo<clang::
   ASTContext>::release(clang::ASTContext * obj) Line 89 C++
   15.   libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::release()
   Line 178 C++
   16.   libclang.dll!llvm::IntrusiveRefCntPtr<clang::ASTContext>::~
   IntrusiveRefCntPtr<clang::ASTContext>() Line 148 C++
   17.   libclang.dll!clang::ASTUnit::~ASTUnit() Line 274 C++
   18.   libclang.dll!clang::ASTUnit::`scalar deleting destructor'(unsigned
   int) C++
   19.   libclang.dll!clang_disposeTranslationUnit(CXTranslationUnitImpl *
   CTUnit) Line 2945 C++


To verify this, we made the following dirty fix that solved the problem for
our limited usecase.

ASTUnit.cpp

static void unmapPreambleFile(const ASTUnit *AU, OnDiskData& D){
  if (!D.PreambleFile.empty()) {
    const ASTContext& Context = AU->getASTContext();
    ASTReader& Reader = (ASTReader&)*Context.ExternalSource;
    if (&Reader){
      serialization::ModuleManager& ModuleMgr = Reader.getModuleManager();
      ModuleMgr.removeModule(D.PreambleFile);
    }
  }
}

static void erasePreambleFile(const ASTUnit *AU) {
  OnDiskData& D = getOnDiskData(AU);
  unmapPreambleFile(AU, D);
  D.CleanPreambleFile();
}

static void removeOnDiskEntry(const ASTUnit *AU) {
  // We require the mutex since we are modifying the structure of the
  // DenseMap.
  llvm::MutexGuard Guard(getOnDiskMutex());
  OnDiskDataMap &M = getOnDiskDataMap();
  OnDiskDataMap::iterator I = M.find(AU);
  if (I != M.end()) {
    OnDiskData* pD = I->second;
    unmapPreambleFile(AU, *pD);
    pD->Cleanup();
    delete pD;
    M.erase(AU);
  }
}

ModuleManager.cpp

void ModuleManager::removeModule(const std::string& FileName){
  for (ModuleIterator M = Chain.begin(), MEnd = Chain.end(); M != MEnd;
++M) {
    serialization::ModuleFile* pModuleFile = *M;
    if (pModuleFile->FileName.compare(FileName)==0){
      delete pModuleFile;
      Chain.erase(M);
      break;
    }
  }
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20131220/6ebad060/attachment.html>