[LLVMbugs] [Bug 20880] New: mmap lock problem

bugzilla-daemon at llvm.org bugzilla-daemon at llvm.org
Mon Sep 8 10:33:50 PDT 2014


            Bug ID: 20880
           Summary: mmap lock problem
           Product: clang
           Version: trunk
          Hardware: PC
                OS: Windows XP
            Status: NEW
          Severity: normal
          Priority: P
         Component: libclang
          Assignee: unassignedclangbugs at nondot.org
          Reporter: yaruopooner at gmail.com
                CC: llvmbugs at cs.uiuc.edu
    Classification: Unclassified

* libclang Report

I would like to file 2 bug reports and a separate request for possible

The environment I am currently using is as follows:
  Windows 7(x64)
  GNU Emacs
  llvm trunk
  libclang.dll(x64) build by VS2013
  clang-server(using libclang.dll)

  emacs uses IPC between clang-server to provide code completions

  Flags used for clang_parseTranslationUnit and clang_reparseTranslationUnit
  CXTranslationUnit_PrecompiledPreamble |

** Bug #1

libclang filesystem uses mmap for any files larger than 16KB. The use of mmap
causes problems when using libclang's completion features via IPC.

Let's say we have files "foo.cpp" and "foo.hpp", and "foo.hpp" contains the
declaration of class Foo. Assume "foo.hpp" is larger than 16KB; this will cause
it to be allocated via mmap.

Then, while a CXTranslationUnit exists for class Foo, trying to save foo.hpp
after adding a new property to class Foo will fail. This is due to
CXTranslationUnit for foo.cpp still locking the file foo.hpp by

The decision to use mmap seems to be dictated by the function shouldUseMmap in
clang-trunk/llvm/lib/Support/MemoryBuffer.cpp, but I am suspecting that the
parameter IsVolatileSize passed into it can contain incorrect value.

The comments for getFile, getOpenFileSlice and getOpenFile functions in
clang-trunk/llvm/lib/Support/MemoryBuffer.h describe IsVolatileSize as intended
to be set to true if the code being parsed is undergoing change.

IsVolatileSize Set to true to indicate that the file size may be changing, e.g.
when libclang tries to parse while the user is editing/updating the file. 

So parsing while saving seems to be supported.

But there seems to be cases where the flag is set to false and as a result the
inclded files get locked. The cases where IsVolatileSize becomes false are:

- SrcMgr::ContentCache::IsSystemFile==true
- SourceManager is constructed with UserFilesAreVolatile==false

Even if IsSystemFile==false, when clang_reparseTranslationUnit or
clang_codeCompleteAt is called, a SourceManager is created with
UserFilesAreVolatile==false. This causes included files to be locked by mmap
after completion.

Experimenting with the default value for the UserFilesAreVolatile parameter to
true, I confirmed that after calling clang_reparseTranslationUnit and/or
clang_codeCompleteAt I can still save foo.hpp. However I have not thoroughly
investigated the implication of changes to other libclang functions.

** Bug #2

The modifications in the last paragraph caused a different problem in code

While still keeping CXTranslationUnit generated for foo.cpp, I tried adding a
new property to class Foo and then saved foo.hpp. Subsequently trying to do
code completions in foo.cpp for methods and properties of class Foo
with clang_codeCompleteAt will result in non-nullptr valid value, but the
number of suggestions are in thousands. This is similar to case when one tries
to call code completions for global scope "::".

The symptom persists until the CXTranslationUnit for foo.cpp is discarded.
After discarding and regenerating CXTranslationUnit, a correct result was
returned. mmap was used for preamble-XXXX.pch in this situation. Maybe there is
some issue regarding creation/cleanup of preamble-XXXX.pch?

It should be noted that setting shouldUseMmap to always return false fixed BOTH
Bug #1 (file lock issue) and Bug #2 (incorrect code completions). So I am
suspecting that both bug stem from mmap handling.

** Request for improvement

With the above in mind I would like to request a new feature.

System include files are flagged as IsSystemFile==true, causing mmap to be used
and the files to be locked. But this can be problematic if one is creating or
editing system library himself, but mmap has the header files locked.

Thus, I would think that it would be more appropriate if user can specifically
dictate the libclang's use of mmap, possibly as follows:

-Always : Suitable if not editing headers
-Default: Current implementation; when system header is not edited.
-Never  : If system header file needs to be edited

This will allow users to change setting per use cases.

Best Regards.

You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20140908/7fdb012e/attachment.html>

More information about the llvm-bugs mailing list