[cfe-dev] libclang locking files for write/delete on windows

[Yaron Keren]

mapped_file_region is owned by a MemoryBufferMMapFile (MemoryBuffer.cpp)
which inherits from MemoryBuffer.

The MemoryBuffer for a source file lives in a ContentCache, owned by
clang::SourceManager::Fileinfos (SourceManager.cpp).

The fileinfos structure is destructed  in clang SourceManager destructor
which happens when clang completes compiling the compilation unit.

To summarize ~mapped_file_region gets called when clang completes compiling
every file.

2014-12-10 23:45 GMT+02:00 Ivan Koster <ivankoster at gmail.com>:

> From
> http://msdn.microsoft.com/en-us/library/windows/desktop/aa366537(v=vs.85).aspx
>>
>> Mapped views of a file mapping object maintain internal references to the
>> object, and a file mapping object does not close until all references to it
>> are released. Therefore, to fully close a file mapping object, an
>> application must unmap all mapped views of the file mapping object by
>> calling *UnmapViewOfFile*
>> <http://msdn.microsoft.com/en-us/library/windows/desktop/aa366882(v=vs.85).aspx> and
>> close the file mapping object handle by calling *CloseHandle*
>> <http://msdn.microsoft.com/en-us/library/windows/desktop/ms724211(v=vs.85).aspx>.
>> These functions can be called in any order.
>
>
> Since CloseHandle is already called after the init, it seems the mapped
> view is still open?
> I'm a C and python programmer and (no C++ experience at all) have an
> incredible hard time reading the code.
> Nevertheless, Mapping in mapped_file_region seems to be the only view
> created and it is unmapped in the destructor:
>
> mapped_file_region::~mapped_file_region() {
>>   if (Mapping)
>>     ::UnmapViewOfFile(Mapping);
>> }
>
>
> Can I conclude that the destructor is not called?
>
>
>
> On Wed, Dec 10, 2014 at 6:57 AM, Kim Gräsman <kim.grasman at gmail.com>
> wrote:
>
>> Hi Reid,
>>
>> Yep; from Path.inc/mapped_region::init:
>>
>>   // Close all the handles except for the view. It will keep the other
>> handles
>>   // alive.
>>   ::CloseHandle(FileMappingHandle);
>>   if (FileDescriptor) {
>>     if (CloseFD)
>>       _close(FileDescriptor); // Also closes FileHandle.
>>   } else
>>     ::CloseHandle(FileHandle);
>>   return std::error_code();
>>
>> But as far as I can reason about it, the mapping is unmapped in
>> ~mapped_region. Maybe the MemoryBuffer backed by the mapping is cached
>> somewhere and lives on?
>>
>> - Kim
>>
>> On Wed, Dec 10, 2014 at 12:27 AM, Reid Kleckner <rnk at google.com> wrote:
>> > Can the memory mapping outlive the file handle? I bet it can.
>> >
>> > On Tue, Dec 9, 2014 at 3:17 PM, Ivan Koster <ivankoster at gmail.com>
>> wrote:
>> >>>
>> >>> Hi Ivan,
>> >>
>> >>
>> >> Hello!
>> >>
>> >>> This is good info; MemoryBuffer::getOpenFile conditionally uses file
>> >>> mappings or normal file reads depending on file size (see
>> >>> MemoryBuffer.cpp/shouldUseMmap).
>> >>>
>> >>> Since the mapping uses a stricter sharing mode than the normal file
>> >>> read, it's more likely to fail. Still not sure who's writing/deleting
>> >>> the file while Clang attempts to map it, can you make that out from
>> >>> your PM logs?
>> >>
>> >>
>> >> I've exported the log to csv and you can find it here:
>> >>
>> https://drive.google.com/file/d/0B8xFlIzMhHcpdzR6OW4wM3BZdGM/view?usp=sharing
>> >>
>> >> This is a log of sublime text opening dummy.c, commanding YCMD
>> >> (python.exe) to analyze the dummy.c, which so and so on then finally
>> makes
>> >> libclang.dll open dummy.h, and finally me killing the python.exe
>> process.
>> >> As you can see, killing the proces does not log anything, even though
>> it
>> >> "frees" the file.
>> >>
>> >> The trace shows two threads of libclang.dll (TID 6888 and TID 4040),
>> but
>> >> they do not operate concurrently. 4040 only starts after 6888 is done
>> with
>> >> dummy.h.
>> >> When I examine the stack traces of the CreateFileMapping operation that
>> >> appears in pairs, they both happen in the same kernel32.dll
>> >> CreateFileMappingA function.
>> >> So the kernel seems to work around the FILE LOCKED WITH ONLY HEADERS
>> thing
>> >> here?
>> >>
>> >> There are no dangling file handles, the trace shows only pairs of open
>> and
>> >> close events. This is confirmed by the windows handle.exe tool, which
>> can
>> >> not find any filehandles to dummy.h even though the file is locked.
>> >> The weird thing is that the windows Process Explorer has a "find file
>> >> handle or DLL" function.
>> >> When I make it search for dummy.h, it finds not a file handle, but a
>> DLL
>> >> inside python.exe called dummy.h.
>> >> Somehow Process Explorer reports that dummy.h is loaded as a DLL?
>> (When I
>> >> kill python.exe this disappears).
>> >>
>> >> I haven't dived so deep into windows stuff before so I'm hoping this
>> >> helps!
>> >> But to be honest, this is starting to not make sense to me. The file is
>> >> locked even though there is no open file handle. Process Explorer
>> reporting
>> >> that it is loaded as a DLL...
>> >>
>> >> Regards,
>> >>
>> >> Ivan
>> >
>> >
>>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20141211/36f22e04/attachment.html>