[llvm-bugs] [Bug 27386] New: Windows llvm::sys::fs::remove algorithm is *VERY* subtly incorrect, resulting in unpredictable "directory not empty" errors

via llvm-bugs llvm-bugs at lists.llvm.org
Sat Apr 16 13:53:15 PDT 2016


            Bug ID: 27386
           Summary: Windows llvm::sys::fs::remove algorithm is *VERY*
                    subtly incorrect, resulting in unpredictable
                    "directory not empty" errors
           Product: libraries
           Version: trunk
          Hardware: PC
                OS: Windows NT
            Status: NEW
          Severity: normal
          Priority: P
         Component: Support Libraries
          Assignee: unassignedbugs at nondot.org
          Reporter: test35965 at gmail.com
                CC: llvm-bugs at lists.llvm.org
    Classification: Unclassified

Created attachment 16224
  --> https://llvm.org/bugs/attachment.cgi?id=16224&action=edit
The MSBuild output of two tests failing because of this bug. There's a third
failure (tools/llvm-symbolizer/ppc64.test), unrelated to this bug.

Under heavy IO load, removing directories on Windows with llvm::sys::fs:remove
will fail unpredictably.

I've just observed the TestWriteWithSystemEncoding test in
./llvm/unittests/Support/ProgramTest.cpp failing because of this. That test
creates a kinda-unique directory named "program-test-%%%%%%" (where % is a
pseudorandom alphanumeric), creates a file named "international-file.txt" in
it, and roundtrip-tests some text (writes some text with one encoding, reads it
with another, checks if it's not garbled). After that, it deletes the file
(llvm::sys::fs::remove(file_pathname.str())) and then the directory
(llvm::sys::fs::remove(TestDirectory.str())). Essentially, when Windows
"Deletes" a file, it doesn't actually Delete the file; when the system is under
load, "international-file.txt" may STILL be in the directory by the time of

The details of WHY this will fail are very counter-intuitive, and you should
see "CppCon 2015: Niall Douglas “Racing The File System"
(https://youtu.be/uhRWMGBjlO8?t=7m35s) for *specifically* why the code is
wrong. Lots of programs get this wrong, Douglas mentions CPython, and I've seen
it in Chromium.

Sidenote: The documentation for RemoveDirectory
(https://msdn.microsoft.com/en-us/library/windows/desktop/aa365488.aspx) is
kinda crappy, because it says "RemoveDirectory removes a directory junction,
even if the contents of the target are not empty; the function removes
directory junctions regardless of the state of the target object.", but
"Creating and Deleting Directories"
(https://msdn.microsoft.com/en-us/library/windows/desktop/aa363872.aspx) says:
"To delete an existing directory, use the RemoveDirectory or
RemoveDirectoryTransacted function. Before removing a directory, you must
ensure that the directory is empty and that you have the delete access
privilege for the directory. To do the latter, call the GetSecurityInfo

You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20160416/f1aee838/attachment.html>

More information about the llvm-bugs mailing list