[llvm-bugs] [Bug 27386] New: Windows llvm::sys::fs::remove algorithm is *VERY* subtly incorrect, resulting in unpredictable "directory not empty" errors

[via llvm-bugs]

https://llvm.org/bugs/show_bug.cgi?id=27386

            Bug ID: 27386
           Summary: Windows llvm::sys::fs::remove algorithm is *VERY*
                    subtly incorrect, resulting in unpredictable
                    "directory not empty" errors
           Product: libraries
           Version: trunk
          Hardware: PC
                OS: Windows NT
            Status: NEW
          Severity: normal
          Priority: P
         Component: Support Libraries
          Assignee: unassignedbugs at nondot.org
          Reporter: test35965 at gmail.com
                CC: llvm-bugs at lists.llvm.org
    Classification: Unclassified

Created attachment 16224
  --> https://llvm.org/bugs/attachment.cgi?id=16224&action=edit
The MSBuild output of two tests failing because of this bug. There's a third
failure (tools/llvm-symbolizer/ppc64.test), unrelated to this bug.

Under heavy IO load, removing directories on Windows with llvm::sys::fs:remove
will fail unpredictably.

I've just observed the TestWriteWithSystemEncoding test in
./llvm/unittests/Support/ProgramTest.cpp failing because of this. That test
creates a kinda-unique directory named "program-test-%%%%%%" (where % is a
pseudorandom alphanumeric), creates a file named "international-file.txt" in
it, and roundtrip-tests some text (writes some text with one encoding, reads it
with another, checks if it's not garbled). After that, it deletes the file
(llvm::sys::fs::remove(file_pathname.str())) and then the directory
(llvm::sys::fs::remove(TestDirectory.str())). Essentially, when Windows
"Deletes" a file, it doesn't actually Delete the file; when the system is under
load, "international-file.txt" may STILL be in the directory by the time of
llvm::sys::fs::remove(TestDirectory.str()).



The details of WHY this will fail are very counter-intuitive, and you should
see "CppCon 2015: Niall Douglas “Racing The File System"
(https://youtu.be/uhRWMGBjlO8?t=7m35s) for *specifically* why the code is
wrong. Lots of programs get this wrong, Douglas mentions CPython, and I've seen
it in Chromium.



Sidenote: The documentation for RemoveDirectory
(https://msdn.microsoft.com/en-us/library/windows/desktop/aa365488.aspx) is
kinda crappy, because it says "RemoveDirectory removes a directory junction,
even if the contents of the target are not empty; the function removes
directory junctions regardless of the state of the target object.", but
"Creating and Deleting Directories"
(https://msdn.microsoft.com/en-us/library/windows/desktop/aa363872.aspx) says:
"To delete an existing directory, use the RemoveDirectory or
RemoveDirectoryTransacted function. Before removing a directory, you must
ensure that the directory is empty and that you have the delete access
privilege for the directory. To do the latter, call the GetSecurityInfo
function."

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20160416/f1aee838/attachment.html>