[llvm] r292743 - [libFuzzer] Avoid undefined behavior, properly discard output to stdout/stderr.

[Marcos Pividori via llvm-commits]

Author: mpividori
Date: Sat Jan 21 19:58:45 2017
New Revision: 292743

URL: http://llvm.org/viewvc/llvm-project?rev=292743&view=rev
Log:
[libFuzzer] Avoid undefined behavior, properly discard output to stdout/stderr.

Fix libFuzzer when setting -close_fd_mask to a non-zero value.
In previous implementation, libFuzzer closes the file descriptors for
stdout/stderr. This has some disavantages:

For `fuzzer-fdmask.test`, we write directly to stdout and stderr using the
file streams stdout and stderr, after the file descriptors are closed, which is
undefined behavior. In Windows, in particular, this was making the test fail.

Also, if we close stdout and we open a new file in libFuzzer, we get the file
descriptor 1, which could generate problem if some code assumes file descriptors
refers to stdout and works directly writing to the file descriptor 1, but it
will be writing to the opened file (for example using std::cout).

Instead of closing the file descriptors, I redirect the output to /dev/null on
linux and nul on Windows.

Differential Revision: https://reviews.llvm.org/D28718

Modified:
    llvm/trunk/lib/Fuzzer/FuzzerIO.cpp
    llvm/trunk/lib/Fuzzer/FuzzerIO.h
    llvm/trunk/lib/Fuzzer/FuzzerIOPosix.cpp
    llvm/trunk/lib/Fuzzer/FuzzerIOWindows.cpp

Modified: llvm/trunk/lib/Fuzzer/FuzzerIO.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerIO.cpp?rev=292743&r1=292742&r2=292743&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerIO.cpp (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerIO.cpp Sat Jan 21 19:58:45 2017
@@ -97,13 +97,13 @@ void DupAndCloseStderr() {
       OutputFile = NewOutputFile;
       if (EF->__sanitizer_set_report_fd)
         EF->__sanitizer_set_report_fd(reinterpret_cast<void *>(OutputFd));
-      CloseFile(2);
+      DiscardOutput(2);
     }
   }
 }
 
 void CloseStdout() {
-  CloseFile(1);
+  DiscardOutput(1);
 }
 
 void Printf(const char *Fmt, ...) {

Modified: llvm/trunk/lib/Fuzzer/FuzzerIO.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerIO.h?rev=292743&r1=292742&r2=292743&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerIO.h (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerIO.h Sat Jan 21 19:58:45 2017
@@ -64,6 +64,8 @@ int DuplicateFile(int Fd);
 
 void RemoveFile(const std::string &Path);
 
+void DiscardOutput(int Fd);
+
 }  // namespace fuzzer
 
 #endif  // LLVM_FUZZER_IO_H

Modified: llvm/trunk/lib/Fuzzer/FuzzerIOPosix.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerIOPosix.cpp?rev=292743&r1=292742&r2=292743&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerIOPosix.cpp (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerIOPosix.cpp Sat Jan 21 19:58:45 2017
@@ -75,6 +75,14 @@ void RemoveFile(const std::string &Path)
   unlink(Path.c_str());
 }
 
+void DiscardOutput(int Fd) {
+  FILE* Temp = fopen("/dev/null", "w");
+  if (!Temp)
+    return;
+  dup2(fileno(Temp), Fd);
+  fclose(Temp);
+}
+
 std::string DirName(const std::string &FileName) {
   char *Tmp = new char[FileName.size() + 1];
   memcpy(Tmp, FileName.c_str(), FileName.size() + 1);

Modified: llvm/trunk/lib/Fuzzer/FuzzerIOWindows.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerIOWindows.cpp?rev=292743&r1=292742&r2=292743&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerIOWindows.cpp (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerIOWindows.cpp Sat Jan 21 19:58:45 2017
@@ -141,6 +141,14 @@ void RemoveFile(const std::string &Path)
   _unlink(Path.c_str());
 }
 
+void DiscardOutput(int Fd) {
+  FILE* Temp = fopen("nul", "w");
+  if (!Temp)
+    return;
+  _dup2(_fileno(Temp), Fd);
+  fclose(Temp);
+}
+
 static bool IsSeparator(char C) {
   return C == '\\' || C == '/';
 }