[llvm] r256892 - Fixing PR25717: fatal IO error writing large outputs to console on Windows.

[Yunzhong Gao via llvm-commits]

Author: ygao
Date: Tue Jan  5 18:50:06 2016
New Revision: 256892

URL: http://llvm.org/viewvc/llvm-project?rev=256892&view=rev
Log:
Fixing PR25717: fatal IO error writing large outputs to console on Windows.

This patch is similar to the Python issue#11395. We need to cap the output
size to 32767 on Windows to work around the size limit of WriteConsole().
Reference: https://bugs.python.org/issue11395

Writing a test for this bug turns out to be harder than I thought. I am
still working on it (see phabricator review D15705).

Differential Revision: http://reviews.llvm.org/D15553


Modified:
    llvm/trunk/lib/Support/Windows/WindowsSupport.h
    llvm/trunk/lib/Support/raw_ostream.cpp

Modified: llvm/trunk/lib/Support/Windows/WindowsSupport.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/Windows/WindowsSupport.h?rev=256892&r1=256891&r2=256892&view=diff
==============================================================================
--- llvm/trunk/lib/Support/Windows/WindowsSupport.h (original)
+++ llvm/trunk/lib/Support/Windows/WindowsSupport.h Tue Jan  5 18:50:06 2016
@@ -30,6 +30,8 @@
 #define _WIN32_WINNT 0x0601
 #define _WIN32_IE    0x0800 // MinGW at it again. FIXME: verify if still needed.
 #define WIN32_LEAN_AND_MEAN
+#define NOGDI
+#define NOMINMAX
 
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringExtras.h"
@@ -44,6 +46,20 @@
 #include <string>
 #include <vector>
 
+#ifndef __CYGWIN__
+#include <VersionHelpers.h>
+#else
+// Cygwin does not have the IsWindows8OrGreater() API.
+inline bool IsWindows8OrGreater() {
+  OSVERSIONINFO osvi = {};
+  osvi.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
+  if (!::GetVersionEx(&osvi))
+    return false;
+  return (osvi.dwMajorVersion > 6 ||
+          (osvi.dwMajorVersion == 6 && osvi.dwMinorVersion >= 2));
+}
+#endif // __CYGWIN__
+
 inline bool MakeErrMsg(std::string* ErrMsg, const std::string& prefix) {
   if (!ErrMsg)
     return true;

Modified: llvm/trunk/lib/Support/raw_ostream.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/raw_ostream.cpp?rev=256892&r1=256891&r2=256892&view=diff
==============================================================================
--- llvm/trunk/lib/Support/raw_ostream.cpp (original)
+++ llvm/trunk/lib/Support/raw_ostream.cpp Tue Jan  5 18:50:06 2016
@@ -57,6 +57,10 @@
 #endif
 #endif
 
+#ifdef LLVM_ON_WIN32
+#include "Windows/WindowsSupport.h"
+#endif
+
 using namespace llvm;
 
 raw_ostream::~raw_ostream() {
@@ -567,8 +571,21 @@ void raw_fd_ostream::write_impl(const ch
   assert(FD >= 0 && "File already closed.");
   pos += Size;
 
+#ifndef LLVM_ON_WIN32
+  bool ShouldWriteInChunks = false;
+#else
+  // Writing a large size of output to Windows console returns ENOMEM. It seems
+  // that, prior to Windows 8, WriteFile() is redirecting to WriteConsole(), and
+  // the latter has a size limit (66000 bytes or less, depending on heap usage).
+  bool ShouldWriteInChunks = !!::_isatty(FD) && !IsWindows8OrGreater();
+#endif
+
   do {
-    ssize_t ret = ::write(FD, Ptr, Size);
+    size_t ChunkSize = Size;
+    if (ChunkSize > 32767 && ShouldWriteInChunks)
+        ChunkSize = 32767;
+
+    ssize_t ret = ::write(FD, Ptr, ChunkSize);
 
     if (ret < 0) {
       // If it's a recoverable error, swallow it and retry the write.