[llvm] r317015 - [Support] Make the default chunk size of raw_fd_ostream to 1 GiB.

[Rui Ueyama via llvm-commits]

Author: ruiu
Date: Tue Oct 31 10:37:20 2017
New Revision: 317015

URL: http://llvm.org/viewvc/llvm-project?rev=317015&view=rev
Log:
[Support] Make the default chunk size of raw_fd_ostream to 1 GiB.

Previously, we call write(2) for each 32767 byte chunk. That is not
efficient because Linux can handle much larger write requests.
This patch changes the chunk size on Linux to 1 GiB.

This patch also changes the default chunks size to SSIZE_MAX. I think
that doesn't in practice change this function's behavior on any operating
system because SSIZE_MAX on 64-bit machine is unrealistically large,
and writing 2 GiB (SSIZE_MAX on 32-bit) on a 32-bit machine by a single
call of write(2) is also unrealistic, as the userspace is usually
limited to 2 GiB. That said, it is in general a good thing to do because
a write larger than SSIZE_MAX is implementation-defined in POSIX.

Differential Revision: https://reviews.llvm.org/D39444

Modified:
    llvm/trunk/lib/Support/raw_ostream.cpp

Modified: llvm/trunk/lib/Support/raw_ostream.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/raw_ostream.cpp?rev=317015&r1=317014&r2=317015&view=diff
==============================================================================
--- llvm/trunk/lib/Support/raw_ostream.cpp (original)
+++ llvm/trunk/lib/Support/raw_ostream.cpp Tue Oct 31 10:37:20 2017
@@ -578,24 +578,25 @@ void raw_fd_ostream::write_impl(const ch
   assert(FD >= 0 && "File already closed.");
   pos += Size;
 
-#ifndef LLVM_ON_WIN32
+  // The maximum write size is limited to SSIZE_MAX because a write
+  // greater than SSIZE_MAX is implementation-defined in POSIX.
+  // Since SSIZE_MAX is not portable, we use SIZE_MAX >> 1 instead.
+  size_t MaxWriteSize = SIZE_MAX >> 1;
+
 #if defined(__linux__)
-  bool ShouldWriteInChunks = true;
-#else
-  bool ShouldWriteInChunks = false;
-#endif
-#else
+  // It is observed that Linux returns EINVAL for a very large write (>2G).
+  // Make it a reasonably small value.
+  MaxWriteSize = 1024 * 1024 * 1024;
+#elif defined(LLVM_ON_WIN32)
   // Writing a large size of output to Windows console returns ENOMEM. It seems
   // that, prior to Windows 8, WriteFile() is redirecting to WriteConsole(), and
   // the latter has a size limit (66000 bytes or less, depending on heap usage).
-  bool ShouldWriteInChunks = !!::_isatty(FD) && !RunningWindows8OrGreater();
+  if (::_isatty(FD) && !RunningWindows8OrGreater())
+    MaxWriteSize = 32767;
 #endif
 
   do {
-    size_t ChunkSize = Size;
-    if (ChunkSize > 32767 && ShouldWriteInChunks)
-        ChunkSize = 32767;
-
+    size_t ChunkSize = std::min(Size, MaxWriteSize);
     ssize_t ret = ::write(FD, Ptr, ChunkSize);
 
     if (ret < 0) {