[PATCH] D15832: [Clang/Support/Windows/Unix] Command lines created by clang may exceed the command length limit set by the OS

[Oleg Ranevskyy via llvm-commits]

iid_iunknown created this revision.
iid_iunknown added a reviewer: rafael.
iid_iunknown added subscribers: llvm-commits, asl.
iid_iunknown set the repository for this revision to rL LLVM.

LLVM part of the patch is D15831.

When clang runs an external tool such as a linker it may create a command line that exceeds the length limit.

Clang uses the llvm::sys::argumentsFitWithinSystemLimits function to check if command line length fits the OS 

limitation. There are two problems in this function that may cause exceeding of the limit:

1. It ignores the length of the program path in its calculations. On the other hand, clang adds the program 

path to the command line when it runs the program.

2. It assumes no space character is inserted after the last argument, which is not true for Windows. The flattenArgs function adds the trailing space for *each* argument. The result of this is that the terminating NULL character is not counted and may be placed beyond the length limit if the command line is exactly 32768 characters long. The WinAPI's CreateProcess does not find the NULL character and fails.


Repository:
  rL LLVM

http://reviews.llvm.org/D15832

Files:
  lib/Driver/Driver.cpp

Index: lib/Driver/Driver.cpp
===================================================================
--- lib/Driver/Driver.cpp
+++ lib/Driver/Driver.cpp
@@ -699,11 +699,11 @@
 }
 
 void Driver::setUpResponseFiles(Compilation &C, Command &Cmd) {
-  // Since argumentsFitWithinSystemLimits() may underestimate system's capacity
+  // Since commandLineFitsWithinSystemLimits() may underestimate system's capacity
   // if the tool does not support response files, there is a chance/ that things
   // will just work without a response file, so we silently just skip it.
   if (Cmd.getCreator().getResponseFilesSupport() == Tool::RF_None ||
-      llvm::sys::argumentsFitWithinSystemLimits(Cmd.getArguments()))
+      llvm::sys::commandLineFitsWithinSystemLimits(Cmd.getExecutable(), Cmd.getArguments()))
     return;
 
   std::string TmpName = GetTemporaryPath("response", "txt");


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D15832.43810.patch
Type: text/x-patch
Size: 887 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20151230/8ed71e39/attachment.bin>