[llvm] r285237 - [lit] Work around Windows MSys command line tokenization bug

[Reid Kleckner via llvm-commits]

Author: rnk
Date: Wed Oct 26 15:29:27 2016
New Revision: 285237

URL: http://llvm.org/viewvc/llvm-project?rev=285237&view=rev
Log:
[lit] Work around Windows MSys command line tokenization bug

Summary:
This will allow us to revert LLD r284768, which added spaces to get MSys
echo to print what we want.

Reviewers: ruiu, inglorion, rafael

Subscribers: modocache, llvm-commits

Differential Revision: https://reviews.llvm.org/D26009

Added:
    llvm/trunk/test/Other/lit-quoting.txt
Modified:
    llvm/trunk/utils/lit/lit/TestRunner.py

Added: llvm/trunk/test/Other/lit-quoting.txt
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Other/lit-quoting.txt?rev=285237&view=auto
==============================================================================
--- llvm/trunk/test/Other/lit-quoting.txt (added)
+++ llvm/trunk/test/Other/lit-quoting.txt Wed Oct 26 15:29:27 2016
@@ -0,0 +1,2 @@
+RUN: echo "\"" | FileCheck %s
+CHECK: {{^"$}}

Modified: llvm/trunk/utils/lit/lit/TestRunner.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=285237&r1=285236&r2=285237&view=diff
==============================================================================
--- llvm/trunk/utils/lit/lit/TestRunner.py (original)
+++ llvm/trunk/utils/lit/lit/TestRunner.py Wed Oct 26 15:29:27 2016
@@ -140,6 +140,62 @@ def executeShCmd(cmd, shenv, results, ti
 
     return (finalExitCode, timeoutInfo)
 
+def quote_windows_command(seq):
+    """
+    Reimplement Python's private subprocess.list2cmdline for MSys compatibility
+
+    Based on CPython implementation here:
+      https://hg.python.org/cpython/file/849826a900d2/Lib/subprocess.py#l422
+
+    Some core util distributions (MSys) don't tokenize command line arguments
+    the same way that MSVC CRT does. Lit rolls its own quoting logic similar to
+    the stock CPython logic to paper over these quoting and tokenization rule
+    differences.
+
+    We use the same algorithm from MSDN as CPython
+    (http://msdn.microsoft.com/en-us/library/17w5ykft.aspx), but we treat more
+    characters as needing quoting, such as double quotes themselves.
+    """
+    result = []
+    needquote = False
+    for arg in seq:
+        bs_buf = []
+
+        # Add a space to separate this argument from the others
+        if result:
+            result.append(' ')
+
+        # This logic differs from upstream list2cmdline.
+        needquote = (" " in arg) or ("\t" in arg) or ("\"" in arg) or not arg
+        if needquote:
+            result.append('"')
+
+        for c in arg:
+            if c == '\\':
+                # Don't know if we need to double yet.
+                bs_buf.append(c)
+            elif c == '"':
+                # Double backslashes.
+                result.append('\\' * len(bs_buf)*2)
+                bs_buf = []
+                result.append('\\"')
+            else:
+                # Normal char
+                if bs_buf:
+                    result.extend(bs_buf)
+                    bs_buf = []
+                result.append(c)
+
+        # Add remaining backslashes, if any.
+        if bs_buf:
+            result.extend(bs_buf)
+
+        if needquote:
+            result.extend(bs_buf)
+            result.append('"')
+
+    return ''.join(result)
+
 def _executeShCmd(cmd, shenv, results, timeoutHelper):
     if timeoutHelper.timeoutReached():
         # Prevent further recursion if the timeout has been hit
@@ -315,6 +371,11 @@ def _executeShCmd(cmd, shenv, results, t
                     named_temp_files.append(f.name)
                     args[i] = f.name
 
+        # On Windows, do our own command line quoting for better compatibility
+        # with some core utility distributions.
+        if kIsWindows:
+            args = quote_windows_command(args)
+
         try:
             procs.append(subprocess.Popen(args, cwd=cmd_shenv.cwd,
                                           executable = executable,