[clang] fd2dff1 - [clang][lex][minimizer] Ensure whitespace between squashed lines

[Jan Svoboda via cfe-commits]

Author: Jan Svoboda
Date: 2022-02-15T09:49:03+01:00
New Revision: fd2dff17c53dbfd89789989bedd949c496e7b6b7

URL: https://github.com/llvm/llvm-project/commit/fd2dff17c53dbfd89789989bedd949c496e7b6b7
DIFF: https://github.com/llvm/llvm-project/commit/fd2dff17c53dbfd89789989bedd949c496e7b6b7.diff

LOG: [clang][lex][minimizer] Ensure whitespace between squashed lines

The minimizer tries to squash multi-line macro definitions into single line. For that to work, contents of each line need to be separated by a space. Since we always strip leading whitespace on lines of a macro definition, the code currently tries to preserve exactly one space that appeared before the backslash.

This means the following code:

```
#define FOO(BAR) \
  #BAR           \
  baz
```

gets minimized into:

```
#define FOO(BAR) #BAR baz
```

However, if there are no spaces before the backslash on line 2:

```
#define FOO(BAR) \
  #BAR\
  baz
```

no space can be preserved, leading to (most likely) malformed macro definition:

```
#define FOO(BAR) #BARbaz
```

This patch makes sure we always put exactly one space at the end of line ending with a backslash.

Reviewed By: arphaman

Differential Revision: https://reviews.llvm.org/D119231

Added: 
    

Modified: 
    clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
    clang/unittests/Lex/DependencyDirectivesSourceMinimizerTest.cpp

Removed: 
    


################################################################################
diff  --git a/clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp b/clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
index f597c56837fb..d247e1471cc8 100644
--- a/clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
+++ b/clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
@@ -270,6 +270,15 @@ static const char *findLastNonSpace(const char *First, const char *Last) {
   return Last;
 }
 
+static const char *findLastNonSpaceNonBackslash(const char *First,
+                                                const char *Last) {
+  assert(First <= Last);
+  while (First != Last &&
+         (isHorizontalWhitespace(Last[-1]) || Last[-1] == '\\'))
+    --Last;
+  return Last;
+}
+
 static const char *findFirstTrailingSpace(const char *First,
                                           const char *Last) {
   const char *LastNonSpace = findLastNonSpace(First, Last);
@@ -434,11 +443,11 @@ void Minimizer::printToNewline(const char *&First, const char *const End) {
       return;
     }
 
-    // Print up to the backslash, backing up over spaces. Preserve at least one
-    // space, as the space matters when tokens are separated by a line
-    // continuation.
-    append(First, findFirstTrailingSpace(
-                      First, LastBeforeTrailingSpace - 1));
+    // Print up to the last character that's not a whitespace or backslash.
+    // Then print exactly one space, which matters when tokens are separated by
+    // a line continuation.
+    append(First, findLastNonSpaceNonBackslash(First, Last));
+    put(' ');
 
     First = Last;
     skipNewline(First, End);

diff  --git a/clang/unittests/Lex/DependencyDirectivesSourceMinimizerTest.cpp b/clang/unittests/Lex/DependencyDirectivesSourceMinimizerTest.cpp
index d65c8de7f0e1..5fd4f6024bff 100644
--- a/clang/unittests/Lex/DependencyDirectivesSourceMinimizerTest.cpp
+++ b/clang/unittests/Lex/DependencyDirectivesSourceMinimizerTest.cpp
@@ -713,6 +713,17 @@ TEST(MinimizeSourceToDependencyDirectivesTest,
       Out.data());
 }
 
+TEST(MinimizeSourceToDependencyDirectivesTest,
+     SupportWhitespaceBeforeLineContinuation) {
+  SmallVector<char, 128> Out;
+
+  ASSERT_FALSE(minimizeSourceToDependencyDirectives("#define FOO(BAR) \\\n"
+                                                    "  #BAR\\\n"
+                                                    "  baz\n",
+                                                    Out));
+  EXPECT_STREQ("#define FOO(BAR) #BAR baz\n", Out.data());
+}
+
 TEST(MinimizeSourceToDependencyDirectivesTest,
      SupportWhitespaceBeforeLineContinuationInStringSkipping) {
   SmallVector<char, 128> Out;