[PATCH] D64297: [JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.

[Sam McCall via Phabricator via llvm-commits]

sammccall created this revision.
sammccall added reviewers: phosek, klimek.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

It's common to use compiler wrappers by setting CC="gomacc clang++".
This results in both args appearing in compile_commands.json, and clang's driver
can't handle this.

This patch attempts to recognize this pattern (by looking for well-known
wrappers) and dropping argv0 in this case.

It conservatively ignores other cases for now:

- wrappers with unknown names
- wrappers that accept -flags
- wrappers where the compiler to use is implied (usually cc or gcc)

This is done at the JSONCompilationDatabase level rather than somewhere more
fundamental, as (hopefully) this isn't a general conceptual problem, but a messy
aspect of the nature of the ecosystem around compile_commands.json.
i.e. compilation databases more tightly tied to the build system should not have
this problem.


Repository:
  rL LLVM

https://reviews.llvm.org/D64297

Files:
  clang/lib/Tooling/JSONCompilationDatabase.cpp
  clang/unittests/Tooling/CompilationDatabaseTest.cpp


Index: clang/unittests/Tooling/CompilationDatabaseTest.cpp
===================================================================
--- clang/unittests/Tooling/CompilationDatabaseTest.cpp
+++ clang/unittests/Tooling/CompilationDatabaseTest.cpp
@@ -370,6 +370,28 @@
   EXPECT_EQ("command4", FoundCommand.CommandLine[0]) << ErrorMessage;
 }
 
+TEST(findCompileArgsInJsonDatabase, ParsesCompilerWrappers) {
+  std::vector<std::pair<std::string, std::string>> Cases = {
+      {"distcc gcc foo.c", "gcc foo.c"},
+      {"gomacc clang++ foo.c", "clang++ foo.c"},
+      {"ccache gcc foo.c", "gcc foo.c"},
+      {"ccache distcc gcc foo.c", "gcc foo.c"},
+
+      {"distcc foo.c", "distcc foo.c"},
+      {"distcc -I/foo/bar foo.c", "distcc -I/foo/bar foo.c"},
+  };
+  std::string ErrorMessage;
+
+  for (const auto &Case : Cases) {
+    std::string DB = R"([{"directory":".", "file":"/foo.c", "command":")" +
+                     Case.first + "\"}]";
+    CompileCommand FoundCommand =
+        findCompileArgsInJsonDatabase("/foo.c", DB, ErrorMessage);
+    EXPECT_EQ(Case.second, llvm::join(FoundCommand.CommandLine, " "))
+        << Case.first;
+  }
+}
+
 static std::vector<std::string> unescapeJsonCommandLine(StringRef Command) {
   std::string JsonDatabase =
     ("[{\"directory\":\"//net/root\", \"file\":\"test\", \"command\": \"" +
Index: clang/lib/Tooling/JSONCompilationDatabase.cpp
===================================================================
--- clang/lib/Tooling/JSONCompilationDatabase.cpp
+++ clang/lib/Tooling/JSONCompilationDatabase.cpp
@@ -256,15 +256,41 @@
   return Commands;
 }
 
+// There are compiler-wrappers (ccache, distcc, gomacc) that take the "real"
+// compiler as an argument, e.g. distcc gcc -O3 foo.c.
+// These end up in compile_commands.json when people set CC="distcc gcc".
+// Clang's driver doesn't understand this, so we need to unwrap.
+static bool unwrapCommand(std::vector<std::string> &Args) {
+  if (Args.size() < 2)
+    return false;
+  StringRef Wrapper = llvm::sys::path::filename(Args.front());
+  if (Wrapper == "distcc" || Wrapper == "gomacc" || Wrapper == "ccache") {
+    // The compiler arg is usually optional. The wrappers themselves don't take
+    // flags, so Args[1] is a compiler flag, an input file, or a compiler.
+    // Input files generally have extensions, compilers generally don't.
+    bool HasCompiler =
+        (Args[1][0] != '-') && !llvm::sys::path::has_extension(Args[1]);
+    if (HasCompiler) {
+      Args.erase(Args.begin());
+      return true;
+    }
+    // If !HasCompiler, wrappers act like GCC. Fine: so do we.
+  }
+  return false;
+}
+
 static std::vector<std::string>
 nodeToCommandLine(JSONCommandLineSyntax Syntax,
                   const std::vector<llvm::yaml::ScalarNode *> &Nodes) {
   SmallString<1024> Storage;
-  if (Nodes.size() == 1)
-    return unescapeCommandLine(Syntax, Nodes[0]->getValue(Storage));
   std::vector<std::string> Arguments;
-  for (const auto *Node : Nodes)
-    Arguments.push_back(Node->getValue(Storage));
+  if (Nodes.size() == 1)
+    Arguments = unescapeCommandLine(Syntax, Nodes[0]->getValue(Storage));
+  else
+    for (const auto *Node : Nodes)
+      Arguments.push_back(Node->getValue(Storage));
+  while (unwrapCommand(Arguments))
+    ;
   return Arguments;
 }
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D64297.208293.patch
Type: text/x-patch
Size: 3299 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190707/551589e0/attachment.bin>