[clang] [clang-scan-deps] Expand response files before the argument adjuster (PR #89950)

[Alexandre Ganea via cfe-commits]

https://github.com/aganea updated https://github.com/llvm/llvm-project/pull/89950

>From f2340c98c95e0d72516fc240ff268fead9f15391 Mon Sep 17 00:00:00 2001
From: Alexandre Ganea <aganea at havenstudios.com>
Date: Wed, 17 Apr 2024 16:28:21 -0400
Subject: [PATCH 1/3] [clang-scan-deps] Expand response files before the
 argument ajuster

Previously, since response (.rsp) files weren't expanded, we only parsed
the command-line as provided in the Clang CDB file. Unfortunately, when
using Unreal Engine, arguments are always generated in a .rsp file.

After this patch, /Fo can be parsed and added to the final command-line.
Without this option, the make targets that are emitted are made up from the
input file name alone. We have some cases where the same input in the project
generates several output files, so we end up with duplicate make targets
in the scan-deps emitted dependency file.
---
 clang/tools/clang-scan-deps/ClangScanDeps.cpp | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/clang/tools/clang-scan-deps/ClangScanDeps.cpp b/clang/tools/clang-scan-deps/ClangScanDeps.cpp
index f42af7e330e17a..7b7f10c4be7421 100644
--- a/clang/tools/clang-scan-deps/ClangScanDeps.cpp
+++ b/clang/tools/clang-scan-deps/ClangScanDeps.cpp
@@ -792,10 +792,15 @@ int clang_scan_deps_main(int argc, char **argv, const llvm::ToolContext &) {
 
   llvm::cl::PrintOptionValues();
 
+  // Expand response files in advance, so that we can "see" all the arguments
+  // when adjusting below.
+  auto ResponseExpander = expandResponseFiles(std::move(Compilations),
+                                              llvm::vfs::getRealFileSystem());
+
   // The command options are rewritten to run Clang in preprocessor only mode.
   auto AdjustingCompilations =
       std::make_unique<tooling::ArgumentsAdjustingCompilations>(
-          std::move(Compilations));
+          std::move(ResponseExpander));
   ResourceDirectoryCache ResourceDirCache;
 
   AdjustingCompilations->appendArgumentsAdjuster(

>From 09e0595fa9f3c9795d6f40a7308d8a912254b1df Mon Sep 17 00:00:00 2001
From: Alexandre Ganea <aganea at havenstudios.com>
Date: Wed, 24 Apr 2024 13:52:55 -0400
Subject: [PATCH 2/3] Add test

---
 .../ClangScanDeps/response-file-clang-cl.c    | 32 +++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 clang/test/ClangScanDeps/response-file-clang-cl.c

diff --git a/clang/test/ClangScanDeps/response-file-clang-cl.c b/clang/test/ClangScanDeps/response-file-clang-cl.c
new file mode 100644
index 00000000000000..78e3d15deb1678
--- /dev/null
+++ b/clang/test/ClangScanDeps/response-file-clang-cl.c
@@ -0,0 +1,32 @@
+// Check that the scanner can adjust arguments by reading .rsp files in advance.
+
+// RUN: rm -rf %t
+// RUN: split-file %s %t
+// RUN: sed -e "s|DIR|%/t|g" %t/cdb.json.template > %t/cdb.json
+// RUN: echo /Fo%t/tu.obj >> %t/args_nested.rsp
+
+// RUN: clang-scan-deps -compilation-database %t/cdb.json > %t/deps.json
+
+// RUN: cat %t/deps.json | sed 's:\\\\\?:/:g' | FileCheck -DPREFIX=%/t %s
+
+// Here we ensure that we got a qualified .obj with its full path, since that's what we're passing with /Fo
+// CHECK: [[PREFIX]]/tu.obj:
+
+//--- cdb.json.template
+[{
+  "file": "DIR/t.cpp",
+  "directory": "DIR",
+  "command": "clang-cl @DIR/args.rsp"
+}]
+
+//--- args.rsp
+ at args_nested.rsp
+/c tu.cpp
+
+//--- args_nested.rsp
+/I include
+
+//--- include/header.h
+
+//--- tu.cpp
+#include "header.h"

>From 812a14f882995b832599222f0a747761378d31b8 Mon Sep 17 00:00:00 2001
From: Alexandre Ganea <aganea at havenstudios.com>
Date: Wed, 24 Apr 2024 17:02:17 -0400
Subject: [PATCH 3/3] Handle /Fo in the Clang driver as well.

---
 clang/lib/Driver/ToolChains/Clang.cpp             | 2 +-
 clang/test/ClangScanDeps/response-file-clang-cl.c | 6 +++++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 5f5d720cf759f4..69f888fcf323a4 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -1069,7 +1069,7 @@ void Clang::AddPreprocessingOptions(Compilation &C, const JobAction &JA,
 
       // If user provided -o, that is the dependency target, except
       // when we are only generating a dependency file.
-      Arg *OutputOpt = Args.getLastArg(options::OPT_o);
+      Arg *OutputOpt = Args.getLastArg(options::OPT_o, options::OPT__SLASH_Fo);
       if (OutputOpt && Output.getType() != types::TY_Dependencies) {
         DepTarget = OutputOpt->getValue();
       } else {
diff --git a/clang/test/ClangScanDeps/response-file-clang-cl.c b/clang/test/ClangScanDeps/response-file-clang-cl.c
index 78e3d15deb1678..77cecfff0b9ca8 100644
--- a/clang/test/ClangScanDeps/response-file-clang-cl.c
+++ b/clang/test/ClangScanDeps/response-file-clang-cl.c
@@ -5,8 +5,12 @@
 // RUN: sed -e "s|DIR|%/t|g" %t/cdb.json.template > %t/cdb.json
 // RUN: echo /Fo%t/tu.obj >> %t/args_nested.rsp
 
+// RUN: echo /c >> %t/args_nested.rsp
 // RUN: clang-scan-deps -compilation-database %t/cdb.json > %t/deps.json
+// RUN: cat %t/deps.json | sed 's:\\\\\?:/:g' | FileCheck -DPREFIX=%/t %s
 
+// RUN: echo /E >> %t/args_nested.rsp
+// RUN: clang-scan-deps -compilation-database %t/cdb.json > %t/deps.json
 // RUN: cat %t/deps.json | sed 's:\\\\\?:/:g' | FileCheck -DPREFIX=%/t %s
 
 // Here we ensure that we got a qualified .obj with its full path, since that's what we're passing with /Fo
@@ -21,7 +25,7 @@
 
 //--- args.rsp
 @args_nested.rsp
-/c tu.cpp
+tu.cpp
 
 //--- args_nested.rsp
 /I include