<div dir="ltr">You could hack it to standardize the yaml output to forward slashes with sed, or clang might wish to do it internally.<div><br></div><div>My current position is that we should pick the path style and slash direction closest to the point at which the path will be emitted. In the past we've tried to make arguments that slashes should be internally one way or another depending on if it's Windows, MinGW, MSVC, or some other criteria, but the codebase is inconsistent. Now I lean towards canonicalizing when printing based on the format in use.</div><div>- For example, if emitting codeview, try to standardize to backslashes to match MSVC.</div><div>- If emitting DWARF, maybe use forward slashes since the consumer will probably be a gnu tool. </div><div>- When implementing -print-*-path in the driver, the user is probably a Makefile, so use forward slashes.</div><div>- When emitting this YAML stuff, it's our own format, so do whatever is convenient.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 30, 2019 at 4:03 PM Michael Spencer <<a href="mailto:bigcheesegs@gmail.com">bigcheesegs@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, Oct 30, 2019 at 3:54 PM Reid Kleckner via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I XFAILED the test on Windows in 52194350cfe. The [/\\] regexes might need to match two slashes, and the PREFIX variable ends up having inconsistent slash direction.</div></blockquote><div><br></div><div>Hmm, fixing the slashes should be easy, but I'm not sure there's a good way to get the PREFIX right. I feel like FileCheck needs an ignore slash direction mode.</div><div><br></div><div>- Michael Spencer<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 30, 2019 at 3:27 PM Michael Spencer via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Author: Michael Spencer<br>
Date: 2019-10-30T15:27:27-07:00<br>
New Revision: 33a745e6fe7e81d3793f7831d2832aa0785ef327<br>
<br>
URL: <a href="https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327" rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327</a><br>
DIFF: <a href="https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327.diff" rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327.diff</a><br>
<br>
LOG: [clang][clang-scan-deps] Add support for extracting full module dependencies.<br>
<br>
This is a recommit of d8a4ef0e685c with the nondeterminism fixed.<br>
<br>
This adds experimental support for extracting a Clang module dependency graph<br>
from a compilation database. The output format is experimental and will change.<br>
It is currently a concatenation of JSON outputs for each compilation. Future<br>
patches will change this to deduplicate modules between compilations.<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D69420" rel="noreferrer" target="_blank">https://reviews.llvm.org/D69420</a><br>
<br>
Added: <br>
clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h<br>
clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp<br>
clang/test/ClangScanDeps/modules-full.cpp<br>
<br>
Modified: <br>
clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h<br>
clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h<br>
clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h<br>
clang/lib/Tooling/DependencyScanning/CMakeLists.txt<br>
clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp<br>
clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp<br>
clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp<br>
clang/tools/clang-scan-deps/ClangScanDeps.cpp<br>
<br>
Removed: <br>
<br>
<br>
<br>
################################################################################<br>
diff --git a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h<br>
index fd8ed80b143c..76edf150dbee 100644<br>
--- a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h<br>
+++ b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h<br>
@@ -30,15 +30,30 @@ enum class ScanningMode {<br>
MinimizedSourcePreprocessing<br>
};<br>
<br>
+/// The format that is output by the dependency scanner.<br>
+enum class ScanningOutputFormat {<br>
+ /// This is the Makefile compatible dep format. This will include all of the<br>
+ /// deps necessary for an implicit modules build, but won't include any<br>
+ /// intermodule dependency information.<br>
+ Make,<br>
+<br>
+ /// This outputs the full module dependency graph suitable for use for<br>
+ /// explicitly building modules.<br>
+ Full,<br>
+};<br>
+<br>
/// The dependency scanning service contains the shared state that is used by<br>
/// the invidual dependency scanning workers.<br>
class DependencyScanningService {<br>
public:<br>
- DependencyScanningService(ScanningMode Mode, bool ReuseFileManager = true,<br>
+ DependencyScanningService(ScanningMode Mode, ScanningOutputFormat Format,<br>
+ bool ReuseFileManager = true,<br>
bool SkipExcludedPPRanges = true);<br>
<br>
ScanningMode getMode() const { return Mode; }<br>
<br>
+ ScanningOutputFormat getFormat() const { return Format; }<br>
+<br>
bool canReuseFileManager() const { return ReuseFileManager; }<br>
<br>
bool canSkipExcludedPPRanges() const { return SkipExcludedPPRanges; }<br>
@@ -49,6 +64,7 @@ class DependencyScanningService {<br>
<br>
private:<br>
const ScanningMode Mode;<br>
+ const ScanningOutputFormat Format;<br>
const bool ReuseFileManager;<br>
/// Set to true to use the preprocessor optimization that skips excluded PP<br>
/// ranges by bumping the buffer pointer in the lexer instead of lexing the<br>
<br>
diff --git a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h<br>
index c950cbe167cd..78b49e4fa0c5 100644<br>
--- a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h<br>
+++ b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h<br>
@@ -40,6 +40,7 @@ class DependencyScanningTool {<br>
StringRef CWD);<br>
<br>
private:<br>
+ const ScanningOutputFormat Format;<br>
DependencyScanningWorker Worker;<br>
const tooling::CompilationDatabase &Compilations;<br>
};<br>
<br>
diff --git a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h<br>
index 45c9fb4f029d..689119330c41 100644<br>
--- a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h<br>
+++ b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h<br>
@@ -15,6 +15,8 @@<br>
#include "clang/Frontend/PCHContainerOperations.h"<br>
#include "clang/Lex/PreprocessorExcludedConditionalDirectiveSkipMapping.h"<br>
#include "clang/Tooling/CompilationDatabase.h"<br>
+#include "clang/Tooling/DependencyScanning/DependencyScanningService.h"<br>
+#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h"<br>
#include "llvm/Support/Error.h"<br>
#include "llvm/Support/FileSystem.h"<br>
#include <string><br>
@@ -26,7 +28,6 @@ class DependencyOutputOptions;<br>
namespace tooling {<br>
namespace dependencies {<br>
<br>
-class DependencyScanningService;<br>
class DependencyScanningWorkerFilesystem;<br>
<br>
class DependencyConsumer {<br>
@@ -36,7 +37,9 @@ class DependencyConsumer {<br>
virtual void handleFileDependency(const DependencyOutputOptions &Opts,<br>
StringRef Filename) = 0;<br>
<br>
- // FIXME: Add support for reporting modular dependencies.<br>
+ virtual void handleModuleDependency(ModuleDeps MD) = 0;<br>
+<br>
+ virtual void handleContextHash(std::string Hash) = 0;<br>
};<br>
<br>
/// An individual dependency scanning worker that is able to run on its own<br>
@@ -73,6 +76,7 @@ class DependencyScanningWorker {<br>
/// The file manager that is reused accross multiple invocations by this<br>
/// worker. If null, the file manager will not be reused.<br>
llvm::IntrusiveRefCntPtr<FileManager> Files;<br>
+ ScanningOutputFormat Format;<br>
};<br>
<br>
} // end namespace dependencies<br>
<br>
diff --git a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h<br>
new file mode 100644<br>
index 000000000000..7a9fc276fcaa<br>
--- /dev/null<br>
+++ b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h<br>
@@ -0,0 +1,94 @@<br>
+//===- ModuleDepCollector.h - Callbacks to collect deps ---------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#ifndef LLVM_CLANG_TOOLING_DEPENDENCY_SCANNING_MODULE_DEP_COLLECTOR_H<br>
+#define LLVM_CLANG_TOOLING_DEPENDENCY_SCANNING_MODULE_DEP_COLLECTOR_H<br>
+<br>
+#include "clang/Basic/LLVM.h"<br>
+#include "clang/Basic/SourceManager.h"<br>
+#include "clang/Frontend/Utils.h"<br>
+#include "clang/Lex/HeaderSearch.h"<br>
+#include "clang/Lex/PPCallbacks.h"<br>
+#include "clang/Serialization/ASTReader.h"<br>
+#include "llvm/ADT/DenseMap.h"<br>
+#include "llvm/ADT/StringSet.h"<br>
+#include "llvm/Support/raw_ostream.h"<br>
+<br>
+#include <string><br>
+<br>
+namespace clang {<br>
+namespace tooling {<br>
+namespace dependencies {<br>
+<br>
+class DependencyConsumer;<br>
+<br>
+struct ModuleDeps {<br>
+ std::string ModuleName;<br>
+ std::string ClangModuleMapFile;<br>
+ std::string ModulePCMPath;<br>
+ std::string ContextHash;<br>
+ llvm::StringSet<> FileDeps;<br>
+ llvm::StringSet<> ClangModuleDeps;<br>
+ bool ImportedByMainFile = false;<br>
+};<br>
+<br>
+class ModuleDepCollector;<br>
+<br>
+class ModuleDepCollectorPP final : public PPCallbacks {<br>
+public:<br>
+ ModuleDepCollectorPP(CompilerInstance &I, ModuleDepCollector &MDC)<br>
+ : Instance(I), MDC(MDC) {}<br>
+<br>
+ void FileChanged(SourceLocation Loc, FileChangeReason Reason,<br>
+ SrcMgr::CharacteristicKind FileType,<br>
+ FileID PrevFID) override;<br>
+ void InclusionDirective(SourceLocation HashLoc, const Token &IncludeTok,<br>
+ StringRef FileName, bool IsAngled,<br>
+ CharSourceRange FilenameRange, const FileEntry *File,<br>
+ StringRef SearchPath, StringRef RelativePath,<br>
+ const Module *Imported,<br>
+ SrcMgr::CharacteristicKind FileType) override;<br>
+<br>
+ void EndOfMainFile() override;<br>
+<br>
+private:<br>
+ CompilerInstance &Instance;<br>
+ ModuleDepCollector &MDC;<br>
+ llvm::DenseSet<const Module *> DirectDeps;<br>
+<br>
+ void handleTopLevelModule(const Module *M);<br>
+ void addAllSubmoduleDeps(const Module *M, ModuleDeps &MD);<br>
+ void addModuleDep(const Module *M, ModuleDeps &MD);<br>
+<br>
+ void addDirectDependencies(const Module *Mod);<br>
+};<br>
+<br>
+class ModuleDepCollector final : public DependencyCollector {<br>
+public:<br>
+ ModuleDepCollector(CompilerInstance &I, DependencyConsumer &C);<br>
+<br>
+ void attachToPreprocessor(Preprocessor &PP) override;<br>
+ void attachToASTReader(ASTReader &R) override;<br>
+<br>
+private:<br>
+ friend ModuleDepCollectorPP;<br>
+<br>
+ CompilerInstance &Instance;<br>
+ DependencyConsumer &Consumer;<br>
+ std::string MainFile;<br>
+ std::string ContextHash;<br>
+ std::vector<std::string> MainDeps;<br>
+ std::unordered_map<std::string, ModuleDeps> Deps;<br>
+};<br>
+<br>
+} // end namespace dependencies<br>
+} // end namespace tooling<br>
+} // end namespace clang<br>
+<br>
+#endif // LLVM_CLANG_TOOLING_DEPENDENCY_SCANNING_MODULE_DEP_COLLECTOR_H<br>
<br>
diff --git a/clang/lib/Tooling/DependencyScanning/CMakeLists.txt b/clang/lib/Tooling/DependencyScanning/CMakeLists.txt<br>
index 05e1aa54f8d4..c6fe207ab2f2 100644<br>
--- a/clang/lib/Tooling/DependencyScanning/CMakeLists.txt<br>
+++ b/clang/lib/Tooling/DependencyScanning/CMakeLists.txt<br>
@@ -8,6 +8,7 @@ add_clang_library(clangDependencyScanning<br>
DependencyScanningService.cpp<br>
DependencyScanningWorker.cpp<br>
DependencyScanningTool.cpp<br>
+ ModuleDepCollector.cpp<br>
<br>
DEPENDS<br>
ClangDriverOptions<br>
<br>
diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp b/clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp<br>
index e5cebe381000..93bb0cde439d 100644<br>
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp<br>
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp<br>
@@ -12,8 +12,8 @@ using namespace clang;<br>
using namespace tooling;<br>
using namespace dependencies;<br>
<br>
-DependencyScanningService::DependencyScanningService(ScanningMode Mode,<br>
- bool ReuseFileManager,<br>
- bool SkipExcludedPPRanges)<br>
- : Mode(Mode), ReuseFileManager(ReuseFileManager),<br>
+DependencyScanningService::DependencyScanningService(<br>
+ ScanningMode Mode, ScanningOutputFormat Format, bool ReuseFileManager,<br>
+ bool SkipExcludedPPRanges)<br>
+ : Mode(Mode), Format(Format), ReuseFileManager(ReuseFileManager),<br>
SkipExcludedPPRanges(SkipExcludedPPRanges) {}<br>
<br>
diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp b/clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp<br>
index d2af1a9d110c..bffd7c338124 100644<br>
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp<br>
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp<br>
@@ -8,6 +8,15 @@<br>
<br>
#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h"<br>
#include "clang/Frontend/Utils.h"<br>
+#include "llvm/Support/JSON.h"<br>
+<br>
+static llvm::json::Array toJSONSorted(const llvm::StringSet<> &Set) {<br>
+ std::vector<llvm::StringRef> Strings;<br>
+ for (auto &&I : Set)<br>
+ Strings.push_back(I.getKey());<br>
+ std::sort(Strings.begin(), Strings.end());<br>
+ return llvm::json::Array(Strings);<br>
+}<br>
<br>
namespace clang{<br>
namespace tooling{<br>
@@ -16,13 +25,14 @@ namespace dependencies{<br>
DependencyScanningTool::DependencyScanningTool(<br>
DependencyScanningService &Service,<br>
const tooling::CompilationDatabase &Compilations)<br>
- : Worker(Service), Compilations(Compilations) {}<br>
+ : Format(Service.getFormat()), Worker(Service), Compilations(Compilations) {<br>
+}<br>
<br>
llvm::Expected<std::string><br>
DependencyScanningTool::getDependencyFile(const std::string &Input,<br>
StringRef CWD) {<br>
/// Prints out all of the gathered dependencies into a string.<br>
- class DependencyPrinterConsumer : public DependencyConsumer {<br>
+ class MakeDependencyPrinterConsumer : public DependencyConsumer {<br>
public:<br>
void handleFileDependency(const DependencyOutputOptions &Opts,<br>
StringRef File) override {<br>
@@ -31,6 +41,14 @@ DependencyScanningTool::getDependencyFile(const std::string &Input,<br>
Dependencies.push_back(File);<br>
}<br>
<br>
+ void handleModuleDependency(ModuleDeps MD) override {<br>
+ // These are ignored for the make format as it can't support the full<br>
+ // set of deps, and handleFileDependency handles enough for implicitly<br>
+ // built modules to work.<br>
+ }<br>
+<br>
+ void handleContextHash(std::string Hash) override {}<br>
+<br>
void printDependencies(std::string &S) {<br>
if (!Opts)<br>
return;<br>
@@ -59,14 +77,88 @@ DependencyScanningTool::getDependencyFile(const std::string &Input,<br>
std::vector<std::string> Dependencies;<br>
};<br>
<br>
- DependencyPrinterConsumer Consumer;<br>
- auto Result =<br>
- Worker.computeDependencies(Input, CWD, Compilations, Consumer);<br>
- if (Result)<br>
- return std::move(Result);<br>
- std::string Output;<br>
- Consumer.printDependencies(Output);<br>
- return Output;<br>
+ class FullDependencyPrinterConsumer : public DependencyConsumer {<br>
+ public:<br>
+ void handleFileDependency(const DependencyOutputOptions &Opts,<br>
+ StringRef File) override {<br>
+ Dependencies.push_back(File);<br>
+ }<br>
+<br>
+ void handleModuleDependency(ModuleDeps MD) override {<br>
+ ModuleDeps[MD.ContextHash + MD.ModuleName] = std::move(MD);<br>
+ }<br>
+<br>
+ void handleContextHash(std::string Hash) override {<br>
+ ContextHash = std::move(Hash);<br>
+ }<br>
+<br>
+ void printDependencies(std::string &S, StringRef MainFile) {<br>
+ // Sort the modules by name to get a deterministic order.<br>
+ std::vector<StringRef> Modules;<br>
+ for (auto &&Dep : ModuleDeps)<br>
+ Modules.push_back(Dep.first);<br>
+ std::sort(Modules.begin(), Modules.end());<br>
+<br>
+ llvm::raw_string_ostream OS(S);<br>
+<br>
+ using namespace llvm::json;<br>
+<br>
+ Array Imports;<br>
+ for (auto &&ModName : Modules) {<br>
+ auto &MD = ModuleDeps[ModName];<br>
+ if (MD.ImportedByMainFile)<br>
+ Imports.push_back(MD.ModuleName);<br>
+ }<br>
+<br>
+ Array Mods;<br>
+ for (auto &&ModName : Modules) {<br>
+ auto &MD = ModuleDeps[ModName];<br>
+ Object Mod{<br>
+ {"name", MD.ModuleName},<br>
+ {"file-deps", toJSONSorted(MD.FileDeps)},<br>
+ {"clang-module-deps", toJSONSorted(MD.ClangModuleDeps)},<br>
+ {"clang-modulemap-file", MD.ClangModuleMapFile},<br>
+ };<br>
+ Mods.push_back(std::move(Mod));<br>
+ }<br>
+<br>
+ Object O{<br>
+ {"input-file", MainFile},<br>
+ {"clang-context-hash", ContextHash},<br>
+ {"file-deps", Dependencies},<br>
+ {"clang-module-deps", std::move(Imports)},<br>
+ {"clang-modules", std::move(Mods)},<br>
+ };<br>
+<br>
+ S = llvm::formatv("{0:2},\n", Value(std::move(O))).str();<br>
+ return;<br>
+ }<br>
+<br>
+ private:<br>
+ std::vector<std::string> Dependencies;<br>
+ std::unordered_map<std::string, ModuleDeps> ModuleDeps;<br>
+ std::string ContextHash;<br>
+ };<br>
+<br>
+ if (Format == ScanningOutputFormat::Make) {<br>
+ MakeDependencyPrinterConsumer Consumer;<br>
+ auto Result =<br>
+ Worker.computeDependencies(Input, CWD, Compilations, Consumer);<br>
+ if (Result)<br>
+ return std::move(Result);<br>
+ std::string Output;<br>
+ Consumer.printDependencies(Output);<br>
+ return Output;<br>
+ } else {<br>
+ FullDependencyPrinterConsumer Consumer;<br>
+ auto Result =<br>
+ Worker.computeDependencies(Input, CWD, Compilations, Consumer);<br>
+ if (Result)<br>
+ return std::move(Result);<br>
+ std::string Output;<br>
+ Consumer.printDependencies(Output, Input);<br>
+ return Output;<br>
+ }<br>
}<br>
<br>
} // end namespace dependencies<br>
<br>
diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp b/clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp<br>
index f382c202f8c2..edf2cf8bd70f 100644<br>
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp<br>
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp<br>
@@ -14,6 +14,7 @@<br>
#include "clang/Frontend/Utils.h"<br>
#include "clang/Lex/PreprocessorOptions.h"<br>
#include "clang/Tooling/DependencyScanning/DependencyScanningService.h"<br>
+#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h"<br>
#include "clang/Tooling/Tooling.h"<br>
<br>
using namespace clang;<br>
@@ -72,9 +73,11 @@ class DependencyScanningAction : public tooling::ToolAction {<br>
DependencyScanningAction(<br>
StringRef WorkingDirectory, DependencyConsumer &Consumer,<br>
llvm::IntrusiveRefCntPtr<DependencyScanningWorkerFilesystem> DepFS,<br>
- ExcludedPreprocessorDirectiveSkipMapping *PPSkipMappings)<br>
+ ExcludedPreprocessorDirectiveSkipMapping *PPSkipMappings,<br>
+ ScanningOutputFormat Format)<br>
: WorkingDirectory(WorkingDirectory), Consumer(Consumer),<br>
- DepFS(std::move(DepFS)), PPSkipMappings(PPSkipMappings) {}<br>
+ DepFS(std::move(DepFS)), PPSkipMappings(PPSkipMappings),<br>
+ Format(Format) {}<br>
<br>
bool runInvocation(std::shared_ptr<CompilerInvocation> Invocation,<br>
FileManager *FileMgr,<br>
@@ -131,9 +134,20 @@ class DependencyScanningAction : public tooling::ToolAction {<br>
// We need at least one -MT equivalent for the generator to work.<br>
if (Opts->Targets.empty())<br>
Opts->Targets = {"clang-scan-deps dependency"};<br>
- Compiler.addDependencyCollector(<br>
- std::make_shared<DependencyConsumerForwarder>(std::move(Opts),<br>
- Consumer));<br>
+<br>
+ switch (Format) {<br>
+ case ScanningOutputFormat::Make:<br>
+ Compiler.addDependencyCollector(<br>
+ std::make_shared<DependencyConsumerForwarder>(std::move(Opts),<br>
+ Consumer));<br>
+ break;<br>
+ case ScanningOutputFormat::Full:<br>
+ Compiler.addDependencyCollector(<br>
+ std::make_shared<ModuleDepCollector>(Compiler, Consumer));<br>
+ break;<br>
+ }<br>
+<br>
+ Consumer.handleContextHash(Compiler.getInvocation().getModuleHash());<br>
<br>
auto Action = std::make_unique<PreprocessOnlyAction>();<br>
const bool Result = Compiler.ExecuteAction(*Action);<br>
@@ -147,12 +161,14 @@ class DependencyScanningAction : public tooling::ToolAction {<br>
DependencyConsumer &Consumer;<br>
llvm::IntrusiveRefCntPtr<DependencyScanningWorkerFilesystem> DepFS;<br>
ExcludedPreprocessorDirectiveSkipMapping *PPSkipMappings;<br>
+ ScanningOutputFormat Format;<br>
};<br>
<br>
} // end anonymous namespace<br>
<br>
DependencyScanningWorker::DependencyScanningWorker(<br>
- DependencyScanningService &Service) {<br>
+ DependencyScanningService &Service)<br>
+ : Format(Service.getFormat()) {<br>
DiagOpts = new DiagnosticOptions();<br>
PCHContainerOps = std::make_shared<PCHContainerOperations>();<br>
RealFS = new ProxyFileSystemWithoutChdir(llvm::vfs::getRealFileSystem());<br>
@@ -195,7 +211,7 @@ llvm::Error DependencyScanningWorker::computeDependencies(<br>
Tool.setPrintErrorMessage(false);<br>
Tool.setDiagnosticConsumer(&DC);<br>
DependencyScanningAction Action(WorkingDirectory, Consumer, DepFS,<br>
- PPSkipMappings.get());<br>
+ PPSkipMappings.get(), Format);<br>
return !Tool.run(&Action);<br>
});<br>
}<br>
<br>
diff --git a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp<br>
new file mode 100644<br>
index 000000000000..7f20ec7056c6<br>
--- /dev/null<br>
+++ b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp<br>
@@ -0,0 +1,136 @@<br>
+//===- ModuleDepCollector.cpp - Callbacks to collect deps -------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h"<br>
+<br>
+#include "clang/Frontend/CompilerInstance.h"<br>
+#include "clang/Lex/Preprocessor.h"<br>
+#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h"<br>
+<br>
+using namespace clang;<br>
+using namespace tooling;<br>
+using namespace dependencies;<br>
+<br>
+void ModuleDepCollectorPP::FileChanged(SourceLocation Loc,<br>
+ FileChangeReason Reason,<br>
+ SrcMgr::CharacteristicKind FileType,<br>
+ FileID PrevFID) {<br>
+ if (Reason != PPCallbacks::EnterFile)<br>
+ return;<br>
+<br>
+ SourceManager &SM = Instance.getSourceManager();<br>
+<br>
+ // Dependency generation really does want to go all the way to the<br>
+ // file entry for a source location to find out what is depended on.<br>
+ // We do not want #line markers to affect dependency generation!<br>
+ Optional<FileEntryRef> File =<br>
+ SM.getFileEntryRefForID(SM.getFileID(SM.getExpansionLoc(Loc)));<br>
+ if (!File)<br>
+ return;<br>
+<br>
+ StringRef FileName =<br>
+ llvm::sys::path::remove_leading_dotslash(File->getName());<br>
+<br>
+ MDC.MainDeps.push_back(FileName);<br>
+}<br>
+<br>
+void ModuleDepCollectorPP::InclusionDirective(<br>
+ SourceLocation HashLoc, const Token &IncludeTok, StringRef FileName,<br>
+ bool IsAngled, CharSourceRange FilenameRange, const FileEntry *File,<br>
+ StringRef SearchPath, StringRef RelativePath, const Module *Imported,<br>
+ SrcMgr::CharacteristicKind FileType) {<br>
+ if (!File && !Imported) {<br>
+ // This is a non-modular include that HeaderSearch failed to find. Add it<br>
+ // here as `FileChanged` will never see it.<br>
+ MDC.MainDeps.push_back(FileName);<br>
+ }<br>
+<br>
+ if (!Imported)<br>
+ return;<br>
+<br>
+ MDC.Deps[MDC.ContextHash + Imported->getTopLevelModule()->getFullModuleName()]<br>
+ .ImportedByMainFile = true;<br>
+ DirectDeps.insert(Imported->getTopLevelModule());<br>
+}<br>
+<br>
+void ModuleDepCollectorPP::EndOfMainFile() {<br>
+ FileID MainFileID = Instance.getSourceManager().getMainFileID();<br>
+ MDC.MainFile =<br>
+ Instance.getSourceManager().getFileEntryForID(MainFileID)->getName();<br>
+<br>
+ for (const Module *M : DirectDeps) {<br>
+ handleTopLevelModule(M);<br>
+ }<br>
+<br>
+ for (auto &&I : MDC.Deps)<br>
+ MDC.Consumer.handleModuleDependency(I.second);<br>
+<br>
+ DependencyOutputOptions Opts;<br>
+ for (auto &&I : MDC.MainDeps)<br>
+ MDC.Consumer.handleFileDependency(Opts, I);<br>
+}<br>
+<br>
+void ModuleDepCollectorPP::handleTopLevelModule(const Module *M) {<br>
+ assert(M == M->getTopLevelModule() && "Expected top level module!");<br>
+<br>
+ auto ModI = MDC.Deps.insert(<br>
+ std::make_pair(MDC.ContextHash + M->getFullModuleName(), ModuleDeps{}));<br>
+<br>
+ if (!ModI.first->second.ModuleName.empty())<br>
+ return;<br>
+<br>
+ ModuleDeps &MD = ModI.first->second;<br>
+<br>
+ const FileEntry *ModuleMap = Instance.getPreprocessor()<br>
+ .getHeaderSearchInfo()<br>
+ .getModuleMap()<br>
+ .getContainingModuleMapFile(M);<br>
+<br>
+ MD.ClangModuleMapFile = ModuleMap ? ModuleMap->getName() : "";<br>
+ MD.ModuleName = M->getFullModuleName();<br>
+ MD.ModulePCMPath = M->getASTFile()->getName();<br>
+ MD.ContextHash = MDC.ContextHash;<br>
+ serialization::ModuleFile *MF =<br>
+ MDC.Instance.getModuleManager()->getModuleManager().lookup(<br>
+ M->getASTFile());<br>
+ MDC.Instance.getModuleManager()->visitInputFiles(<br>
+ *MF, true, true, [&](const serialization::InputFile &IF, bool isSystem) {<br>
+ MD.FileDeps.insert(IF.getFile()->getName());<br>
+ });<br>
+<br>
+ addAllSubmoduleDeps(M, MD);<br>
+}<br>
+<br>
+void ModuleDepCollectorPP::addAllSubmoduleDeps(const Module *M,<br>
+ ModuleDeps &MD) {<br>
+ addModuleDep(M, MD);<br>
+<br>
+ for (const Module *SubM : M->submodules())<br>
+ addAllSubmoduleDeps(SubM, MD);<br>
+}<br>
+<br>
+void ModuleDepCollectorPP::addModuleDep(const Module *M, ModuleDeps &MD) {<br>
+ for (const Module *Import : M->Imports) {<br>
+ if (Import->getTopLevelModule() != M->getTopLevelModule()) {<br>
+ MD.ClangModuleDeps.insert(Import->getTopLevelModuleName());<br>
+ handleTopLevelModule(Import->getTopLevelModule());<br>
+ }<br>
+ }<br>
+}<br>
+<br>
+ModuleDepCollector::ModuleDepCollector(CompilerInstance &I,<br>
+ DependencyConsumer &C)<br>
+ : Instance(I), Consumer(C), ContextHash(I.getInvocation().getModuleHash()) {<br>
+}<br>
+<br>
+void ModuleDepCollector::attachToPreprocessor(Preprocessor &PP) {<br>
+ PP.addPPCallbacks(std::make_unique<ModuleDepCollectorPP>(Instance, *this));<br>
+}<br>
+<br>
+void ModuleDepCollector::attachToASTReader(ASTReader &R) {}<br>
<br>
diff --git a/clang/test/ClangScanDeps/modules-full.cpp b/clang/test/ClangScanDeps/modules-full.cpp<br>
new file mode 100644<br>
index 000000000000..f8bff06d8f74<br>
--- /dev/null<br>
+++ b/clang/test/ClangScanDeps/modules-full.cpp<br>
@@ -0,0 +1,74 @@<br>
+// RUN: rm -rf %t.dir<br>
+// RUN: rm -rf %t.cdb<br>
+// RUN: rm -rf %t.module-cache<br>
+// RUN: mkdir -p %t.dir<br>
+// RUN: cp %s %t.dir/modules_cdb_input.cpp<br>
+// RUN: cp %s %t.dir/modules_cdb_input2.cpp<br>
+// RUN: mkdir %t.dir/Inputs<br>
+// RUN: cp %S/Inputs/header.h %t.dir/Inputs/header.h<br>
+// RUN: cp %S/Inputs/header2.h %t.dir/Inputs/header2.h<br>
+// RUN: cp %S/Inputs/module.modulemap %t.dir/Inputs/module.modulemap<br>
+// RUN: sed -e "s|DIR|%/t.dir|g" %S/Inputs/modules_cdb.json > %t.cdb<br>
+//<br>
+// RUN: echo %t.dir > %t.result<br>
+// RUN: clang-scan-deps -compilation-database %t.cdb -j 1 \<br>
+// RUN: -mode preprocess-minimized-sources -format experimental-full >> %t.result<br>
+// RUN: cat %t.result | FileCheck --check-prefixes=CHECK %s<br>
+<br>
+#include "header.h"<br>
+<br>
+// CHECK: [[PREFIX:(.*[/\\])+[a-zA-Z0-9.-]+]]<br>
+// CHECK-NEXT: {<br>
+// CHECK-NEXT: "clang-context-hash": "[[CONTEXT_HASH:[A-Z0-9]+]]",<br>
+// CHECK-NEXT: "clang-module-deps": [<br>
+// CHECK-NEXT: "header1"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "clang-modules": [<br>
+// CHECK-NEXT: {<br>
+// CHECK-NEXT: "clang-module-deps": [<br>
+// CHECK-NEXT: "header2"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "clang-modulemap-file": "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}module.modulemap",<br>
+// CHECK-NEXT: "file-deps": [<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}header.h",<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}module.modulemap"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "name": "header1"<br>
+// CHECK-NEXT: },<br>
+// CHECK-NEXT: {<br>
+// CHECK-NEXT: "clang-module-deps": [],<br>
+// CHECK-NEXT: "clang-modulemap-file": "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}module.modulemap",<br>
+// CHECK-NEXT: "file-deps": [<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}header2.h",<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}module.modulemap"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "name": "header2"<br>
+// CHECK-NEXT: }<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "file-deps": [<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}modules_cdb_input2.cpp"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "input-file": "[[PREFIX]]{{[/\\]}}modules_cdb_input2.cpp"<br>
+// CHECK-NEXT:},<br>
+// CHECK-NEXT:{<br>
+// CHECK-NOT: "clang-context-hash": "[[CONTEXT_HASH]]",<br>
+// CHECK-NEXT: "clang-context-hash": "{{[A-Z0-9]+}}",<br>
+// CHECK-NEXT: "clang-module-deps": [<br>
+// CHECK-NEXT: "header1"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "clang-modules": [<br>
+// CHECK-NEXT: {<br>
+// CHECK-NEXT: "clang-module-deps": [],<br>
+// CHECK-NEXT: "clang-modulemap-file": "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}module.modulemap",<br>
+// CHECK-NEXT: "file-deps": [<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}header.h",<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}Inputs{{[/\\]}}module.modulemap"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "name": "header1"<br>
+// CHECK-NEXT: }<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "file-deps": [<br>
+// CHECK-NEXT: "[[PREFIX]]{{[/\\]}}modules_cdb_input.cpp"<br>
+// CHECK-NEXT: ],<br>
+// CHECK-NEXT: "input-file": "[[PREFIX]]{{[/\\]}}modules_cdb_input.cpp"<br>
+// CHECK-NEXT:},<br>
<br>
diff --git a/clang/tools/clang-scan-deps/ClangScanDeps.cpp b/clang/tools/clang-scan-deps/ClangScanDeps.cpp<br>
index d57983ed1664..a6abb4a9600b 100644<br>
--- a/clang/tools/clang-scan-deps/ClangScanDeps.cpp<br>
+++ b/clang/tools/clang-scan-deps/ClangScanDeps.cpp<br>
@@ -59,6 +59,17 @@ static llvm::cl::opt<ScanningMode> ScanMode(<br>
llvm::cl::init(ScanningMode::MinimizedSourcePreprocessing),<br>
llvm::cl::cat(DependencyScannerCategory));<br>
<br>
+static llvm::cl::opt<ScanningOutputFormat> Format(<br>
+ "format", llvm::cl::desc("The output format for the dependencies"),<br>
+ llvm::cl::values(clEnumValN(ScanningOutputFormat::Make, "make",<br>
+ "Makefile compatible dep file"),<br>
+ clEnumValN(ScanningOutputFormat::Full, "experimental-full",<br>
+ "Full dependency graph suitable"<br>
+ " for explicitly building modules. This format "<br>
+ "is experimental and will change.")),<br>
+ llvm::cl::init(ScanningOutputFormat::Make),<br>
+ llvm::cl::cat(DependencyScannerCategory));<br>
+<br>
llvm::cl::opt<unsigned><br>
NumThreads("j", llvm::cl::Optional,<br>
llvm::cl::desc("Number of worker threads to use (default: use "<br>
@@ -200,7 +211,7 @@ int main(int argc, const char **argv) {<br>
// Print out the dependency results to STDOUT by default.<br>
SharedStream DependencyOS(llvm::outs());<br>
<br>
- DependencyScanningService Service(ScanMode, ReuseFileManager,<br>
+ DependencyScanningService Service(ScanMode, Format, ReuseFileManager,<br>
SkipExcludedPPRanges);<br>
#if LLVM_ENABLE_THREADS<br>
unsigned NumWorkers =<br>
<br>
<br>
<br>
_______________________________________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br>
</blockquote></div>
_______________________________________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br>
</blockquote></div></div>
</blockquote></div>