[clang] [clang-tools-extra] [clangd] [C++20] [Modules] Build Database Support (PR #199384)
Matthew Asplund via cfe-commits
cfe-commits at lists.llvm.org
Sat May 23 16:27:16 PDT 2026
https://github.com/mwasplund created https://github.com/llvm/llvm-project/pull/199384
## **DRAFT**
Sending initial PR to get feedback on the high level direction for the change. Once we agree on the architecture I will do a polish pass and update this PR for final review.
### What
Extending Support for Modules as outlined in https://github.com/clangd/clangd/issues/1293. Add support for loading [build database files](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2977r2.html) as a preferred option for understanding compilation commands and module dependencies..
### Why
For the first time C++ Modules introduced a dependency on compilation artifacts that must be up to date and compatible with the tooling that consumes them. Build systems are responsible for ensuring binary module interfaces are compiled before they are consumed and properly resolved during builds, however tooling also shares this problem. We can attempt to rebuild the dependency graph by parsing the compilation flags along with raw source scanning, however this information is incomplete and expensive to scan all files in a project. It is preferable to have the build system explicitly define these dependencies in a build_database.json as it is the single source of truth for compiling and resolving imports and the json file can function as a cache for fast LSP initialization.
### How
* Extend the CompilationDatabse interface to expose a new ModuleManager interface. These functions mirror the functions that exists in the ProjectModules to effectively push down the logic to the database when possible (the database itself has the info required).
* This is the design decision that I would like the most feedback on. I found the nullable getter to be the simplest first pass as it did not disrupt too much existing code and allowed me to easily check for the ability to resolve module references.
* We could also extend the CompilationDatabase itself to handle module resolution and allow legacy database implementations to ignore this new functionality.
* We could also extend the CompilationDatabase with a child interface for BuildDatabase that contains these new functions. This design would cause a lot of runtime checks to propagate down the usage flow for knowing which interface to use and there are a lot of wrappers for the CompilationDatabase interface.
* Create new BuildDatabaseProjectModules implementation that resolves module to source file lookups using the new ModuleManager interfaces if the CDB in references has a ModuleManager available.
* Update CompoundProjectModules to prioritize using the BuildDatabase over the CompileCommands and still fall back and validate using the scanner.
* Create new JSONBuildDatabase that builds on the JSONCompilationDatabase to parse the [schema for build database json files](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2977r2.html).
* Update the DirectoryBasedGlobalCompilationDatabase to prioritize loading the build_database.json file in the root or `build` directory over the compile_commands.json file.
### Tested
* Ran check-clangd
* Verified clangd in neovim for cmake generated sample
* Verified clangd in neovim for complex soup generated project. Can now resolve outside project structure.
### TODO
* Cleanup format (my clang tidy seems wrong)
* Add tests
* Deduplicate shared logic in json loaders.
>From 477b3130e1b64160268e480d61568ade76993127 Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Wed, 13 May 2026 12:33:48 -0700
Subject: [PATCH 01/10] Move Compilation Database out of module file cache. It
is not used in this class at all.
---
clang-tools-extra/clangd/ModulesBuilder.cpp | 11 +++++------
clang-tools-extra/clangd/ModulesBuilder.h | 2 +-
2 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/clang-tools-extra/clangd/ModulesBuilder.cpp b/clang-tools-extra/clangd/ModulesBuilder.cpp
index 706fd459e15ec..62d5e21f7c857 100644
--- a/clang-tools-extra/clangd/ModulesBuilder.cpp
+++ b/clang-tools-extra/clangd/ModulesBuilder.cpp
@@ -730,8 +730,7 @@ bool ReusablePrerequisiteModules::canReuse(
/// variants do not collide.
class ModuleFileCache {
public:
- ModuleFileCache(const GlobalCompilationDatabase &CDB) : CDB(CDB) {}
- const GlobalCompilationDatabase &getCDB() const { return CDB; }
+ ModuleFileCache() {}
std::shared_ptr<const ModuleFile> getModule(StringRef ModuleName,
PathRef ModuleUnitSource,
@@ -762,8 +761,6 @@ class ModuleFileCache {
return Key;
}
- const GlobalCompilationDatabase &CDB;
-
llvm::StringMap<std::weak_ptr<const ModuleFile>> ModuleFiles;
std::mutex ModuleFilesMutex;
};
@@ -1008,12 +1005,13 @@ void garbageCollectModuleCache(PathRef CacheRoot) {
class ModulesBuilder::ModulesBuilderImpl {
public:
- ModulesBuilderImpl(const GlobalCompilationDatabase &CDB) : Cache(CDB) {}
+ ModulesBuilderImpl(const GlobalCompilationDatabase &CDB)
+ : CDB(CDB), Cache() {}
ModuleNameToSourceCache &getProjectModulesCache() {
return ProjectModulesCache;
}
- const GlobalCompilationDatabase &getCDB() const { return Cache.getCDB(); }
+ const GlobalCompilationDatabase &getCDB() const { return CDB; }
llvm::Error
getOrBuildModuleFile(PathRef RequiredSource, StringRef ModuleName,
@@ -1029,6 +1027,7 @@ class ModulesBuilder::ModulesBuilderImpl {
/// Runs GC once for the cache root owning a project root.
void garbageCollectModuleCacheForProjectRoot(PathRef ProjectRoot);
+ const GlobalCompilationDatabase &CDB;
ModuleFileCache Cache;
ModuleNameToSourceCache ProjectModulesCache;
std::mutex GarbageCollectedProjectRootsMutex;
diff --git a/clang-tools-extra/clangd/ModulesBuilder.h b/clang-tools-extra/clangd/ModulesBuilder.h
index b0e110b92b6a7..f91e2a4720c46 100644
--- a/clang-tools-extra/clangd/ModulesBuilder.h
+++ b/clang-tools-extra/clangd/ModulesBuilder.h
@@ -81,7 +81,7 @@ class PrerequisiteModules {
/// This class handles building module files for a given source file.
///
-/// In the future, we want the class to manage the module files acorss
+/// In the future, we want the class to manage the module files across
/// different versions and different source files.
class ModulesBuilder {
public:
>From 2976b3cc5cb987a5e6c3b658a8c514dcef2c8b26 Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Wed, 13 May 2026 22:13:11 -0700
Subject: [PATCH 02/10] Create build database clone from json compilation
database
---
clang-tools-extra/clangd/ClangdLSPServer.cpp | 56 ++--
.../clangd/GlobalCompilationDatabase.cpp | 37 ++-
.../include/clang/Tooling/JSONBuildDatabase.h | 137 ++++++++
clang/lib/Tooling/CMakeLists.txt | 1 +
clang/lib/Tooling/CompilationDatabase.cpp | 35 +-
clang/lib/Tooling/JSONBuildDatabase.cpp | 307 ++++++++++++++++++
clang/lib/Tooling/JSONCompilationDatabase.cpp | 58 ++--
7 files changed, 556 insertions(+), 75 deletions(-)
create mode 100644 clang/include/clang/Tooling/JSONBuildDatabase.h
create mode 100644 clang/lib/Tooling/JSONBuildDatabase.cpp
diff --git a/clang-tools-extra/clangd/ClangdLSPServer.cpp b/clang-tools-extra/clangd/ClangdLSPServer.cpp
index 04f58ab6446d1..9d08352479106 100644
--- a/clang-tools-extra/clangd/ClangdLSPServer.cpp
+++ b/clang-tools-extra/clangd/ClangdLSPServer.cpp
@@ -136,10 +136,9 @@ CodeAction toCodeAction(const Fix &F, const URIForFile &File,
Edit.textDocument = VersionedTextDocumentIdentifier{{File}, Version};
for (const auto &E : F.Edits)
Edit.edits.push_back(
- {E.range, E.newText,
- SupportChangeAnnotation ? E.annotationId : ""});
+ {E.range, E.newText, SupportChangeAnnotation ? E.annotationId : ""});
if (SupportChangeAnnotation) {
- for (const auto &[AID, Annotation]: F.Annotations)
+ for (const auto &[AID, Annotation] : F.Annotations)
Action.edit->changeAnnotations[AID] = Annotation;
}
}
@@ -908,24 +907,24 @@ void ClangdLSPServer::onRename(const RenameParams &Params,
if (!Server->getDraft(File))
return Reply(llvm::make_error<LSPError>(
"onRename called for non-added file", ErrorCode::InvalidParams));
- Server->rename(File, Params.position, Params.newName, Opts.Rename,
- [File, Params, Reply = std::move(Reply),
- this](llvm::Expected<RenameResult> R) mutable {
- if (!R)
- return Reply(R.takeError());
- if (auto Err = validateEdits(*Server, R->GlobalChanges))
- return Reply(std::move(Err));
- WorkspaceEdit Result;
- // FIXME: use documentChanges if SupportDocumentChanges is
- // true.
- Result.changes.emplace();
- for (const auto &Rep : R->GlobalChanges) {
- (*Result
- .changes)[URI::createFile(Rep.first()).toString()] =
- Rep.second.asTextEdits();
- }
- Reply(Result);
- });
+ Server->rename(
+ File, Params.position, Params.newName, Opts.Rename,
+ [File, Params, Reply = std::move(Reply),
+ this](llvm::Expected<RenameResult> R) mutable {
+ if (!R)
+ return Reply(R.takeError());
+ if (auto Err = validateEdits(*Server, R->GlobalChanges))
+ return Reply(std::move(Err));
+ WorkspaceEdit Result;
+ // FIXME: use documentChanges if SupportDocumentChanges is
+ // true.
+ Result.changes.emplace();
+ for (const auto &Rep : R->GlobalChanges) {
+ (*Result.changes)[URI::createFile(Rep.first()).toString()] =
+ Rep.second.asTextEdits();
+ }
+ Reply(Result);
+ });
}
void ClangdLSPServer::onDocumentDidClose(
@@ -1069,7 +1068,7 @@ void ClangdLSPServer::onCodeAction(const CodeActionParams &Params,
std::map<ClangdServer::DiagRef, clangd::Diagnostic> ToLSPDiags;
ClangdServer::CodeActionInputs Inputs;
- for (const auto& LSPDiag : Params.context.diagnostics) {
+ for (const auto &LSPDiag : Params.context.diagnostics) {
if (auto DiagRef = getDiagRef(File.file(), LSPDiag)) {
ToLSPDiags[*DiagRef] = LSPDiag;
Inputs.Diagnostics.push_back(*DiagRef);
@@ -1078,13 +1077,9 @@ void ClangdLSPServer::onCodeAction(const CodeActionParams &Params,
Inputs.File = File.file();
Inputs.Selection = Params.range;
Inputs.RequestedActionKinds = Params.context.only;
- Inputs.TweakFilter = [this](const Tweak &T) {
- return Opts.TweakFilter(T);
- };
- auto CB = [this,
- Reply = std::move(Reply),
- ToLSPDiags = std::move(ToLSPDiags), File,
- Selection = Params.range](
+ Inputs.TweakFilter = [this](const Tweak &T) { return Opts.TweakFilter(T); };
+ auto CB = [this, Reply = std::move(Reply), ToLSPDiags = std::move(ToLSPDiags),
+ File, Selection = Params.range](
llvm::Expected<ClangdServer::CodeActionResult> Fixits) mutable {
if (!Fixits)
return Reply(Fixits.takeError());
@@ -1093,8 +1088,7 @@ void ClangdLSPServer::onCodeAction(const CodeActionParams &Params,
for (const auto &QF : Fixits->QuickFixes) {
CAs.push_back(toCodeAction(QF.F, File, Version, SupportsDocumentChanges,
SupportsChangeAnnotation));
- if (auto It = ToLSPDiags.find(QF.Diag);
- It != ToLSPDiags.end()) {
+ if (auto It = ToLSPDiags.find(QF.Diag); It != ToLSPDiags.end()) {
CAs.back().diagnostics = {It->second};
}
}
diff --git a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
index adb771ecbbaad..8ba66ca41d312 100644
--- a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
+++ b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
@@ -18,6 +18,7 @@
#include "clang/Tooling/ArgumentsAdjusters.h"
#include "clang/Tooling/CompilationDatabase.h"
#include "clang/Tooling/CompilationDatabasePluginRegistry.h"
+#include "clang/Tooling/JSONBuildDatabase.h"
#include "clang/Tooling/JSONCompilationDatabase.h"
#include "clang/Tooling/Tooling.h"
#include "llvm/ADT/PointerIntPair.h"
@@ -133,6 +134,8 @@ class DirectoryBasedGlobalCompilationDatabase::DirectoryCache {
// shared_ptr so we can overwrite this when callers are still using the CDB.
std::shared_ptr<tooling::CompilationDatabase> CDB;
// File metadata for the CDB files we support tracking directly.
+ CachedFile BuildDatabaseJson;
+ CachedFile BuildBuildDatabaseJson;
CachedFile CompileCommandsJson;
CachedFile BuildCompileCommandsJson;
CachedFile CompileFlagsTxt;
@@ -146,7 +149,9 @@ class DirectoryBasedGlobalCompilationDatabase::DirectoryCache {
public:
DirectoryCache(llvm::StringRef Path)
- : CompileCommandsJson(Path, "compile_commands.json"),
+ : BuildDatabaseJson(Path, "build_database.json"),
+ BuildBuildDatabaseJson(Path, "build/build_database.json"),
+ CompileCommandsJson(Path, "compile_commands.json"),
BuildCompileCommandsJson(Path, "build/compile_commands.json"),
CompileFlagsTxt(Path, "compile_flags.txt"), Path(Path) {
assert(llvm::sys::path::is_absolute(Path));
@@ -247,7 +252,25 @@ DirectoryBasedGlobalCompilationDatabase::DirectoryCache::CachedFile::load(
// Adapt CDB-loading functions to a common interface for DirectoryCache::load().
static std::unique_ptr<tooling::CompilationDatabase>
-parseJSON(PathRef Path, llvm::StringRef Data, std::string &Error) {
+parseBuildDatabaseJSON(PathRef Path, llvm::StringRef Data, std::string &Error) {
+ if (auto CDB = tooling::JSONCompilationDatabase::loadFromBuffer(
+ Data, Error, tooling::JSONCommandLineSyntax::AutoDetect)) {
+ // FS used for expanding response files.
+ // FIXME: ExpandResponseFilesDatabase appears not to provide the usual
+ // thread-safety guarantees, as the access to FS is not locked!
+ // For now, use the real FS, which is known to be threadsafe (if we don't
+ // use/change working directory, which ExpandResponseFilesDatabase doesn't).
+ // NOTE: response files have to be expanded before inference because
+ // inference needs full command line to check/fix driver mode and file type.
+ auto FS = llvm::vfs::getRealFileSystem();
+ return tooling::inferMissingCompileCommands(
+ expandResponseFiles(std::move(CDB), std::move(FS)));
+ }
+ return nullptr;
+}
+static std::unique_ptr<tooling::CompilationDatabase>
+parseComplilationCommandsJSON(PathRef Path, llvm::StringRef Data,
+ std::string &Error) {
if (auto CDB = tooling::JSONCompilationDatabase::loadFromBuffer(
Data, Error, tooling::JSONCommandLineSyntax::AutoDetect)) {
// FS used for expanding response files.
@@ -286,9 +309,12 @@ bool DirectoryBasedGlobalCompilationDatabase::DirectoryCache::load(
/*Data*/ llvm::StringRef,
/*ErrorMsg*/ std::string &);
};
- for (const auto &Entry : {CDBFile{&CompileCommandsJson, parseJSON},
- CDBFile{&BuildCompileCommandsJson, parseJSON},
- CDBFile{&CompileFlagsTxt, parseFixed}}) {
+ for (const auto &Entry :
+ {CDBFile{&BuildDatabaseJson, parseBuildDatabaseJSON},
+ CDBFile{&BuildBuildDatabaseJson, parseBuildDatabaseJSON},
+ CDBFile{&CompileCommandsJson, parseComplilationCommandsJSON},
+ CDBFile{&BuildCompileCommandsJson, parseComplilationCommandsJSON},
+ CDBFile{&CompileFlagsTxt, parseFixed}}) {
bool Active = ActiveCachedFile == Entry.File;
auto Loaded = Entry.File->load(FS, Active);
switch (Loaded.Result) {
@@ -333,6 +359,7 @@ bool DirectoryBasedGlobalCompilationDatabase::DirectoryCache::load(
tooling::CompilationDatabasePluginRegistry::entries()) {
// Avoid duplicating the special cases handled above.
if (Entry.getName() == "fixed-compilation-database" ||
+ Entry.getName() == "json-build-database" ||
Entry.getName() == "json-compilation-database")
continue;
auto Plugin = Entry.instantiate();
diff --git a/clang/include/clang/Tooling/JSONBuildDatabase.h b/clang/include/clang/Tooling/JSONBuildDatabase.h
new file mode 100644
index 0000000000000..83680dad28a8e
--- /dev/null
+++ b/clang/include/clang/Tooling/JSONBuildDatabase.h
@@ -0,0 +1,137 @@
+//===- JSONBuildDatabase.h --------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// The JSONBuildDatabase finds build databases supplied as a file
+// 'compile_commands.json'.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_JSONBUILDDATABASE_H
+#define LLVM_CLANG_TOOLING_JSONBUILDDATABASE_H
+
+#include "clang/Basic/LLVM.h"
+#include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/FileMatchTrie.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/Support/YAMLParser.h"
+#include <memory>
+#include <string>
+#include <tuple>
+#include <utility>
+#include <vector>
+
+namespace clang {
+namespace tooling {
+
+/// A JSON based build database.
+///
+/// JSON compilation database files must contain a list of JSON objects which
+/// provide the command lines in the attributes 'directory', 'command',
+/// 'arguments' and 'file':
+/// [
+/// { "directory": "<working directory of the compile>",
+/// "command": "<compile command line>",
+/// "file": "<path to source file>"
+/// },
+/// { "directory": "<working directory of the compile>",
+/// "arguments": ["<raw>", "<command>" "<line>" "<parameters>"],
+/// "file": "<path to source file>"
+/// },
+/// ...
+/// ]
+/// Each object entry defines one compile action. The specified file is
+/// considered to be the main source file for the translation unit.
+///
+/// 'command' is a full command line that will be unescaped.
+///
+/// 'arguments' is a list of command line arguments that will not be unescaped.
+///
+/// JSON build databases can for example be generated in CMake projects
+/// by setting the flag -DCMAKE_EXPORT_BUILD_DATABASE.
+class JSONBuildDatabase : public CompilationDatabase {
+public:
+ /// Loads a JSON build database from the specified file.
+ ///
+ /// Returns NULL and sets ErrorMessage if the database could not be
+ /// loaded from the given file.
+ static std::unique_ptr<JSONBuildDatabase>
+ loadFromFile(StringRef FilePath, std::string &ErrorMessage);
+
+ /// Loads a JSON build database from a data buffer.
+ ///
+ /// Returns NULL and sets ErrorMessage if the database could not be loaded.
+ static std::unique_ptr<JSONBuildDatabase>
+ loadFromBuffer(StringRef DatabaseString, std::string &ErrorMessage);
+
+ /// Returns all compile commands in which the specified file was
+ /// compiled.
+ ///
+ /// FIXME: Currently FilePath must be an absolute path inside the
+ /// source directory which does not have symlinks resolved.
+ std::vector<CompileCommand>
+ getCompileCommands(StringRef FilePath) const override;
+
+ /// Returns the list of all files available in the build database.
+ ///
+ /// These are the 'file' entries of the JSON objects.
+ std::vector<std::string> getAllFiles() const override;
+
+ /// Returns all compile commands for all the files in the build
+ /// database.
+ std::vector<CompileCommand> getAllCompileCommands() const override;
+
+private:
+ /// Constructs a JSON build database on a memory buffer.
+ JSONBuildDatabase(std::unique_ptr<llvm::MemoryBuffer> Database)
+ : Database(std::move(Database)),
+ YAMLStream(this->Database->getBuffer(), SM) {}
+
+ /// Parses the database file and creates the index.
+ ///
+ /// Returns whether parsing succeeded. Sets ErrorMessage if parsing
+ /// failed.
+ bool parse(std::string &ErrorMessage);
+
+ // Tuple (directory, filename, commandline, output) where 'commandline'
+ // points to the corresponding scalar nodes in the YAML stream.
+ // If the command line contains a single argument, it is a shell-escaped
+ // command line.
+ // Otherwise, each entry in the command line vector is a literal
+ // argument to the compiler.
+ // The output field may be a nullptr.
+ using CompileCommandRef =
+ std::tuple<llvm::yaml::ScalarNode *, llvm::yaml::ScalarNode *,
+ std::vector<llvm::yaml::ScalarNode *>,
+ llvm::yaml::ScalarNode *>;
+
+ /// Converts the given array of CompileCommandRefs to CompileCommands.
+ void getCommands(ArrayRef<CompileCommandRef> CommandsRef,
+ std::vector<CompileCommand> &Commands) const;
+
+ // Maps file paths to the compile command lines for that file.
+ llvm::StringMap<std::vector<CompileCommandRef>> IndexByFile;
+
+ /// All the compile commands in the order that they were provided in the
+ /// JSON stream.
+ std::vector<CompileCommandRef> AllCommands;
+
+ FileMatchTrie MatchTrie;
+
+ std::unique_ptr<llvm::MemoryBuffer> Database;
+ llvm::SourceMgr SM;
+ llvm::yaml::Stream YAMLStream;
+};
+
+} // namespace tooling
+} // namespace clang
+
+#endif // LLVM_CLANG_TOOLING_JSONBUILDDATABASE_H
diff --git a/clang/lib/Tooling/CMakeLists.txt b/clang/lib/Tooling/CMakeLists.txt
index 0972ecb08437f..00b27d6d61cfe 100644
--- a/clang/lib/Tooling/CMakeLists.txt
+++ b/clang/lib/Tooling/CMakeLists.txt
@@ -24,6 +24,7 @@ add_clang_library(clangTooling
FixIt.cpp
GuessTargetAndModeCompilationDatabase.cpp
InterpolatingCompilationDatabase.cpp
+ JSONBuildDatabase.cpp
JSONCompilationDatabase.cpp
LocateToolCompilationDatabase.cpp
Refactoring.cpp
diff --git a/clang/lib/Tooling/CompilationDatabase.cpp b/clang/lib/Tooling/CompilationDatabase.cpp
index 4070bb81c6f74..187e1223f5cef 100644
--- a/clang/lib/Tooling/CompilationDatabase.cpp
+++ b/clang/lib/Tooling/CompilationDatabase.cpp
@@ -84,7 +84,8 @@ findCompilationDatabaseFromDirectory(StringRef Directory,
if (!HasErrorMessage) {
ErrorStream << "No compilation database found in " << Directory.str()
- << " or any parent directory\n" << LoadErrorMessage;
+ << " or any parent directory\n"
+ << LoadErrorMessage;
HasErrorMessage = true;
}
@@ -105,7 +106,8 @@ CompilationDatabase::autoDetectFromSource(StringRef SourceFile,
if (!DB)
ErrorMessage = ("Could not auto-detect compilation database for file \"" +
- SourceFile + "\"\n" + ErrorMessage).str();
+ SourceFile + "\"\n" + ErrorMessage)
+ .str();
return DB;
}
@@ -118,8 +120,10 @@ CompilationDatabase::autoDetectFromDirectory(StringRef SourceDir,
findCompilationDatabaseFromDirectory(AbsolutePath, ErrorMessage);
if (!DB)
- ErrorMessage = ("Could not auto-detect compilation database from directory \"" +
- SourceDir + "\"\n" + ErrorMessage).str();
+ ErrorMessage =
+ ("Could not auto-detect compilation database from directory \"" +
+ SourceDir + "\"\n" + ErrorMessage)
+ .str();
return DB;
}
@@ -141,9 +145,7 @@ namespace {
struct CompileJobAnalyzer {
SmallVector<std::string, 2> Inputs;
- void run(const driver::Action *A) {
- runImpl(A, false);
- }
+ void run(const driver::Action *A) { runImpl(A, false); }
private:
void runImpl(const driver::Action *A, bool Collect) {
@@ -198,7 +200,7 @@ class UnusedInputDiagConsumer : public DiagnosticConsumer {
// They are not used for syntax checking, and could confuse targets
// which don't support these options.
struct FilterUnusedFlags {
- bool operator() (StringRef S) {
+ bool operator()(StringRef S) {
return (S == "-no-integrated-as") || S.starts_with("-Wa,");
}
};
@@ -366,11 +368,10 @@ FixedCompilationDatabase::loadFromBuffer(StringRef Directory, StringRef Data,
FixedCompilationDatabase::FixedCompilationDatabase(
const Twine &Directory, ArrayRef<std::string> CommandLine) {
std::vector<std::string> ToolCommandLine(1, GetClangToolCommand());
- ToolCommandLine.insert(ToolCommandLine.end(),
- CommandLine.begin(), CommandLine.end());
+ ToolCommandLine.insert(ToolCommandLine.end(), CommandLine.begin(),
+ CommandLine.end());
CompileCommands.emplace_back(Directory, StringRef(),
- std::move(ToolCommandLine),
- StringRef());
+ std::move(ToolCommandLine), StringRef());
}
std::vector<CompileCommand>
@@ -395,15 +396,19 @@ class FixedCompilationDatabasePlugin : public CompilationDatabasePlugin {
} // namespace
static CompilationDatabasePluginRegistry::Add<FixedCompilationDatabasePlugin>
-X("fixed-compilation-database", "Reads plain-text flags file");
+ X("fixed-compilation-database", "Reads plain-text flags file");
namespace clang {
namespace tooling {
// This anchor is used to force the linker to link in the generated object file
// and thus register the JSONCompilationDatabasePlugin.
-extern volatile int JSONAnchorSource;
-[[maybe_unused]] static int JSONAnchorDest = JSONAnchorSource;
+extern volatile int JSONBuildAnchorSource;
+[[maybe_unused]] static int JSONBuildAnchorDest = JSONBuildAnchorSource;
+
+extern volatile int JSONCompilationAnchorSource;
+[[maybe_unused]] static int JSONCompilationAnchorDest =
+ JSONCompilationAnchorSource;
} // namespace tooling
} // namespace clang
diff --git a/clang/lib/Tooling/JSONBuildDatabase.cpp b/clang/lib/Tooling/JSONBuildDatabase.cpp
new file mode 100644
index 0000000000000..efa7eb063510b
--- /dev/null
+++ b/clang/lib/Tooling/JSONBuildDatabase.cpp
@@ -0,0 +1,307 @@
+//===- JSONBuildDatabase.cpp ----------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains the implementation of the JSONBuildDatabase.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/JSONBuildDatabase.h"
+#include "clang/Basic/LLVM.h"
+#include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/CompilationDatabasePluginRegistry.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/ErrorOr.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/StringSaver.h"
+#include "llvm/Support/VirtualFileSystem.h"
+#include "llvm/Support/YAMLParser.h"
+#include "llvm/Support/raw_ostream.h"
+#include "llvm/TargetParser/Host.h"
+#include <cassert>
+#include <memory>
+#include <optional>
+#include <string>
+#include <system_error>
+#include <utility>
+#include <vector>
+
+using namespace clang;
+using namespace tooling;
+
+namespace {
+// This plugin locates a nearby compile_command.json file, and also infers
+// compile commands for files not present in the database.
+class JSONBuildDatabasePlugin : public CompilationDatabasePlugin {
+ std::unique_ptr<CompilationDatabase>
+ loadFromDirectory(StringRef Directory, std::string &ErrorMessage) override {
+ SmallString<1024> JSONDatabasePath(Directory);
+ llvm::sys::path::append(JSONDatabasePath, "compile_commands.json");
+ auto Base = JSONBuildDatabase::loadFromFile(JSONDatabasePath, ErrorMessage);
+ return Base ? inferTargetAndDriverMode(
+ inferMissingCompileCommands(expandResponseFiles(
+ std::move(Base), llvm::vfs::getRealFileSystem())))
+ : nullptr;
+ }
+};
+
+} // namespace
+
+// Register the JSONBuildDatabasePlugin with the
+// CompilationDatabasePluginRegistry using this statically initialized variable.
+static CompilationDatabasePluginRegistry::Add<JSONBuildDatabasePlugin>
+ X("json-build-database", "Reads JSON formatted build databases");
+
+namespace clang {
+namespace tooling {
+
+// This anchor is used to force the linker to link in the generated object file
+// and thus register the JSONBuildDatabasePlugin.
+volatile int JSONBuildAnchorSource = 0;
+
+} // namespace tooling
+} // namespace clang
+
+std::unique_ptr<JSONBuildDatabase>
+JSONBuildDatabase::loadFromFile(StringRef FilePath, std::string &ErrorMessage) {
+ // Don't mmap: if we're a long-lived process, the build system may overwrite.
+ llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> DatabaseBuffer =
+ llvm::MemoryBuffer::getFile(FilePath, /*IsText=*/false,
+ /*RequiresNullTerminator=*/true,
+ /*IsVolatile=*/true);
+ if (std::error_code Result = DatabaseBuffer.getError()) {
+ ErrorMessage = "Error while opening JSON database: " + Result.message();
+ return nullptr;
+ }
+ std::unique_ptr<JSONBuildDatabase> Database(
+ new JSONBuildDatabase(std::move(*DatabaseBuffer)));
+ if (!Database->parse(ErrorMessage))
+ return nullptr;
+ return Database;
+}
+
+std::unique_ptr<JSONBuildDatabase>
+JSONBuildDatabase::loadFromBuffer(StringRef DatabaseString,
+ std::string &ErrorMessage) {
+ std::unique_ptr<llvm::MemoryBuffer> DatabaseBuffer(
+ llvm::MemoryBuffer::getMemBufferCopy(DatabaseString));
+ std::unique_ptr<JSONBuildDatabase> Database(
+ new JSONBuildDatabase(std::move(DatabaseBuffer)));
+ if (!Database->parse(ErrorMessage))
+ return nullptr;
+ return Database;
+}
+
+std::vector<CompileCommand>
+JSONBuildDatabase::getCompileCommands(StringRef FilePath) const {
+ SmallString<128> NativeFilePath;
+ llvm::sys::path::native(FilePath, NativeFilePath);
+
+ std::string Error;
+ llvm::raw_string_ostream ES(Error);
+ StringRef Match = MatchTrie.findEquivalent(NativeFilePath, ES);
+ if (Match.empty())
+ return {};
+ const auto CommandsRefI = IndexByFile.find(Match);
+ if (CommandsRefI == IndexByFile.end())
+ return {};
+ std::vector<CompileCommand> Commands;
+ getCommands(CommandsRefI->getValue(), Commands);
+ return Commands;
+}
+
+std::vector<std::string> JSONBuildDatabase::getAllFiles() const {
+ std::vector<std::string> Result;
+ for (const auto &CommandRef : IndexByFile)
+ Result.push_back(CommandRef.first().str());
+ return Result;
+}
+
+std::vector<CompileCommand> JSONBuildDatabase::getAllCompileCommands() const {
+ std::vector<CompileCommand> Commands;
+ getCommands(AllCommands, Commands);
+ return Commands;
+}
+
+static llvm::StringRef stripExecutableExtension(llvm::StringRef Name) {
+ Name.consume_back(".exe");
+ return Name;
+}
+
+// There are compiler-wrappers (ccache, distcc) that take the "real"
+// compiler as an argument, e.g. distcc gcc -O3 foo.c.
+// These end up in compile_commands.json when people set CC="distcc gcc".
+// Clang's driver doesn't understand this, so we need to unwrap.
+static bool unwrapCommand(std::vector<std::string> &Args) {
+ if (Args.size() < 2)
+ return false;
+ StringRef Wrapper =
+ stripExecutableExtension(llvm::sys::path::filename(Args.front()));
+ if (Wrapper == "distcc" || Wrapper == "ccache" || Wrapper == "sccache") {
+ // Most of these wrappers support being invoked 3 ways:
+ // `distcc g++ file.c` This is the mode we're trying to match.
+ // We need to drop `distcc`.
+ // `distcc file.c` This acts like compiler is cc or similar.
+ // Clang's driver can handle this, no change needed.
+ // `g++ file.c` g++ is a symlink to distcc.
+ // We don't even notice this case, and all is well.
+ //
+ // We need to distinguish between the first and second case.
+ // The wrappers themselves don't take flags, so Args[1] is a compiler flag,
+ // an input file, or a compiler. Inputs have extensions, compilers don't.
+ bool HasCompiler =
+ (Args[1][0] != '-') &&
+ !llvm::sys::path::has_extension(stripExecutableExtension(Args[1]));
+ if (HasCompiler) {
+ Args.erase(Args.begin());
+ return true;
+ }
+ // If !HasCompiler, wrappers act like GCC. Fine: so do we.
+ }
+ return false;
+}
+
+static std::vector<std::string>
+nodeToCommandLine(const std::vector<llvm::yaml::ScalarNode *> &Nodes) {
+ SmallString<1024> Storage;
+ std::vector<std::string> Arguments;
+ for (const auto *Node : Nodes)
+ Arguments.push_back(std::string(Node->getValue(Storage)));
+ // There may be multiple wrappers: using distcc and ccache together is common.
+ while (unwrapCommand(Arguments))
+ ;
+ return Arguments;
+}
+
+void JSONBuildDatabase::getCommands(
+ ArrayRef<CompileCommandRef> CommandsRef,
+ std::vector<CompileCommand> &Commands) const {
+ for (const auto &CommandRef : CommandsRef) {
+ SmallString<8> DirectoryStorage;
+ SmallString<32> FilenameStorage;
+ SmallString<32> OutputStorage;
+ auto Output = std::get<3>(CommandRef);
+ Commands.emplace_back(std::get<0>(CommandRef)->getValue(DirectoryStorage),
+ std::get<1>(CommandRef)->getValue(FilenameStorage),
+ nodeToCommandLine(std::get<2>(CommandRef)),
+ Output ? Output->getValue(OutputStorage) : "");
+ }
+}
+
+bool JSONBuildDatabase::parse(std::string &ErrorMessage) {
+ llvm::yaml::document_iterator I = YAMLStream.begin();
+ if (I == YAMLStream.end()) {
+ ErrorMessage = "Error while parsing YAML.";
+ return false;
+ }
+ llvm::yaml::Node *Root = I->getRoot();
+ if (!Root) {
+ ErrorMessage = "Error while parsing YAML.";
+ return false;
+ }
+ auto *Array = dyn_cast<llvm::yaml::SequenceNode>(Root);
+ if (!Array) {
+ ErrorMessage = "Expected array.";
+ return false;
+ }
+ for (auto &NextObject : *Array) {
+ auto *Object = dyn_cast<llvm::yaml::MappingNode>(&NextObject);
+ if (!Object) {
+ ErrorMessage = "Expected object.";
+ return false;
+ }
+ llvm::yaml::ScalarNode *Directory = nullptr;
+ std::optional<std::vector<llvm::yaml::ScalarNode *>> Command;
+ llvm::yaml::ScalarNode *File = nullptr;
+ llvm::yaml::ScalarNode *Output = nullptr;
+ for (auto &NextKeyValue : *Object) {
+ auto *KeyString =
+ dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+ if (!KeyString) {
+ ErrorMessage = "Expected strings as key.";
+ return false;
+ }
+ SmallString<10> KeyStorage;
+ StringRef KeyValue = KeyString->getValue(KeyStorage);
+ llvm::yaml::Node *Value = NextKeyValue.getValue();
+ if (!Value) {
+ ErrorMessage = "Expected value.";
+ return false;
+ }
+ auto *ValueString = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ auto *SequenceString = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (KeyValue == "arguments") {
+ if (!SequenceString) {
+ ErrorMessage = "Expected sequence as value.";
+ return false;
+ }
+ Command = std::vector<llvm::yaml::ScalarNode *>();
+ for (auto &Argument : *SequenceString) {
+ auto *Scalar = dyn_cast<llvm::yaml::ScalarNode>(&Argument);
+ if (!Scalar) {
+ ErrorMessage = "Only strings are allowed in 'arguments'.";
+ return false;
+ }
+ Command->push_back(Scalar);
+ }
+ } else {
+ if (!ValueString) {
+ ErrorMessage = "Expected string as value.";
+ return false;
+ }
+ if (KeyValue == "directory") {
+ Directory = ValueString;
+ } else if (KeyValue == "command") {
+ if (!Command)
+ Command = std::vector<llvm::yaml::ScalarNode *>(1, ValueString);
+ } else if (KeyValue == "file") {
+ File = ValueString;
+ } else if (KeyValue == "output") {
+ Output = ValueString;
+ } else {
+ ErrorMessage =
+ ("Unknown key: \"" + KeyString->getRawValue() + "\"").str();
+ return false;
+ }
+ }
+ }
+ if (!File) {
+ ErrorMessage = "Missing key: \"file\".";
+ return false;
+ }
+ if (!Command) {
+ ErrorMessage = "Missing key: \"command\" or \"arguments\".";
+ return false;
+ }
+ if (!Directory) {
+ ErrorMessage = "Missing key: \"directory\".";
+ return false;
+ }
+ SmallString<8> FileStorage;
+ StringRef FileName = File->getValue(FileStorage);
+ SmallString<128> NativeFilePath;
+ if (llvm::sys::path::is_relative(FileName)) {
+ SmallString<8> DirectoryStorage;
+ SmallString<128> AbsolutePath(Directory->getValue(DirectoryStorage));
+ llvm::sys::path::append(AbsolutePath, FileName);
+ llvm::sys::path::native(AbsolutePath, NativeFilePath);
+ } else {
+ llvm::sys::path::native(FileName, NativeFilePath);
+ }
+ llvm::sys::path::remove_dots(NativeFilePath, /*remove_dot_dot=*/true);
+ auto Cmd = CompileCommandRef(Directory, File, *Command, Output);
+ IndexByFile[NativeFilePath].push_back(Cmd);
+ AllCommands.push_back(Cmd);
+ MatchTrie.insert(NativeFilePath);
+ }
+ return true;
+}
diff --git a/clang/lib/Tooling/JSONCompilationDatabase.cpp b/clang/lib/Tooling/JSONCompilationDatabase.cpp
index 0efa75970d986..9155b01f3fe13 100644
--- a/clang/lib/Tooling/JSONCompilationDatabase.cpp
+++ b/clang/lib/Tooling/JSONCompilationDatabase.cpp
@@ -45,9 +45,9 @@ namespace {
/// Assumes \-escaping for quoted arguments (see the documentation of
/// unescapeCommandLine(...)).
class CommandLineArgumentParser {
- public:
+public:
CommandLineArgumentParser(StringRef CommandLine)
- : Input(CommandLine), Position(Input.begin()-1) {}
+ : Input(CommandLine), Position(Input.begin() - 1) {}
std::vector<std::string> parse() {
bool HasMoreInput = true;
@@ -59,46 +59,56 @@ class CommandLineArgumentParser {
return CommandLine;
}
- private:
+private:
// All private methods return true if there is more input available.
bool parseStringInto(std::string &String) {
do {
if (*Position == '"') {
- if (!parseDoubleQuotedStringInto(String)) return false;
+ if (!parseDoubleQuotedStringInto(String))
+ return false;
} else if (*Position == '\'') {
- if (!parseSingleQuotedStringInto(String)) return false;
+ if (!parseSingleQuotedStringInto(String))
+ return false;
} else {
- if (!parseFreeStringInto(String)) return false;
+ if (!parseFreeStringInto(String))
+ return false;
}
} while (*Position != ' ');
return true;
}
bool parseDoubleQuotedStringInto(std::string &String) {
- if (!next()) return false;
+ if (!next())
+ return false;
while (*Position != '"') {
- if (!skipEscapeCharacter()) return false;
+ if (!skipEscapeCharacter())
+ return false;
String.push_back(*Position);
- if (!next()) return false;
+ if (!next())
+ return false;
}
return next();
}
bool parseSingleQuotedStringInto(std::string &String) {
- if (!next()) return false;
+ if (!next())
+ return false;
while (*Position != '\'') {
String.push_back(*Position);
- if (!next()) return false;
+ if (!next())
+ return false;
}
return next();
}
bool parseFreeStringInto(std::string &String) {
do {
- if (!skipEscapeCharacter()) return false;
+ if (!skipEscapeCharacter())
+ return false;
String.push_back(*Position);
- if (!next()) return false;
+ if (!next())
+ return false;
} while (*Position != ' ' && *Position != '"' && *Position != '\'');
return true;
}
@@ -112,7 +122,8 @@ class CommandLineArgumentParser {
bool nextNonWhitespace() {
do {
- if (!next()) return false;
+ if (!next())
+ return false;
} while (*Position == ' ');
return true;
}
@@ -172,14 +183,15 @@ class JSONCompilationDatabasePlugin : public CompilationDatabasePlugin {
// Register the JSONCompilationDatabasePlugin with the
// CompilationDatabasePluginRegistry using this statically initialized variable.
static CompilationDatabasePluginRegistry::Add<JSONCompilationDatabasePlugin>
-X("json-compilation-database", "Reads JSON formatted compilation databases");
+ X("json-compilation-database",
+ "Reads JSON formatted compilation databases");
namespace clang {
namespace tooling {
// This anchor is used to force the linker to link in the generated object file
// and thus register the JSONCompilationDatabasePlugin.
-volatile int JSONAnchorSource = 0;
+volatile int JSONCompilationAnchorSource = 0;
} // namespace tooling
} // namespace clang
@@ -235,8 +247,7 @@ JSONCompilationDatabase::getCompileCommands(StringRef FilePath) const {
return Commands;
}
-std::vector<std::string>
-JSONCompilationDatabase::getAllFiles() const {
+std::vector<std::string> JSONCompilationDatabase::getAllFiles() const {
std::vector<std::string> Result;
for (const auto &CommandRef : IndexByFile)
Result.push_back(CommandRef.first().str());
@@ -312,11 +323,10 @@ void JSONCompilationDatabase::getCommands(
SmallString<32> FilenameStorage;
SmallString<32> OutputStorage;
auto Output = std::get<3>(CommandRef);
- Commands.emplace_back(
- std::get<0>(CommandRef)->getValue(DirectoryStorage),
- std::get<1>(CommandRef)->getValue(FilenameStorage),
- nodeToCommandLine(Syntax, std::get<2>(CommandRef)),
- Output ? Output->getValue(OutputStorage) : "");
+ Commands.emplace_back(std::get<0>(CommandRef)->getValue(DirectoryStorage),
+ std::get<1>(CommandRef)->getValue(FilenameStorage),
+ nodeToCommandLine(Syntax, std::get<2>(CommandRef)),
+ Output ? Output->getValue(OutputStorage) : "");
}
}
@@ -346,7 +356,7 @@ bool JSONCompilationDatabase::parse(std::string &ErrorMessage) {
std::optional<std::vector<llvm::yaml::ScalarNode *>> Command;
llvm::yaml::ScalarNode *File = nullptr;
llvm::yaml::ScalarNode *Output = nullptr;
- for (auto& NextKeyValue : *Object) {
+ for (auto &NextKeyValue : *Object) {
auto *KeyString =
dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
if (!KeyString) {
>From 1da7510749db19f6f5f73c78554c343bac64e499 Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Sun, 17 May 2026 19:29:59 -0700
Subject: [PATCH 03/10] Parse full build_database
---
.../clangd/GlobalCompilationDatabase.cpp | 3 +-
.../include/clang/Tooling/JSONBuildDatabase.h | 3 +
clang/lib/Tooling/JSONBuildDatabase.cpp | 313 ++++++++++++++----
3 files changed, 247 insertions(+), 72 deletions(-)
diff --git a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
index 8ba66ca41d312..11205a1b7c646 100644
--- a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
+++ b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
@@ -253,8 +253,7 @@ DirectoryBasedGlobalCompilationDatabase::DirectoryCache::CachedFile::load(
// Adapt CDB-loading functions to a common interface for DirectoryCache::load().
static std::unique_ptr<tooling::CompilationDatabase>
parseBuildDatabaseJSON(PathRef Path, llvm::StringRef Data, std::string &Error) {
- if (auto CDB = tooling::JSONCompilationDatabase::loadFromBuffer(
- Data, Error, tooling::JSONCommandLineSyntax::AutoDetect)) {
+ if (auto CDB = tooling::JSONBuildDatabase::loadFromBuffer(Data, Error)) {
// FS used for expanding response files.
// FIXME: ExpandResponseFilesDatabase appears not to provide the usual
// thread-safety guarantees, as the access to FS is not locked!
diff --git a/clang/include/clang/Tooling/JSONBuildDatabase.h b/clang/include/clang/Tooling/JSONBuildDatabase.h
index 83680dad28a8e..403aae91a19bd 100644
--- a/clang/include/clang/Tooling/JSONBuildDatabase.h
+++ b/clang/include/clang/Tooling/JSONBuildDatabase.h
@@ -100,6 +100,9 @@ class JSONBuildDatabase : public CompilationDatabase {
/// Returns whether parsing succeeded. Sets ErrorMessage if parsing
/// failed.
bool parse(std::string &ErrorMessage);
+ bool parseRoot(std::string &ErrorMessage, llvm::yaml::MappingNode *Object);
+ bool parseSet(std::string &ErrorMessage, llvm::yaml::MappingNode *Object);
+ bool parseTU(std::string &ErrorMessage, llvm::yaml::MappingNode *Object);
// Tuple (directory, filename, commandline, output) where 'commandline'
// points to the corresponding scalar nodes in the YAML stream.
diff --git a/clang/lib/Tooling/JSONBuildDatabase.cpp b/clang/lib/Tooling/JSONBuildDatabase.cpp
index efa7eb063510b..1d631d6b928dd 100644
--- a/clang/lib/Tooling/JSONBuildDatabase.cpp
+++ b/clang/lib/Tooling/JSONBuildDatabase.cpp
@@ -30,8 +30,11 @@
#include <cassert>
#include <memory>
#include <optional>
+#include <signal.h>
#include <string>
#include <system_error>
+#include <thread>
+#include <unistd.h>
#include <utility>
#include <vector>
@@ -208,100 +211,270 @@ bool JSONBuildDatabase::parse(std::string &ErrorMessage) {
ErrorMessage = "Error while parsing YAML.";
return false;
}
- auto *Array = dyn_cast<llvm::yaml::SequenceNode>(Root);
- if (!Array) {
- ErrorMessage = "Expected array.";
+ auto *RootObject = dyn_cast<llvm::yaml::MappingNode>(Root);
+ if (!RootObject) {
+ ErrorMessage = "Expected object at root.";
return false;
}
- for (auto &NextObject : *Array) {
- auto *Object = dyn_cast<llvm::yaml::MappingNode>(&NextObject);
- if (!Object) {
- ErrorMessage = "Expected object.";
+ return parseRoot(ErrorMessage, RootObject);
+}
+
+bool JSONBuildDatabase::parseRoot(std::string &ErrorMessage,
+ llvm::yaml::MappingNode *RootObject) {
+ llvm::yaml::ScalarNode *Version = nullptr;
+ llvm::yaml::ScalarNode *Revision = nullptr;
+ llvm::yaml::SequenceNode *Sets = nullptr;
+ for (auto &NextKeyValue : *RootObject) {
+ auto *KeyString =
+ dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+ if (!KeyString) {
+ ErrorMessage = "Expected strings as key.";
return false;
}
- llvm::yaml::ScalarNode *Directory = nullptr;
- std::optional<std::vector<llvm::yaml::ScalarNode *>> Command;
- llvm::yaml::ScalarNode *File = nullptr;
- llvm::yaml::ScalarNode *Output = nullptr;
- for (auto &NextKeyValue : *Object) {
- auto *KeyString =
- dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
- if (!KeyString) {
- ErrorMessage = "Expected strings as key.";
+ SmallString<10> KeyStorage;
+ StringRef KeyValue = KeyString->getValue(KeyStorage);
+ llvm::yaml::Node *Value = NextKeyValue.getValue();
+ if (!Value) {
+ ErrorMessage = "Expected value.";
+ return false;
+ }
+ if (KeyValue == "version") {
+ Version = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!Version) {
+ ErrorMessage = "Expected string as value for \"version\".";
+ return false;
+ }
+ } else if (KeyValue == "revision") {
+ Revision = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!Revision) {
+ ErrorMessage = "Expected string as value for \"revision\".";
return false;
}
- SmallString<10> KeyStorage;
- StringRef KeyValue = KeyString->getValue(KeyStorage);
- llvm::yaml::Node *Value = NextKeyValue.getValue();
- if (!Value) {
- ErrorMessage = "Expected value.";
+ } else if (KeyValue == "sets") {
+ Sets = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!Sets) {
+ ErrorMessage = "Expected array as value for \"sets\".";
return false;
}
- auto *ValueString = dyn_cast<llvm::yaml::ScalarNode>(Value);
- auto *SequenceString = dyn_cast<llvm::yaml::SequenceNode>(Value);
- if (KeyValue == "arguments") {
- if (!SequenceString) {
- ErrorMessage = "Expected sequence as value.";
+ for (auto &NextObject : *Sets) {
+ auto *SetObject = dyn_cast<llvm::yaml::MappingNode>(&NextObject);
+ if (!RootObject) {
+ ErrorMessage = "Expected sets item as object.";
return false;
}
- Command = std::vector<llvm::yaml::ScalarNode *>();
- for (auto &Argument : *SequenceString) {
- auto *Scalar = dyn_cast<llvm::yaml::ScalarNode>(&Argument);
- if (!Scalar) {
- ErrorMessage = "Only strings are allowed in 'arguments'.";
- return false;
- }
- Command->push_back(Scalar);
+ if (!parseSet(ErrorMessage, SetObject)) {
+ return false;
}
- } else {
- if (!ValueString) {
- ErrorMessage = "Expected string as value.";
+ }
+ } else {
+ ErrorMessage =
+ ("Unknown key in root: \"" + KeyString->getRawValue() + "\"").str();
+ return false;
+ }
+ }
+ // Check required fields
+ if (!Version) {
+ ErrorMessage = "Missing key in root: \"version\".";
+ return false;
+ }
+ if (!Sets) {
+ ErrorMessage = "Missing key in root: \"sets\".";
+ return false;
+ }
+ // Check compatible version
+ if (Version->getRawValue() != "1") {
+ ErrorMessage =
+ ("Unsupported version: \"" + Version->getRawValue() + "\"").str();
+ return false;
+ }
+ return true;
+}
+
+bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
+ llvm::yaml::MappingNode *SetObject) {
+ llvm::yaml::SequenceNode *BaselineArguments = nullptr;
+ llvm::yaml::ScalarNode *FamilyName = nullptr;
+ llvm::yaml::ScalarNode *Name = nullptr;
+ llvm::yaml::SequenceNode *VisibleSets = nullptr;
+ llvm::yaml::SequenceNode *TUs = nullptr;
+ for (auto &NextKeyValue : *SetObject) {
+ auto *KeyString =
+ dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+ if (!KeyString) {
+ ErrorMessage = "Expected strings as key.";
+ return false;
+ }
+ SmallString<10> KeyStorage;
+ StringRef KeyValue = KeyString->getValue(KeyStorage);
+ llvm::yaml::Node *Value = NextKeyValue.getValue();
+ if (!Value) {
+ ErrorMessage = "Expected value.";
+ return false;
+ }
+ if (KeyValue == "baseline-arguments") {
+ BaselineArguments = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!BaselineArguments) {
+ ErrorMessage = "Expected array as value for \"version\".";
+ return false;
+ }
+ } else if (KeyValue == "family-name") {
+ FamilyName = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!FamilyName) {
+ ErrorMessage = "Expected string as value for \"family-name\".";
+ return false;
+ }
+ } else if (KeyValue == "name") {
+ Name = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!Name) {
+ ErrorMessage = "Expected string as value for \"name\".";
+ return false;
+ }
+ } else if (KeyValue == "visible-sets") {
+ VisibleSets = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!VisibleSets) {
+ ErrorMessage = "Expected array as value for \"visible-sets\".";
+ return false;
+ }
+ } else if (KeyValue == "translation-units") {
+ TUs = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!TUs) {
+ ErrorMessage = "Expected array as value for \"translation-units\".";
+ return false;
+ }
+ for (auto &NextObject : *TUs) {
+ auto *TUObject = dyn_cast<llvm::yaml::MappingNode>(&NextObject);
+ if (!TUObject) {
+ ErrorMessage = "Expected translation-units item as object.";
return false;
}
- if (KeyValue == "directory") {
- Directory = ValueString;
- } else if (KeyValue == "command") {
- if (!Command)
- Command = std::vector<llvm::yaml::ScalarNode *>(1, ValueString);
- } else if (KeyValue == "file") {
- File = ValueString;
- } else if (KeyValue == "output") {
- Output = ValueString;
- } else {
- ErrorMessage =
- ("Unknown key: \"" + KeyString->getRawValue() + "\"").str();
+ if (!parseTU(ErrorMessage, TUObject)) {
return false;
}
}
- }
- if (!File) {
- ErrorMessage = "Missing key: \"file\".";
+ } else {
+ ErrorMessage =
+ ("Unknown key in set: \"" + KeyString->getRawValue() + "\"").str();
return false;
}
- if (!Command) {
- ErrorMessage = "Missing key: \"command\" or \"arguments\".";
+ }
+ // Check required fields
+ if (!BaselineArguments) {
+ ErrorMessage = "Missing key in set: \"baseline-arguments\".";
+ return false;
+ }
+ if (!FamilyName) {
+ ErrorMessage = "Missing key in set: \"family-name\".";
+ return false;
+ }
+ if (!Name) {
+ ErrorMessage = "Missing key in set: \"name\".";
+ return false;
+ }
+ if (!TUs) {
+ ErrorMessage = "Missing key in set: \"translation-units\".";
+ return false;
+ }
+ return true;
+}
+
+bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
+ llvm::yaml::MappingNode *TUObject) {
+ llvm::yaml::SequenceNode *Arguments = nullptr;
+ llvm::yaml::ScalarNode *Language = nullptr;
+ llvm::yaml::SequenceNode *LocalArguments = nullptr;
+ llvm::yaml::ScalarNode *WorkingDirectory = nullptr;
+ llvm::yaml::ScalarNode *Private = nullptr;
+ llvm::yaml::ScalarNode *Source = nullptr;
+ llvm::yaml::ScalarNode *Object = nullptr;
+ llvm::yaml::SequenceNode *Provides = nullptr;
+ llvm::yaml::SequenceNode *Requires = nullptr;
+ for (auto &NextKeyValue : *TUObject) {
+ auto *KeyString =
+ dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+ if (!KeyString) {
+ ErrorMessage = "Expected strings as key.";
return false;
}
- if (!Directory) {
- ErrorMessage = "Missing key: \"directory\".";
+ SmallString<10> KeyStorage;
+ StringRef KeyValue = KeyString->getValue(KeyStorage);
+ llvm::yaml::Node *Value = NextKeyValue.getValue();
+ if (!Value) {
+ ErrorMessage = "Expected value.";
return false;
}
- SmallString<8> FileStorage;
- StringRef FileName = File->getValue(FileStorage);
- SmallString<128> NativeFilePath;
- if (llvm::sys::path::is_relative(FileName)) {
- SmallString<8> DirectoryStorage;
- SmallString<128> AbsolutePath(Directory->getValue(DirectoryStorage));
- llvm::sys::path::append(AbsolutePath, FileName);
- llvm::sys::path::native(AbsolutePath, NativeFilePath);
+ if (KeyValue == "arguments") {
+ Arguments = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!Arguments) {
+ ErrorMessage = "Expected array as value for \"arguments\".";
+ return false;
+ }
+ } else if (KeyValue == "language") {
+ Language = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!Language) {
+ ErrorMessage = "Expected string as value for \"language\".";
+ return false;
+ }
+ } else if (KeyValue == "local-arguments") {
+ LocalArguments = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!LocalArguments) {
+ ErrorMessage = "Expected array as value for \"local-arguments\".";
+ return false;
+ }
+ } else if (KeyValue == "working-directory") {
+ WorkingDirectory = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!WorkingDirectory) {
+ ErrorMessage = "Expected string as value for \"working-directory\".";
+ return false;
+ }
+ } else if (KeyValue == "private") {
+ Private = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!Private) {
+ ErrorMessage = "Expected string as value for \"private\".";
+ return false;
+ }
+ } else if (KeyValue == "source") {
+ Source = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!Source) {
+ ErrorMessage = "Expected string as value for \"source\".";
+ return false;
+ }
+ } else if (KeyValue == "object") {
+ Object = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!Object) {
+ ErrorMessage = "Expected string as value for \"object\".";
+ return false;
+ }
+ } else if (KeyValue == "provides") {
+ Provides = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!Provides) {
+ ErrorMessage = "Expected array as value for \"provides\".";
+ return false;
+ }
+ } else if (KeyValue == "requires") {
+ Requires = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ if (!Requires) {
+ ErrorMessage = "Expected array as value for \"requires\".";
+ return false;
+ }
} else {
- llvm::sys::path::native(FileName, NativeFilePath);
+ ErrorMessage = ("Unknown key in translation-unit: \"" +
+ KeyString->getRawValue() + "\"")
+ .str();
+ return false;
}
- llvm::sys::path::remove_dots(NativeFilePath, /*remove_dot_dot=*/true);
- auto Cmd = CompileCommandRef(Directory, File, *Command, Output);
- IndexByFile[NativeFilePath].push_back(Cmd);
- AllCommands.push_back(Cmd);
- MatchTrie.insert(NativeFilePath);
+ }
+ // Check required fields
+ if (!Source) {
+ ErrorMessage = "Missing key in translation-unit: \"source\".";
+ return false;
+ }
+ if (!Language) {
+ ErrorMessage = "Missing key in translation-unit: \"language\".";
+ return false;
+ }
+ if (!Arguments) {
+ ErrorMessage = "Missing key in translation-unit: \"arguments\".";
+ return false;
}
return true;
}
>From 5c9e9953f0f1210f3b9409b7fa70119a8760e7b8 Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Mon, 18 May 2026 17:04:32 -0700
Subject: [PATCH 04/10] Finish parse
---
clang/lib/Tooling/JSONBuildDatabase.cpp | 43 ++++++++++++++++++++-----
1 file changed, 35 insertions(+), 8 deletions(-)
diff --git a/clang/lib/Tooling/JSONBuildDatabase.cpp b/clang/lib/Tooling/JSONBuildDatabase.cpp
index 1d631d6b928dd..4c3e75610c989 100644
--- a/clang/lib/Tooling/JSONBuildDatabase.cpp
+++ b/clang/lib/Tooling/JSONBuildDatabase.cpp
@@ -380,13 +380,14 @@ bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
llvm::yaml::MappingNode *TUObject) {
llvm::yaml::SequenceNode *Arguments = nullptr;
+ std::vector<llvm::yaml::ScalarNode *> Command;
llvm::yaml::ScalarNode *Language = nullptr;
llvm::yaml::SequenceNode *LocalArguments = nullptr;
- llvm::yaml::ScalarNode *WorkingDirectory = nullptr;
+ llvm::yaml::ScalarNode *WorkDirectory = nullptr;
llvm::yaml::ScalarNode *Private = nullptr;
llvm::yaml::ScalarNode *Source = nullptr;
llvm::yaml::ScalarNode *Object = nullptr;
- llvm::yaml::SequenceNode *Provides = nullptr;
+ llvm::yaml::MappingNode *Provides = nullptr;
llvm::yaml::SequenceNode *Requires = nullptr;
for (auto &NextKeyValue : *TUObject) {
auto *KeyString =
@@ -408,6 +409,14 @@ bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
ErrorMessage = "Expected array as value for \"arguments\".";
return false;
}
+ for (auto &Argument : *Arguments) {
+ auto *Scalar = dyn_cast<llvm::yaml::ScalarNode>(&Argument);
+ if (!Scalar) {
+ ErrorMessage = "Only strings are allowed in 'arguments'.";
+ return false;
+ }
+ Command.push_back(Scalar);
+ }
} else if (KeyValue == "language") {
Language = dyn_cast<llvm::yaml::ScalarNode>(Value);
if (!Language) {
@@ -420,10 +429,10 @@ bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
ErrorMessage = "Expected array as value for \"local-arguments\".";
return false;
}
- } else if (KeyValue == "working-directory") {
- WorkingDirectory = dyn_cast<llvm::yaml::ScalarNode>(Value);
- if (!WorkingDirectory) {
- ErrorMessage = "Expected string as value for \"working-directory\".";
+ } else if (KeyValue == "work-directory") {
+ WorkDirectory = dyn_cast<llvm::yaml::ScalarNode>(Value);
+ if (!WorkDirectory) {
+ ErrorMessage = "Expected string as value for \"work-directory\".";
return false;
}
} else if (KeyValue == "private") {
@@ -445,9 +454,9 @@ bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
return false;
}
} else if (KeyValue == "provides") {
- Provides = dyn_cast<llvm::yaml::SequenceNode>(Value);
+ Provides = dyn_cast<llvm::yaml::MappingNode>(Value);
if (!Provides) {
- ErrorMessage = "Expected array as value for \"provides\".";
+ ErrorMessage = "Expected object as value for \"provides\".";
return false;
}
} else if (KeyValue == "requires") {
@@ -476,5 +485,23 @@ bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
ErrorMessage = "Missing key in translation-unit: \"arguments\".";
return false;
}
+ SmallString<8> FileStorage;
+ StringRef FileName = Source->getValue(FileStorage);
+ SmallString<128> NativeFilePath;
+ if (llvm::sys::path::is_relative(FileName)) {
+ SmallString<8> DirectoryStorage;
+ SmallString<128> AbsolutePath(WorkDirectory->getValue(DirectoryStorage));
+ llvm::sys::path::append(AbsolutePath, FileName);
+ llvm::sys::path::native(AbsolutePath, NativeFilePath);
+ } else {
+ llvm::sys::path::native(FileName, NativeFilePath);
+ }
+ llvm::sys::path::remove_dots(NativeFilePath, /*remove_dot_dot=*/true);
+ auto Cmd = CompileCommandRef(WorkDirectory, Source, Command, Object);
+
+ IndexByFile[NativeFilePath].push_back(Cmd);
+ AllCommands.push_back(Cmd);
+ MatchTrie.insert(NativeFilePath);
+
return true;
}
>From 6fbed3bc11520b13f172deb068e7cd349234df90 Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Thu, 21 May 2026 21:19:52 -0700
Subject: [PATCH 05/10] expose module functions from compile database (hack for
now to get up and running
---
.../clangd/GlobalCompilationDatabase.cpp | 1 +
clang-tools-extra/clangd/ProjectModules.h | 2 +-
.../clang/Tooling/CommonOptionsParser.h | 12 ++++++----
.../clang/Tooling/CompilationDatabase.h | 23 +++++++++++++++++--
.../include/clang/Tooling/JSONBuildDatabase.h | 4 ++++
.../clang/Tooling/JSONCompilationDatabase.h | 4 ++++
clang/lib/Tooling/CommonOptionsParser.cpp | 19 ++++++++++-----
clang/lib/Tooling/CompilationDatabase.cpp | 9 ++++++++
...ExpandResponseFilesCompilationDatabase.cpp | 8 +++++++
.../GuessTargetAndModeCompilationDatabase.cpp | 8 +++++++
.../InterpolatingCompilationDatabase.cpp | 21 +++++++++++------
clang/lib/Tooling/JSONBuildDatabase.cpp | 16 +++++++++----
clang/lib/Tooling/JSONCompilationDatabase.cpp | 10 ++++++++
.../Tooling/LocateToolCompilationDatabase.cpp | 8 +++++++
14 files changed, 120 insertions(+), 25 deletions(-)
diff --git a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
index 11205a1b7c646..522dc46764dc7 100644
--- a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
+++ b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
@@ -267,6 +267,7 @@ parseBuildDatabaseJSON(PathRef Path, llvm::StringRef Data, std::string &Error) {
}
return nullptr;
}
+// Adapt CDB-loading functions to a common interface for DirectoryCache::load().
static std::unique_ptr<tooling::CompilationDatabase>
parseComplilationCommandsJSON(PathRef Path, llvm::StringRef Data,
std::string &Error) {
diff --git a/clang-tools-extra/clangd/ProjectModules.h b/clang-tools-extra/clangd/ProjectModules.h
index fbf658bafe150..ee0136bfa51a7 100644
--- a/clang-tools-extra/clangd/ProjectModules.h
+++ b/clang-tools-extra/clangd/ProjectModules.h
@@ -60,7 +60,7 @@ class ProjectModules {
virtual ~ProjectModules() = default;
};
-/// Providing modules information for the project by scanning every file.
+/// Providing modules information for the project
std::unique_ptr<ProjectModules> getProjectModules(
std::shared_ptr<const clang::tooling::CompilationDatabase> CDB,
const ThreadsafeFS &TFS);
diff --git a/clang/include/clang/Tooling/CommonOptionsParser.h b/clang/include/clang/Tooling/CommonOptionsParser.h
index 09310fd9d2d4e..0a785858d3a35 100644
--- a/clang/include/clang/Tooling/CommonOptionsParser.h
+++ b/clang/include/clang/Tooling/CommonOptionsParser.h
@@ -92,9 +92,7 @@ class CommonOptionsParser {
const char *Overview = nullptr);
/// Returns a reference to the loaded compilations database.
- CompilationDatabase &getCompilations() {
- return *Compilations;
- }
+ CompilationDatabase &getCompilations() { return *Compilations; }
/// Returns a list of source file paths to process.
const std::vector<std::string> &getSourcePathList() const {
@@ -135,6 +133,10 @@ class ArgumentsAdjustingCompilations : public CompilationDatabase {
std::vector<CompileCommand> getAllCompileCommands() const override;
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override;
+ std::optional<std::string> getModuleName(StringRef FilePath) const override;
+
private:
std::unique_ptr<CompilationDatabase> Compilations;
std::vector<ArgumentsAdjuster> Adjusters;
@@ -143,7 +145,7 @@ class ArgumentsAdjustingCompilations : public CompilationDatabase {
adjustCommands(std::vector<CompileCommand> Commands) const;
};
-} // namespace tooling
-} // namespace clang
+} // namespace tooling
+} // namespace clang
#endif // LLVM_CLANG_TOOLING_COMMONOPTIONSPARSER_H
diff --git a/clang/include/clang/Tooling/CompilationDatabase.h b/clang/include/clang/Tooling/CompilationDatabase.h
index 36fe0812ebe97..91f86dd15126a 100644
--- a/clang/include/clang/Tooling/CompilationDatabase.h
+++ b/clang/include/clang/Tooling/CompilationDatabase.h
@@ -126,8 +126,23 @@ class CompilationDatabase {
/// $ clang++ -o production a.cc b.cc -DPRODUCTION
/// A compilation database representing the project would return both command
/// lines for a.cc and b.cc and only the first command line for t.cc.
- virtual std::vector<CompileCommand> getCompileCommands(
- StringRef FilePath) const = 0;
+ virtual std::vector<CompileCommand>
+ getCompileCommands(StringRef FilePath) const = 0;
+
+ // Returns all required modules for the specified file.
+ //
+ // This is the set of imported modules that are required to compile this file.
+ virtual std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const = 0;
+
+ // Returns the module name for the specified file.
+ //
+ // Will return no value when the module information is unknown (not provided)
+ // or unknowable (C). Empty string indicates this file does not produce a
+ // named module. Otherwise returns the name of the module exported by this
+ // file
+ virtual std::optional<std::string>
+ getModuleName(StringRef FilePath) const = 0;
/// Returns the list of all files available in the compilation database.
///
@@ -207,6 +222,10 @@ class FixedCompilationDatabase : public CompilationDatabase {
std::vector<CompileCommand>
getCompileCommands(StringRef FilePath) const override;
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override;
+ std::optional<std::string> getModuleName(StringRef FilePath) const override;
+
private:
/// This is built up to contain a single entry vector to be returned from
/// getCompileCommands after adding the positional argument.
diff --git a/clang/include/clang/Tooling/JSONBuildDatabase.h b/clang/include/clang/Tooling/JSONBuildDatabase.h
index 403aae91a19bd..4d8c4dc2f86e2 100644
--- a/clang/include/clang/Tooling/JSONBuildDatabase.h
+++ b/clang/include/clang/Tooling/JSONBuildDatabase.h
@@ -89,6 +89,10 @@ class JSONBuildDatabase : public CompilationDatabase {
/// database.
std::vector<CompileCommand> getAllCompileCommands() const override;
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override;
+ std::optional<std::string> getModuleName(StringRef FilePath) const override;
+
private:
/// Constructs a JSON build database on a memory buffer.
JSONBuildDatabase(std::unique_ptr<llvm::MemoryBuffer> Database)
diff --git a/clang/include/clang/Tooling/JSONCompilationDatabase.h b/clang/include/clang/Tooling/JSONCompilationDatabase.h
index 96582457c63d5..1762a5e523ef1 100644
--- a/clang/include/clang/Tooling/JSONCompilationDatabase.h
+++ b/clang/include/clang/Tooling/JSONCompilationDatabase.h
@@ -92,6 +92,10 @@ class JSONCompilationDatabase : public CompilationDatabase {
/// database.
std::vector<CompileCommand> getAllCompileCommands() const override;
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override;
+ std::optional<std::string> getModuleName(StringRef FilePath) const override;
+
private:
/// Constructs a JSON compilation database on a memory buffer.
JSONCompilationDatabase(std::unique_ptr<llvm::MemoryBuffer> Database,
diff --git a/clang/lib/Tooling/CommonOptionsParser.cpp b/clang/lib/Tooling/CommonOptionsParser.cpp
index c8c3ca98323e2..f51047cb8f4b7 100644
--- a/clang/lib/Tooling/CommonOptionsParser.cpp
+++ b/clang/lib/Tooling/CommonOptionsParser.cpp
@@ -57,13 +57,12 @@ void ArgumentsAdjustingCompilations::appendArgumentsAdjuster(
Adjusters.push_back(std::move(Adjuster));
}
-std::vector<CompileCommand> ArgumentsAdjustingCompilations::getCompileCommands(
- StringRef FilePath) const {
+std::vector<CompileCommand>
+ArgumentsAdjustingCompilations::getCompileCommands(StringRef FilePath) const {
return adjustCommands(Compilations->getCompileCommands(FilePath));
}
-std::vector<std::string>
-ArgumentsAdjustingCompilations::getAllFiles() const {
+std::vector<std::string> ArgumentsAdjustingCompilations::getAllFiles() const {
return Compilations->getAllFiles();
}
@@ -80,6 +79,15 @@ std::vector<CompileCommand> ArgumentsAdjustingCompilations::adjustCommands(
return Commands;
}
+std::vector<std::string>
+ArgumentsAdjustingCompilations::getRequiredModules(StringRef FilePath) const {
+ return {};
+}
+std::optional<std::string>
+ArgumentsAdjustingCompilations::getModuleName(StringRef FilePath) const {
+ return std::nullopt;
+}
+
llvm::Error CommonOptionsParser::init(
int &argc, const char **argv, cl::OptionCategory &Category,
llvm::cl::NumOccurrencesFlag OccurrencesFlag, const char *Overview) {
@@ -140,8 +148,7 @@ llvm::Error CommonOptionsParser::init(
}
}
auto AdjustingCompilations =
- std::make_unique<ArgumentsAdjustingCompilations>(
- std::move(Compilations));
+ std::make_unique<ArgumentsAdjustingCompilations>(std::move(Compilations));
Adjuster =
getInsertArgumentAdjuster(ArgsBefore, ArgumentInsertPosition::BEGIN);
Adjuster = combineAdjusters(
diff --git a/clang/lib/Tooling/CompilationDatabase.cpp b/clang/lib/Tooling/CompilationDatabase.cpp
index 187e1223f5cef..1b92c238f3d64 100644
--- a/clang/lib/Tooling/CompilationDatabase.cpp
+++ b/clang/lib/Tooling/CompilationDatabase.cpp
@@ -382,6 +382,15 @@ FixedCompilationDatabase::getCompileCommands(StringRef FilePath) const {
return Result;
}
+std::vector<std::string>
+FixedCompilationDatabase::getRequiredModules(StringRef FilePath) const {
+ return {};
+}
+std::optional<std::string>
+FixedCompilationDatabase::getModuleName(StringRef FilePath) const {
+ return std::nullopt;
+}
+
namespace {
class FixedCompilationDatabasePlugin : public CompilationDatabasePlugin {
diff --git a/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp b/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
index 22d0057a28f81..360c1ba8b9f06 100644
--- a/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
+++ b/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
@@ -43,6 +43,14 @@ class ExpandResponseFilesDatabase : public CompilationDatabase {
return expand(Base->getAllCompileCommands());
}
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override {
+ return {};
+ }
+ std::optional<std::string> getModuleName(StringRef FilePath) const override {
+ return std::nullopt;
+ }
+
private:
std::vector<CompileCommand> expand(std::vector<CompileCommand> Cmds) const {
for (auto &Cmd : Cmds)
diff --git a/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp b/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
index b6c1c0952aca9..eeb73f9e1bc45 100644
--- a/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
+++ b/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
@@ -34,6 +34,14 @@ class TargetAndModeAdderDatabase : public CompilationDatabase {
return addTargetAndMode(Base->getCompileCommands(FilePath));
}
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override {
+ return {};
+ }
+ std::optional<std::string> getModuleName(StringRef FilePath) const override {
+ return std::nullopt;
+ }
+
private:
std::vector<CompileCommand>
addTargetAndMode(std::vector<CompileCommand> Cmds) const {
diff --git a/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp b/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
index 08ee02f639a4f..b67421e4de4d1 100644
--- a/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
+++ b/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
@@ -201,10 +201,9 @@ struct TransferableCommand {
// Strip input and output files.
if (Opt.matches(OPT_INPUT) || Opt.matches(OPT_o) ||
- (ClangCLMode && (Opt.matches(OPT__SLASH_Fa) ||
- Opt.matches(OPT__SLASH_Fe) ||
- Opt.matches(OPT__SLASH_Fi) ||
- Opt.matches(OPT__SLASH_Fo))))
+ (ClangCLMode &&
+ (Opt.matches(OPT__SLASH_Fa) || Opt.matches(OPT__SLASH_Fe) ||
+ Opt.matches(OPT__SLASH_Fi) || Opt.matches(OPT__SLASH_Fo))))
continue;
// ...including when the inputs are passed after --.
@@ -224,8 +223,8 @@ struct TransferableCommand {
continue;
}
- Cmd.CommandLine.insert(Cmd.CommandLine.end(),
- OldArgs.data() + OldPos, OldArgs.data() + Pos);
+ Cmd.CommandLine.insert(Cmd.CommandLine.end(), OldArgs.data() + OldPos,
+ OldArgs.data() + Pos);
}
// Make use of -std iff -x was missing.
@@ -544,7 +543,7 @@ class FileIndex {
StringSaver Strings;
// Indexes of candidates by certain substrings.
// String is lowercase and sorted, index points into OriginalPaths.
- std::vector<SubstringAndIndex> Paths; // Full path.
+ std::vector<SubstringAndIndex> Paths; // Full path.
// Lang types obtained by guessing on the corresponding path. I-th element is
// a type for the I-th path.
std::vector<types::ID> Types;
@@ -584,6 +583,14 @@ class InterpolatingCompilationDatabase : public CompilationDatabase {
return Inner->getAllCompileCommands();
}
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override {
+ return {};
+ }
+ std::optional<std::string> getModuleName(StringRef FilePath) const override {
+ return std::nullopt;
+ }
+
private:
std::unique_ptr<CompilationDatabase> Inner;
FileIndex Index;
diff --git a/clang/lib/Tooling/JSONBuildDatabase.cpp b/clang/lib/Tooling/JSONBuildDatabase.cpp
index 4c3e75610c989..2ad4313b99437 100644
--- a/clang/lib/Tooling/JSONBuildDatabase.cpp
+++ b/clang/lib/Tooling/JSONBuildDatabase.cpp
@@ -30,10 +30,8 @@
#include <cassert>
#include <memory>
#include <optional>
-#include <signal.h>
#include <string>
#include <system_error>
-#include <thread>
#include <unistd.h>
#include <utility>
#include <vector>
@@ -42,13 +40,13 @@ using namespace clang;
using namespace tooling;
namespace {
-// This plugin locates a nearby compile_command.json file, and also infers
+// This plugin locates a nearby build_database.json file, and also infers
// compile commands for files not present in the database.
class JSONBuildDatabasePlugin : public CompilationDatabasePlugin {
std::unique_ptr<CompilationDatabase>
loadFromDirectory(StringRef Directory, std::string &ErrorMessage) override {
SmallString<1024> JSONDatabasePath(Directory);
- llvm::sys::path::append(JSONDatabasePath, "compile_commands.json");
+ llvm::sys::path::append(JSONDatabasePath, "build_database.json");
auto Base = JSONBuildDatabase::loadFromFile(JSONDatabasePath, ErrorMessage);
return Base ? inferTargetAndDriverMode(
inferMissingCompileCommands(expandResponseFiles(
@@ -135,6 +133,16 @@ std::vector<CompileCommand> JSONBuildDatabase::getAllCompileCommands() const {
return Commands;
}
+std::vector<std::string>
+JSONBuildDatabase::getRequiredModules(StringRef FilePath) const {
+ return {};
+}
+
+std::optional<std::string>
+JSONBuildDatabase::getModuleName(StringRef FilePath) const {
+ return "tset";
+}
+
static llvm::StringRef stripExecutableExtension(llvm::StringRef Name) {
Name.consume_back(".exe");
return Name;
diff --git a/clang/lib/Tooling/JSONCompilationDatabase.cpp b/clang/lib/Tooling/JSONCompilationDatabase.cpp
index 9155b01f3fe13..adaf94a1e0e9c 100644
--- a/clang/lib/Tooling/JSONCompilationDatabase.cpp
+++ b/clang/lib/Tooling/JSONCompilationDatabase.cpp
@@ -261,6 +261,16 @@ JSONCompilationDatabase::getAllCompileCommands() const {
return Commands;
}
+std::vector<std::string>
+JSONCompilationDatabase::getRequiredModules(StringRef FilePath) const {
+ return {};
+}
+
+std::optional<std::string>
+JSONCompilationDatabase::getModuleName(StringRef FilePath) const {
+ return std::nullopt;
+}
+
static llvm::StringRef stripExecutableExtension(llvm::StringRef Name) {
Name.consume_back(".exe");
return Name;
diff --git a/clang/lib/Tooling/LocateToolCompilationDatabase.cpp b/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
index 033f69f3760c6..34ced90c32379 100644
--- a/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
+++ b/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
@@ -36,6 +36,14 @@ class LocationAdderDatabase : public CompilationDatabase {
return addLocation(Base->getCompileCommands(FilePath));
}
+ std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const override {
+ return {};
+ }
+ std::optional<std::string> getModuleName(StringRef FilePath) const override {
+ return std::nullopt;
+ }
+
private:
std::vector<CompileCommand>
addLocation(std::vector<CompileCommand> Cmds) const {
>From 4d577dc44565187086e5d9f29971660ceba450fa Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Fri, 22 May 2026 12:57:21 -0700
Subject: [PATCH 06/10] Finish building up set lookup and implement module
manager interface
---
clang-tools-extra/clangd/ProjectModules.cpp | 109 ++++++++-
.../clang/Tooling/CommonOptionsParser.h | 4 -
.../clang/Tooling/CompilationDatabase.h | 55 +++--
.../include/clang/Tooling/JSONBuildDatabase.h | 62 +++--
.../clang/Tooling/JSONCompilationDatabase.h | 4 -
clang/lib/Tooling/CommonOptionsParser.cpp | 9 -
clang/lib/Tooling/CompilationDatabase.cpp | 9 -
...ExpandResponseFilesCompilationDatabase.cpp | 8 -
.../GuessTargetAndModeCompilationDatabase.cpp | 8 -
.../InterpolatingCompilationDatabase.cpp | 8 -
clang/lib/Tooling/JSONBuildDatabase.cpp | 214 +++++++++++++++---
clang/lib/Tooling/JSONCompilationDatabase.cpp | 10 -
.../Tooling/LocateToolCompilationDatabase.cpp | 8 -
13 files changed, 358 insertions(+), 150 deletions(-)
diff --git a/clang-tools-extra/clangd/ProjectModules.cpp b/clang-tools-extra/clangd/ProjectModules.cpp
index d3727171bff12..50f094be3c0cf 100644
--- a/clang-tools-extra/clangd/ProjectModules.cpp
+++ b/clang-tools-extra/clangd/ProjectModules.cpp
@@ -347,6 +347,80 @@ class ScanningAllProjectModules : public ProjectModules {
CommandMangler Mangler;
};
+/// Reads project module information directly from build database.
+///
+/// The build database contains the translation unit sets and their visibility
+/// into other sets. This heirarchy allows for a single project to contain
+/// multiple isolated linked programs that guarantee unique module names within
+/// the context of a set and all visible sets while allowing for duplicate
+/// module names within the larger project.
+///
+/// Note the build database can be stale, so results from this backend
+/// should be treated as preferred hints rather than unquestionable truth.
+/// The compound layer below validates or falls back when needed.
+class BuildDatabaseProjectModules : public ProjectModules {
+public:
+ BuildDatabaseProjectModules(
+ std::shared_ptr<const clang::tooling::CompilationDatabase> CDB)
+ : CDB(std::move(CDB)) {}
+
+ std::vector<std::string> getRequiredModules(PathRef File) override {
+ const auto *ModuleManager = CDB->getModuleManager();
+ if (ModuleManager)
+ return ModuleManager->getRequiredModules(File);
+
+ return {};
+ }
+
+ std::string getModuleNameForSource(PathRef File) override {
+ const auto *ModuleManager = CDB->getModuleManager();
+ if (ModuleManager) {
+ auto ModuleName = ModuleManager->getModuleName(File);
+ if (ModuleName)
+ return *ModuleName;
+ }
+
+ return "";
+ }
+
+ ModuleNameState getModuleNameState(llvm::StringRef ModuleName) override {
+ const auto *ModuleManager = CDB->getModuleManager();
+ if (ModuleManager)
+ return (ModuleNameState)ModuleManager->getModuleNameState(ModuleName);
+
+ return ModuleNameState::Unknown;
+ }
+
+ std::string getSourceForModuleName(llvm::StringRef ModuleName,
+ PathRef RequiredSourceFile) override {
+ const auto *ModuleManager = CDB->getModuleManager();
+ if (ModuleManager)
+ return ModuleManager->getSourceForModuleName(ModuleName,
+ RequiredSourceFile);
+
+ return "";
+ }
+
+ void setCommandMangler(CommandMangler Mangler) override {
+ this->Mangler = std::move(Mangler);
+ }
+
+private:
+ std::shared_ptr<const clang::tooling::CompilationDatabase> CDB;
+ CommandMangler Mangler;
+
+ llvm::StringMap<std::string> PCMToSource;
+
+ using DistinctSourceSet = llvm::StringSet<>;
+ llvm::StringMap<DistinctSourceSet> ModuleNameToDistinctSources;
+
+ struct RecoveredModuleName {
+ std::string Name;
+ bool Ambiguous = false;
+ };
+ llvm::StringMap<RecoveredModuleName> SourceToModuleName;
+};
+
/// Reads project module information directly from compile commands.
///
/// The key observation is that compile commands may already encode the mapping
@@ -528,7 +602,8 @@ class CompoundProjectModules : public ProjectModules {
CompoundProjectModules(
std::shared_ptr<const clang::tooling::CompilationDatabase> CDB,
const ThreadsafeFS &TFS)
- : CompileCommands(
+ : BuildDatabase(std::make_unique<BuildDatabaseProjectModules>(CDB)),
+ CompileCommands(
std::make_unique<CompileCommandsProjectModules>(CDB, TFS)),
Scanning(
std::make_unique<ScanningAllProjectModules>(std::move(CDB), TFS)) {}
@@ -545,20 +620,33 @@ class CompoundProjectModules : public ProjectModules {
std::string getSourceForModuleName(llvm::StringRef ModuleName,
PathRef RequiredSourceFile) override {
- auto FromCompileCommands =
- CompileCommands->getSourceForModuleName(ModuleName, RequiredSourceFile);
- // Check if the source still declares the module.
- // This is to validate compile-command-derived results may be stale and
- // scan a single file is fast enough. We just don't want to scan the project
- // entirely.
- if (!FromCompileCommands.empty() &&
- Scanning->getModuleNameForSource(FromCompileCommands) == ModuleName)
- return FromCompileCommands;
+ auto FromBuildDatabase =
+ BuildDatabase->getSourceForModuleName(ModuleName, RequiredSourceFile);
+ if (!FromBuildDatabase.empty()) {
+ // Check if the source still declares the module.
+ // This is to validate compile-command-derived results may be stale and
+ // scan a single file is fast enough. We just don't want to scan the
+ // project entirely.
+ if (Scanning->getModuleNameForSource(FromBuildDatabase) == ModuleName)
+ return FromBuildDatabase;
+ } else {
+ // The build database does not have module knowledge. Fall back to parse
+ // compile command.
+ auto FromCompileCommands = CompileCommands->getSourceForModuleName(
+ ModuleName, RequiredSourceFile);
+ if (!FromCompileCommands.empty() &&
+ Scanning->getModuleNameForSource(FromCompileCommands) == ModuleName)
+ return FromCompileCommands;
+ }
return Scanning->getSourceForModuleName(ModuleName, RequiredSourceFile);
}
ModuleNameState getModuleNameState(llvm::StringRef ModuleName) override {
+ auto FromBuildDatabase = BuildDatabase->getModuleNameState(ModuleName);
+ if (FromBuildDatabase != ModuleNameState::Unknown)
+ return FromBuildDatabase;
+
auto FromCompileCommands = CompileCommands->getModuleNameState(ModuleName);
if (FromCompileCommands != ModuleNameState::Unknown)
return FromCompileCommands;
@@ -577,6 +665,7 @@ class CompoundProjectModules : public ProjectModules {
}
private:
+ std::unique_ptr<BuildDatabaseProjectModules> BuildDatabase;
std::unique_ptr<CompileCommandsProjectModules> CompileCommands;
std::unique_ptr<ScanningAllProjectModules> Scanning;
CommandMangler Mangler;
diff --git a/clang/include/clang/Tooling/CommonOptionsParser.h b/clang/include/clang/Tooling/CommonOptionsParser.h
index 0a785858d3a35..98deaf174fb53 100644
--- a/clang/include/clang/Tooling/CommonOptionsParser.h
+++ b/clang/include/clang/Tooling/CommonOptionsParser.h
@@ -133,10 +133,6 @@ class ArgumentsAdjustingCompilations : public CompilationDatabase {
std::vector<CompileCommand> getAllCompileCommands() const override;
- std::vector<std::string>
- getRequiredModules(StringRef FilePath) const override;
- std::optional<std::string> getModuleName(StringRef FilePath) const override;
-
private:
std::unique_ptr<CompilationDatabase> Compilations;
std::vector<ArgumentsAdjuster> Adjusters;
diff --git a/clang/include/clang/Tooling/CompilationDatabase.h b/clang/include/clang/Tooling/CompilationDatabase.h
index 91f86dd15126a..c0d35a07ee94e 100644
--- a/clang/include/clang/Tooling/CompilationDatabase.h
+++ b/clang/include/clang/Tooling/CompilationDatabase.h
@@ -76,6 +76,36 @@ struct CompileCommand {
}
};
+class ModuleManager {
+public:
+ enum class ModuleNameState {
+ Unknown,
+ Unique,
+ Multiple,
+ };
+
+ // Returns all required modules for the specified file.
+ //
+ // This is the set of imported modules that are required to compile this file.
+ virtual std::vector<std::string>
+ getRequiredModules(StringRef FilePath) const = 0;
+
+ // Returns the module name for the specified file.
+ //
+ // Will return no value when the module information is unknown (not provided)
+ // or unknowable (C). Empty string indicates this file does not produce a
+ // named module. Otherwise returns the name of the module exported by this
+ // file
+ virtual std::optional<std::string>
+ getModuleName(StringRef FilePath) const = 0;
+
+ virtual ModuleNameState getModuleNameState(StringRef ModuleName) const = 0;
+
+ virtual std::string
+ getSourceForModuleName(StringRef ModuleName,
+ StringRef RequiredSourceFile) const = 0;
+};
+
/// Interface for compilation databases.
///
/// A compilation database allows the user to retrieve compile command lines
@@ -129,21 +159,6 @@ class CompilationDatabase {
virtual std::vector<CompileCommand>
getCompileCommands(StringRef FilePath) const = 0;
- // Returns all required modules for the specified file.
- //
- // This is the set of imported modules that are required to compile this file.
- virtual std::vector<std::string>
- getRequiredModules(StringRef FilePath) const = 0;
-
- // Returns the module name for the specified file.
- //
- // Will return no value when the module information is unknown (not provided)
- // or unknowable (C). Empty string indicates this file does not produce a
- // named module. Otherwise returns the name of the module exported by this
- // file
- virtual std::optional<std::string>
- getModuleName(StringRef FilePath) const = 0;
-
/// Returns the list of all files available in the compilation database.
///
/// By default, returns nothing. Implementations should override this if they
@@ -160,6 +175,12 @@ class CompilationDatabase {
/// By default, this is implemented in terms of getAllFiles() and
/// getCompileCommands(). Subclasses may override this for efficiency.
virtual std::vector<CompileCommand> getAllCompileCommands() const;
+
+ /// Returns the module manager.
+ ///
+ /// By default, returns nothing. Implementations should override this if they
+ /// can enumerate their source files.
+ virtual const ModuleManager *getModuleManager() const { return nullptr; }
};
/// A compilation database that returns a single compile command line.
@@ -222,10 +243,6 @@ class FixedCompilationDatabase : public CompilationDatabase {
std::vector<CompileCommand>
getCompileCommands(StringRef FilePath) const override;
- std::vector<std::string>
- getRequiredModules(StringRef FilePath) const override;
- std::optional<std::string> getModuleName(StringRef FilePath) const override;
-
private:
/// This is built up to contain a single entry vector to be returned from
/// getCompileCommands after adding the positional argument.
diff --git a/clang/include/clang/Tooling/JSONBuildDatabase.h b/clang/include/clang/Tooling/JSONBuildDatabase.h
index 4d8c4dc2f86e2..38c401e05ad66 100644
--- a/clang/include/clang/Tooling/JSONBuildDatabase.h
+++ b/clang/include/clang/Tooling/JSONBuildDatabase.h
@@ -20,6 +20,7 @@
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/StringMap.h"
#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSet.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/SourceMgr.h"
#include "llvm/Support/YAMLParser.h"
@@ -57,7 +58,7 @@ namespace tooling {
///
/// JSON build databases can for example be generated in CMake projects
/// by setting the flag -DCMAKE_EXPORT_BUILD_DATABASE.
-class JSONBuildDatabase : public CompilationDatabase {
+class JSONBuildDatabase : public CompilationDatabase, ModuleManager {
public:
/// Loads a JSON build database from the specified file.
///
@@ -89,16 +90,43 @@ class JSONBuildDatabase : public CompilationDatabase {
/// database.
std::vector<CompileCommand> getAllCompileCommands() const override;
+ const ModuleManager *getModuleManager() const override;
+
std::vector<std::string>
getRequiredModules(StringRef FilePath) const override;
std::optional<std::string> getModuleName(StringRef FilePath) const override;
+ ModuleNameState getModuleNameState(StringRef ModuleName) const override;
+
+ std::string
+ getSourceForModuleName(StringRef ModuleName,
+ StringRef RequiredSourceFile) const override;
+
private:
/// Constructs a JSON build database on a memory buffer.
JSONBuildDatabase(std::unique_ptr<llvm::MemoryBuffer> Database)
: Database(std::move(Database)),
YAMLStream(this->Database->getBuffer(), SM) {}
+ // Container for a compile command references where 'commandline'
+ // points to the corresponding scalar nodes in the YAML stream.
+ // The output field may be a nullptr.
+ struct TranslationUnitRef {
+ llvm::yaml::ScalarNode *SetName;
+ llvm::yaml::ScalarNode *Directory;
+ llvm::yaml::ScalarNode *Filename;
+ std::vector<llvm::yaml::ScalarNode *> CommandLine;
+ llvm::yaml::ScalarNode *Output;
+ llvm::yaml::ScalarNode *ProvidesModuleName;
+ llvm::yaml::ScalarNode *ProvidesModulePCM;
+ std::vector<llvm::yaml::ScalarNode *> RequiredModules;
+ };
+
+ struct TranslationUnitSet {
+ std::vector<llvm::yaml::ScalarNode *> VisibleSets;
+ std::vector<TranslationUnitRef> TranslationUnits;
+ };
+
/// Parses the database file and creates the index.
///
/// Returns whether parsing succeeded. Sets ErrorMessage if parsing
@@ -106,30 +134,28 @@ class JSONBuildDatabase : public CompilationDatabase {
bool parse(std::string &ErrorMessage);
bool parseRoot(std::string &ErrorMessage, llvm::yaml::MappingNode *Object);
bool parseSet(std::string &ErrorMessage, llvm::yaml::MappingNode *Object);
- bool parseTU(std::string &ErrorMessage, llvm::yaml::MappingNode *Object);
+ bool parseTU(std::string &ErrorMessage, llvm::yaml::MappingNode *Object,
+ TranslationUnitRef &TURef);
- // Tuple (directory, filename, commandline, output) where 'commandline'
- // points to the corresponding scalar nodes in the YAML stream.
- // If the command line contains a single argument, it is a shell-escaped
- // command line.
- // Otherwise, each entry in the command line vector is a literal
- // argument to the compiler.
- // The output field may be a nullptr.
- using CompileCommandRef =
- std::tuple<llvm::yaml::ScalarNode *, llvm::yaml::ScalarNode *,
- std::vector<llvm::yaml::ScalarNode *>,
- llvm::yaml::ScalarNode *>;
+ const TranslationUnitRef *getTUForSource(StringRef FilePath) const;
+ const TranslationUnitRef *getTUForModule(StringRef ModuleName,
+ StringRef SetName) const;
- /// Converts the given array of CompileCommandRefs to CompileCommands.
- void getCommands(ArrayRef<CompileCommandRef> CommandsRef,
+ /// Converts the given array of TranslationUnitRefs to CompileCommands.
+ void getCommands(ArrayRef<TranslationUnitRef> CommandsRef,
std::vector<CompileCommand> &Commands) const;
- // Maps file paths to the compile command lines for that file.
- llvm::StringMap<std::vector<CompileCommandRef>> IndexByFile;
+ // Maps file paths to the translation units for that file.
+ llvm::StringMap<std::vector<TranslationUnitRef>> IndexByFile;
+ llvm::StringMap<TranslationUnitSet> IndexBySet;
/// All the compile commands in the order that they were provided in the
/// JSON stream.
- std::vector<CompileCommandRef> AllCommands;
+ std::vector<TranslationUnitRef> AllCommands;
+
+ // Module name state lookup to track unique names
+ using DistinctSourceSet = llvm::StringSet<>;
+ llvm::StringMap<DistinctSourceSet> ModuleNameToDistinctSources;
FileMatchTrie MatchTrie;
diff --git a/clang/include/clang/Tooling/JSONCompilationDatabase.h b/clang/include/clang/Tooling/JSONCompilationDatabase.h
index 1762a5e523ef1..96582457c63d5 100644
--- a/clang/include/clang/Tooling/JSONCompilationDatabase.h
+++ b/clang/include/clang/Tooling/JSONCompilationDatabase.h
@@ -92,10 +92,6 @@ class JSONCompilationDatabase : public CompilationDatabase {
/// database.
std::vector<CompileCommand> getAllCompileCommands() const override;
- std::vector<std::string>
- getRequiredModules(StringRef FilePath) const override;
- std::optional<std::string> getModuleName(StringRef FilePath) const override;
-
private:
/// Constructs a JSON compilation database on a memory buffer.
JSONCompilationDatabase(std::unique_ptr<llvm::MemoryBuffer> Database,
diff --git a/clang/lib/Tooling/CommonOptionsParser.cpp b/clang/lib/Tooling/CommonOptionsParser.cpp
index f51047cb8f4b7..980b38bc0f4d6 100644
--- a/clang/lib/Tooling/CommonOptionsParser.cpp
+++ b/clang/lib/Tooling/CommonOptionsParser.cpp
@@ -79,15 +79,6 @@ std::vector<CompileCommand> ArgumentsAdjustingCompilations::adjustCommands(
return Commands;
}
-std::vector<std::string>
-ArgumentsAdjustingCompilations::getRequiredModules(StringRef FilePath) const {
- return {};
-}
-std::optional<std::string>
-ArgumentsAdjustingCompilations::getModuleName(StringRef FilePath) const {
- return std::nullopt;
-}
-
llvm::Error CommonOptionsParser::init(
int &argc, const char **argv, cl::OptionCategory &Category,
llvm::cl::NumOccurrencesFlag OccurrencesFlag, const char *Overview) {
diff --git a/clang/lib/Tooling/CompilationDatabase.cpp b/clang/lib/Tooling/CompilationDatabase.cpp
index 1b92c238f3d64..187e1223f5cef 100644
--- a/clang/lib/Tooling/CompilationDatabase.cpp
+++ b/clang/lib/Tooling/CompilationDatabase.cpp
@@ -382,15 +382,6 @@ FixedCompilationDatabase::getCompileCommands(StringRef FilePath) const {
return Result;
}
-std::vector<std::string>
-FixedCompilationDatabase::getRequiredModules(StringRef FilePath) const {
- return {};
-}
-std::optional<std::string>
-FixedCompilationDatabase::getModuleName(StringRef FilePath) const {
- return std::nullopt;
-}
-
namespace {
class FixedCompilationDatabasePlugin : public CompilationDatabasePlugin {
diff --git a/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp b/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
index 360c1ba8b9f06..22d0057a28f81 100644
--- a/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
+++ b/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
@@ -43,14 +43,6 @@ class ExpandResponseFilesDatabase : public CompilationDatabase {
return expand(Base->getAllCompileCommands());
}
- std::vector<std::string>
- getRequiredModules(StringRef FilePath) const override {
- return {};
- }
- std::optional<std::string> getModuleName(StringRef FilePath) const override {
- return std::nullopt;
- }
-
private:
std::vector<CompileCommand> expand(std::vector<CompileCommand> Cmds) const {
for (auto &Cmd : Cmds)
diff --git a/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp b/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
index eeb73f9e1bc45..b6c1c0952aca9 100644
--- a/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
+++ b/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
@@ -34,14 +34,6 @@ class TargetAndModeAdderDatabase : public CompilationDatabase {
return addTargetAndMode(Base->getCompileCommands(FilePath));
}
- std::vector<std::string>
- getRequiredModules(StringRef FilePath) const override {
- return {};
- }
- std::optional<std::string> getModuleName(StringRef FilePath) const override {
- return std::nullopt;
- }
-
private:
std::vector<CompileCommand>
addTargetAndMode(std::vector<CompileCommand> Cmds) const {
diff --git a/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp b/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
index b67421e4de4d1..93b1a6eab9e28 100644
--- a/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
+++ b/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
@@ -583,14 +583,6 @@ class InterpolatingCompilationDatabase : public CompilationDatabase {
return Inner->getAllCompileCommands();
}
- std::vector<std::string>
- getRequiredModules(StringRef FilePath) const override {
- return {};
- }
- std::optional<std::string> getModuleName(StringRef FilePath) const override {
- return std::nullopt;
- }
-
private:
std::unique_ptr<CompilationDatabase> Inner;
FileIndex Index;
diff --git a/clang/lib/Tooling/JSONBuildDatabase.cpp b/clang/lib/Tooling/JSONBuildDatabase.cpp
index 2ad4313b99437..4dd54ad891556 100644
--- a/clang/lib/Tooling/JSONBuildDatabase.cpp
+++ b/clang/lib/Tooling/JSONBuildDatabase.cpp
@@ -112,11 +112,11 @@ JSONBuildDatabase::getCompileCommands(StringRef FilePath) const {
StringRef Match = MatchTrie.findEquivalent(NativeFilePath, ES);
if (Match.empty())
return {};
- const auto CommandsRefI = IndexByFile.find(Match);
- if (CommandsRefI == IndexByFile.end())
+ const auto TURefI = IndexByFile.find(Match);
+ if (TURefI == IndexByFile.end())
return {};
std::vector<CompileCommand> Commands;
- getCommands(CommandsRefI->getValue(), Commands);
+ getCommands(TURefI->getValue(), Commands);
return Commands;
}
@@ -133,14 +133,107 @@ std::vector<CompileCommand> JSONBuildDatabase::getAllCompileCommands() const {
return Commands;
}
+const ModuleManager *JSONBuildDatabase::getModuleManager() const {
+ return this;
+}
+
std::vector<std::string>
JSONBuildDatabase::getRequiredModules(StringRef FilePath) const {
- return {};
+
+ const auto *TURef = getTUForSource(FilePath);
+ std::vector<std::string> RequiredModules;
+ if (TURef) {
+ for (const auto &RequiredModule : TURef->RequiredModules) {
+ SmallString<8> RequiredModuleStorage;
+ RequiredModules.emplace_back(
+ RequiredModule->getValue(RequiredModuleStorage));
+ }
+ }
+ return RequiredModules;
}
std::optional<std::string>
JSONBuildDatabase::getModuleName(StringRef FilePath) const {
- return "tset";
+ const auto *TURef = getTUForSource(FilePath);
+ if (TURef && TURef->ProvidesModuleName) {
+ SmallString<8> ModuleNameStorage;
+ return TURef->ProvidesModuleName->getValue(ModuleNameStorage).str();
+ }
+ return std::nullopt;
+}
+
+ModuleManager::ModuleNameState
+JSONBuildDatabase::getModuleNameState(StringRef ModuleName) const {
+ auto It = ModuleNameToDistinctSources.find(ModuleName);
+ if (It == ModuleNameToDistinctSources.end())
+ return ModuleNameState::Unknown;
+ return It->second.size() > 1 ? ModuleNameState::Multiple
+ : ModuleNameState::Unique;
+}
+
+std::string
+JSONBuildDatabase::getSourceForModuleName(StringRef ModuleName,
+ StringRef RequiredSourceFile) const {
+
+ const auto *RequiredSourceTURef = getTUForSource(RequiredSourceFile);
+ if (RequiredSourceTURef) {
+ SmallString<8> SetNameStorage;
+ SmallString<8> FilenameStorage;
+ auto SetName = RequiredSourceTURef->SetName->getValue(SetNameStorage);
+ // First attempt to find the matching module in the current set
+ const auto *ModuleTURef = getTUForModule(ModuleName, SetName);
+ if (ModuleTURef)
+ return ModuleTURef->Filename->getValue(FilenameStorage).str();
+ // Could not find in current set, check all visible sets
+ const auto TUSetI = IndexBySet.find(SetName);
+ if (TUSetI == IndexBySet.end())
+ return {};
+ for (const auto &VisibleSet : TUSetI->getValue().VisibleSets) {
+ ModuleTURef =
+ getTUForModule(ModuleName, VisibleSet->getValue(SetNameStorage));
+ if (ModuleTURef)
+ return ModuleTURef->Filename->getValue(FilenameStorage).str();
+ }
+ }
+ return {};
+}
+
+const JSONBuildDatabase::TranslationUnitRef *
+JSONBuildDatabase::getTUForSource(StringRef FilePath) const {
+ SmallString<128> NativeFilePath;
+ llvm::sys::path::native(FilePath, NativeFilePath);
+
+ std::string Error;
+ llvm::raw_string_ostream ES(Error);
+ StringRef Match = MatchTrie.findEquivalent(NativeFilePath, ES);
+ if (Match.empty())
+ return {};
+ const auto TURefI = IndexByFile.find(Match);
+ if (TURefI == IndexByFile.end())
+ return {};
+ // Return the first reference in the build database
+ // Not ideal, but without context this is the best we can do
+ for (const auto &TURef : TURefI->getValue()) {
+ return &TURef;
+ }
+ return nullptr;
+}
+
+const JSONBuildDatabase::TranslationUnitRef *
+JSONBuildDatabase::getTUForModule(StringRef ModuleName,
+ StringRef SetName) const {
+ const auto TUSetI = IndexBySet.find(SetName);
+ if (TUSetI == IndexBySet.end())
+ return {};
+ // Return the first reference in the build database
+ // Not ideal, but without context this is the best we can do
+ for (const auto &TURef : TUSetI->getValue().TranslationUnits) {
+
+ SmallString<8> ModuleNameStorage;
+ if (TURef.ProvidesModuleName->getValue(ModuleNameStorage) == ModuleName)
+ return &TURef;
+ }
+ return nullptr;
}
static llvm::StringRef stripExecutableExtension(llvm::StringRef Name) {
@@ -167,8 +260,9 @@ static bool unwrapCommand(std::vector<std::string> &Args) {
// We don't even notice this case, and all is well.
//
// We need to distinguish between the first and second case.
- // The wrappers themselves don't take flags, so Args[1] is a compiler flag,
- // an input file, or a compiler. Inputs have extensions, compilers don't.
+ // The wrappers themselves don't take flags, so Args[1] is a compiler
+ // flag, an input file, or a compiler. Inputs have extensions, compilers
+ // don't.
bool HasCompiler =
(Args[1][0] != '-') &&
!llvm::sys::path::has_extension(stripExecutableExtension(Args[1]));
@@ -187,24 +281,25 @@ nodeToCommandLine(const std::vector<llvm::yaml::ScalarNode *> &Nodes) {
std::vector<std::string> Arguments;
for (const auto *Node : Nodes)
Arguments.push_back(std::string(Node->getValue(Storage)));
- // There may be multiple wrappers: using distcc and ccache together is common.
+ // There may be multiple wrappers: using distcc and ccache together is
+ // common.
while (unwrapCommand(Arguments))
;
return Arguments;
}
void JSONBuildDatabase::getCommands(
- ArrayRef<CompileCommandRef> CommandsRef,
+ ArrayRef<TranslationUnitRef> TUsRef,
std::vector<CompileCommand> &Commands) const {
- for (const auto &CommandRef : CommandsRef) {
+ for (const auto &TURef : TUsRef) {
SmallString<8> DirectoryStorage;
SmallString<32> FilenameStorage;
SmallString<32> OutputStorage;
- auto Output = std::get<3>(CommandRef);
- Commands.emplace_back(std::get<0>(CommandRef)->getValue(DirectoryStorage),
- std::get<1>(CommandRef)->getValue(FilenameStorage),
- nodeToCommandLine(std::get<2>(CommandRef)),
- Output ? Output->getValue(OutputStorage) : "");
+ Commands.emplace_back(TURef.Directory->getValue(DirectoryStorage),
+ TURef.Filename->getValue(FilenameStorage),
+ nodeToCommandLine(TURef.CommandLine),
+ TURef.Output ? TURef.Output->getValue(OutputStorage)
+ : "");
}
}
@@ -305,6 +400,7 @@ bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
llvm::yaml::ScalarNode *Name = nullptr;
llvm::yaml::SequenceNode *VisibleSets = nullptr;
llvm::yaml::SequenceNode *TUs = nullptr;
+ std::vector<TranslationUnitRef> TURefs = {};
for (auto &NextKeyValue : *SetObject) {
auto *KeyString =
dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
@@ -355,9 +451,11 @@ bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
ErrorMessage = "Expected translation-units item as object.";
return false;
}
- if (!parseTU(ErrorMessage, TUObject)) {
+ TranslationUnitRef TURef = {};
+ if (!parseTU(ErrorMessage, TUObject, TURef)) {
return false;
}
+ TURefs.push_back(std::move(TURef));
}
} else {
ErrorMessage =
@@ -382,11 +480,34 @@ bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
ErrorMessage = "Missing key in set: \"translation-units\".";
return false;
}
+ // Finalize the translation unit refs now that we have all the set info
+ for (auto &TURef : TURefs) {
+ // Attach the parent set name for easy lookups
+ TURef.SetName = Name;
+ // Build up the native file path
+ SmallString<8> FileStorage;
+ StringRef FileName = TURef.Filename->getValue(FileStorage);
+ SmallString<128> NativeFilePath;
+ if (llvm::sys::path::is_relative(FileName)) {
+ SmallString<8> DirectoryStorage;
+ SmallString<128> AbsolutePath(
+ TURef.Directory->getValue(DirectoryStorage));
+ llvm::sys::path::append(AbsolutePath, FileName);
+ llvm::sys::path::native(AbsolutePath, NativeFilePath);
+ } else {
+ llvm::sys::path::native(FileName, NativeFilePath);
+ }
+ llvm::sys::path::remove_dots(NativeFilePath, /*remove_dot_dot=*/true);
+ IndexByFile[NativeFilePath].push_back(TURef);
+ AllCommands.push_back(TURef);
+ MatchTrie.insert(NativeFilePath);
+ }
return true;
}
bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
- llvm::yaml::MappingNode *TUObject) {
+ llvm::yaml::MappingNode *TUObject,
+ TranslationUnitRef &TURef) {
llvm::yaml::SequenceNode *Arguments = nullptr;
std::vector<llvm::yaml::ScalarNode *> Command;
llvm::yaml::ScalarNode *Language = nullptr;
@@ -396,7 +517,10 @@ bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
llvm::yaml::ScalarNode *Source = nullptr;
llvm::yaml::ScalarNode *Object = nullptr;
llvm::yaml::MappingNode *Provides = nullptr;
+ llvm::yaml::ScalarNode *ProvidesModuleName = nullptr;
+ llvm::yaml::ScalarNode *ProvidesModulePCM = nullptr;
llvm::yaml::SequenceNode *Requires = nullptr;
+ std::vector<llvm::yaml::ScalarNode *> RequiredModules;
for (auto &NextKeyValue : *TUObject) {
auto *KeyString =
dyn_cast_if_present<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
@@ -467,12 +591,43 @@ bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
ErrorMessage = "Expected object as value for \"provides\".";
return false;
}
+ for (auto &NextProvidesKeyValue : *Provides) {
+ // The spec allows multiple of module provide, but C++ only allows one
+ if (ProvidesModuleName) {
+ ErrorMessage = "TU can only provide one module.";
+ return false;
+ }
+
+ auto *ProvidesKeyString = dyn_cast_if_present<llvm::yaml::ScalarNode>(
+ NextProvidesKeyValue.getKey());
+ if (!ProvidesKeyString) {
+ ErrorMessage = "Expected strings as key.";
+ return false;
+ }
+ auto *ProvidesValue = dyn_cast_if_present<llvm::yaml::ScalarNode>(
+ NextProvidesKeyValue.getValue());
+ if (!ProvidesValue) {
+ ErrorMessage = "Expected string as provides value.";
+ return false;
+ }
+
+ ProvidesModuleName = ProvidesKeyString;
+ ProvidesModulePCM = ProvidesValue;
+ }
} else if (KeyValue == "requires") {
Requires = dyn_cast<llvm::yaml::SequenceNode>(Value);
if (!Requires) {
ErrorMessage = "Expected array as value for \"requires\".";
return false;
}
+ for (auto &RequiredModule : *Requires) {
+ auto *Scalar = dyn_cast<llvm::yaml::ScalarNode>(&RequiredModule);
+ if (!Scalar) {
+ ErrorMessage = "Only strings are allowed in 'requires'.";
+ return false;
+ }
+ RequiredModules.push_back(Scalar);
+ }
} else {
ErrorMessage = ("Unknown key in translation-unit: \"" +
KeyString->getRawValue() + "\"")
@@ -493,23 +648,12 @@ bool JSONBuildDatabase::parseTU(std::string &ErrorMessage,
ErrorMessage = "Missing key in translation-unit: \"arguments\".";
return false;
}
- SmallString<8> FileStorage;
- StringRef FileName = Source->getValue(FileStorage);
- SmallString<128> NativeFilePath;
- if (llvm::sys::path::is_relative(FileName)) {
- SmallString<8> DirectoryStorage;
- SmallString<128> AbsolutePath(WorkDirectory->getValue(DirectoryStorage));
- llvm::sys::path::append(AbsolutePath, FileName);
- llvm::sys::path::native(AbsolutePath, NativeFilePath);
- } else {
- llvm::sys::path::native(FileName, NativeFilePath);
- }
- llvm::sys::path::remove_dots(NativeFilePath, /*remove_dot_dot=*/true);
- auto Cmd = CompileCommandRef(WorkDirectory, Source, Command, Object);
-
- IndexByFile[NativeFilePath].push_back(Cmd);
- AllCommands.push_back(Cmd);
- MatchTrie.insert(NativeFilePath);
-
+ TURef.Directory = WorkDirectory;
+ TURef.Filename = Source;
+ TURef.CommandLine = std::move(Command);
+ TURef.Output = Object;
+ TURef.ProvidesModuleName = ProvidesModuleName;
+ TURef.ProvidesModulePCM = ProvidesModulePCM;
+ TURef.RequiredModules = std::move(RequiredModules);
return true;
}
diff --git a/clang/lib/Tooling/JSONCompilationDatabase.cpp b/clang/lib/Tooling/JSONCompilationDatabase.cpp
index adaf94a1e0e9c..9155b01f3fe13 100644
--- a/clang/lib/Tooling/JSONCompilationDatabase.cpp
+++ b/clang/lib/Tooling/JSONCompilationDatabase.cpp
@@ -261,16 +261,6 @@ JSONCompilationDatabase::getAllCompileCommands() const {
return Commands;
}
-std::vector<std::string>
-JSONCompilationDatabase::getRequiredModules(StringRef FilePath) const {
- return {};
-}
-
-std::optional<std::string>
-JSONCompilationDatabase::getModuleName(StringRef FilePath) const {
- return std::nullopt;
-}
-
static llvm::StringRef stripExecutableExtension(llvm::StringRef Name) {
Name.consume_back(".exe");
return Name;
diff --git a/clang/lib/Tooling/LocateToolCompilationDatabase.cpp b/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
index 34ced90c32379..033f69f3760c6 100644
--- a/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
+++ b/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
@@ -36,14 +36,6 @@ class LocationAdderDatabase : public CompilationDatabase {
return addLocation(Base->getCompileCommands(FilePath));
}
- std::vector<std::string>
- getRequiredModules(StringRef FilePath) const override {
- return {};
- }
- std::optional<std::string> getModuleName(StringRef FilePath) const override {
- return std::nullopt;
- }
-
private:
std::vector<CompileCommand>
addLocation(std::vector<CompileCommand> Cmds) const {
>From d4656c91e9a1b29242df23b2514fe561b6438ca4 Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Fri, 22 May 2026 15:28:18 -0700
Subject: [PATCH 07/10] Build set index and pass through module manager
---
.../clang/Tooling/CommonOptionsParser.h | 2 ++
.../clang/Tooling/CompilationDatabase.h | 4 ++-
.../clang/Tooling/JSONCompilationDatabase.h | 2 ++
...ExpandResponseFilesCompilationDatabase.cpp | 4 +++
.../GuessTargetAndModeCompilationDatabase.cpp | 2 ++
.../InterpolatingCompilationDatabase.cpp | 4 +++
clang/lib/Tooling/JSONBuildDatabase.cpp | 26 ++++++++++++++++---
.../Tooling/LocateToolCompilationDatabase.cpp | 2 ++
8 files changed, 42 insertions(+), 4 deletions(-)
diff --git a/clang/include/clang/Tooling/CommonOptionsParser.h b/clang/include/clang/Tooling/CommonOptionsParser.h
index 98deaf174fb53..1083fc046ea3f 100644
--- a/clang/include/clang/Tooling/CommonOptionsParser.h
+++ b/clang/include/clang/Tooling/CommonOptionsParser.h
@@ -133,6 +133,8 @@ class ArgumentsAdjustingCompilations : public CompilationDatabase {
std::vector<CompileCommand> getAllCompileCommands() const override;
+ const ModuleManager *getModuleManager() const override { return nullptr; }
+
private:
std::unique_ptr<CompilationDatabase> Compilations;
std::vector<ArgumentsAdjuster> Adjusters;
diff --git a/clang/include/clang/Tooling/CompilationDatabase.h b/clang/include/clang/Tooling/CompilationDatabase.h
index c0d35a07ee94e..72739849d6ba2 100644
--- a/clang/include/clang/Tooling/CompilationDatabase.h
+++ b/clang/include/clang/Tooling/CompilationDatabase.h
@@ -180,7 +180,7 @@ class CompilationDatabase {
///
/// By default, returns nothing. Implementations should override this if they
/// can enumerate their source files.
- virtual const ModuleManager *getModuleManager() const { return nullptr; }
+ virtual const ModuleManager *getModuleManager() const = 0;
};
/// A compilation database that returns a single compile command line.
@@ -243,6 +243,8 @@ class FixedCompilationDatabase : public CompilationDatabase {
std::vector<CompileCommand>
getCompileCommands(StringRef FilePath) const override;
+ const ModuleManager *getModuleManager() const override { return nullptr; }
+
private:
/// This is built up to contain a single entry vector to be returned from
/// getCompileCommands after adding the positional argument.
diff --git a/clang/include/clang/Tooling/JSONCompilationDatabase.h b/clang/include/clang/Tooling/JSONCompilationDatabase.h
index 96582457c63d5..65e384b86f45d 100644
--- a/clang/include/clang/Tooling/JSONCompilationDatabase.h
+++ b/clang/include/clang/Tooling/JSONCompilationDatabase.h
@@ -92,6 +92,8 @@ class JSONCompilationDatabase : public CompilationDatabase {
/// database.
std::vector<CompileCommand> getAllCompileCommands() const override;
+ const ModuleManager *getModuleManager() const override { return nullptr; }
+
private:
/// Constructs a JSON compilation database on a memory buffer.
JSONCompilationDatabase(std::unique_ptr<llvm::MemoryBuffer> Database,
diff --git a/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp b/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
index 22d0057a28f81..cfc6334107e5e 100644
--- a/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
+++ b/clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp
@@ -43,6 +43,10 @@ class ExpandResponseFilesDatabase : public CompilationDatabase {
return expand(Base->getAllCompileCommands());
}
+ const ModuleManager *getModuleManager() const override {
+ return Base->getModuleManager();
+ }
+
private:
std::vector<CompileCommand> expand(std::vector<CompileCommand> Cmds) const {
for (auto &Cmd : Cmds)
diff --git a/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp b/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
index b6c1c0952aca9..2bfd88a7c3222 100644
--- a/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
+++ b/clang/lib/Tooling/GuessTargetAndModeCompilationDatabase.cpp
@@ -34,6 +34,8 @@ class TargetAndModeAdderDatabase : public CompilationDatabase {
return addTargetAndMode(Base->getCompileCommands(FilePath));
}
+ const ModuleManager *getModuleManager() const override { return nullptr; }
+
private:
std::vector<CompileCommand>
addTargetAndMode(std::vector<CompileCommand> Cmds) const {
diff --git a/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp b/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
index 93b1a6eab9e28..a7d59f22f6015 100644
--- a/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
+++ b/clang/lib/Tooling/InterpolatingCompilationDatabase.cpp
@@ -583,6 +583,10 @@ class InterpolatingCompilationDatabase : public CompilationDatabase {
return Inner->getAllCompileCommands();
}
+ const ModuleManager *getModuleManager() const override {
+ return Inner->getModuleManager();
+ }
+
private:
std::unique_ptr<CompilationDatabase> Inner;
FileIndex Index;
diff --git a/clang/lib/Tooling/JSONBuildDatabase.cpp b/clang/lib/Tooling/JSONBuildDatabase.cpp
index 4dd54ad891556..f3f4f613d0c9c 100644
--- a/clang/lib/Tooling/JSONBuildDatabase.cpp
+++ b/clang/lib/Tooling/JSONBuildDatabase.cpp
@@ -139,7 +139,6 @@ const ModuleManager *JSONBuildDatabase::getModuleManager() const {
std::vector<std::string>
JSONBuildDatabase::getRequiredModules(StringRef FilePath) const {
-
const auto *TURef = getTUForSource(FilePath);
std::vector<std::string> RequiredModules;
if (TURef) {
@@ -174,7 +173,6 @@ JSONBuildDatabase::getModuleNameState(StringRef ModuleName) const {
std::string
JSONBuildDatabase::getSourceForModuleName(StringRef ModuleName,
StringRef RequiredSourceFile) const {
-
const auto *RequiredSourceTURef = getTUForSource(RequiredSourceFile);
if (RequiredSourceTURef) {
SmallString<8> SetNameStorage;
@@ -385,7 +383,8 @@ bool JSONBuildDatabase::parseRoot(std::string &ErrorMessage,
return false;
}
// Check compatible version
- if (Version->getRawValue() != "1") {
+ SmallString<10> VersionStorage;
+ if (Version->getValue(VersionStorage).str() != "1") {
ErrorMessage =
("Unsupported version: \"" + Version->getRawValue() + "\"").str();
return false;
@@ -399,6 +398,7 @@ bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
llvm::yaml::ScalarNode *FamilyName = nullptr;
llvm::yaml::ScalarNode *Name = nullptr;
llvm::yaml::SequenceNode *VisibleSets = nullptr;
+ std::vector<llvm::yaml::ScalarNode *> VisibleSetsRefs = {};
llvm::yaml::SequenceNode *TUs = nullptr;
std::vector<TranslationUnitRef> TURefs = {};
for (auto &NextKeyValue : *SetObject) {
@@ -439,6 +439,14 @@ bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
ErrorMessage = "Expected array as value for \"visible-sets\".";
return false;
}
+ for (auto &VisibleSet : *VisibleSets) {
+ auto *Scalar = dyn_cast<llvm::yaml::ScalarNode>(&VisibleSet);
+ if (!Scalar) {
+ ErrorMessage = "Only strings are allowed in 'visible-sets'.";
+ return false;
+ }
+ VisibleSetsRefs.push_back(Scalar);
+ }
} else if (KeyValue == "translation-units") {
TUs = dyn_cast<llvm::yaml::SequenceNode>(Value);
if (!TUs) {
@@ -501,7 +509,19 @@ bool JSONBuildDatabase::parseSet(std::string &ErrorMessage,
IndexByFile[NativeFilePath].push_back(TURef);
AllCommands.push_back(TURef);
MatchTrie.insert(NativeFilePath);
+ if (TURef.ProvidesModuleName) {
+ SmallString<8> ModuleNameStorage;
+ ModuleNameToDistinctSources
+ [TURef.ProvidesModuleName->getValue(ModuleNameStorage).str()]
+ .insert(NativeFilePath);
+ }
}
+ // Generate lookup for each set
+ TranslationUnitSet TUSet = {};
+ TUSet.TranslationUnits = std::move(TURefs);
+ TUSet.VisibleSets = std::move(VisibleSetsRefs);
+ SmallString<8> NameStorage;
+ IndexBySet[Name->getValue(NameStorage)] = std::move(TUSet);
return true;
}
diff --git a/clang/lib/Tooling/LocateToolCompilationDatabase.cpp b/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
index 033f69f3760c6..e14bf69d3c863 100644
--- a/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
+++ b/clang/lib/Tooling/LocateToolCompilationDatabase.cpp
@@ -36,6 +36,8 @@ class LocationAdderDatabase : public CompilationDatabase {
return addLocation(Base->getCompileCommands(FilePath));
}
+ const ModuleManager *getModuleManager() const override { return nullptr; }
+
private:
std::vector<CompileCommand>
addLocation(std::vector<CompileCommand> Cmds) const {
>From e5409da6bb72282e1c3e85ac4808aa54c84f934a Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Fri, 22 May 2026 21:29:40 -0700
Subject: [PATCH 08/10] Fix seg fault
---
clang/lib/Tooling/JSONBuildDatabase.cpp | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/clang/lib/Tooling/JSONBuildDatabase.cpp b/clang/lib/Tooling/JSONBuildDatabase.cpp
index f3f4f613d0c9c..63b15f2849069 100644
--- a/clang/lib/Tooling/JSONBuildDatabase.cpp
+++ b/clang/lib/Tooling/JSONBuildDatabase.cpp
@@ -228,7 +228,8 @@ JSONBuildDatabase::getTUForModule(StringRef ModuleName,
for (const auto &TURef : TUSetI->getValue().TranslationUnits) {
SmallString<8> ModuleNameStorage;
- if (TURef.ProvidesModuleName->getValue(ModuleNameStorage) == ModuleName)
+ if (TURef.ProvidesModuleName &&
+ TURef.ProvidesModuleName->getValue(ModuleNameStorage) == ModuleName)
return &TURef;
}
return nullptr;
>From 77a7c156799e5ebf0416b6d3dc28dcc9ce16ae0f Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Sat, 23 May 2026 12:41:41 -0700
Subject: [PATCH 09/10] Stub out tests for now
---
.../clangd/unittests/PrerequisiteModulesTest.cpp | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp b/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp
index b88213decea44..ad878c7d7f96e 100644
--- a/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp
+++ b/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp
@@ -142,6 +142,10 @@ class PerFileModulesCompilationDatabase : public GlobalCompilationDatabase {
std::vector<std::string> getAllFiles() const override { return CDB.Files; }
+ tooling::ModuleManager *getModuleManager() const override {
+ return nullptr;
+ }
+
private:
const PerFileModulesCompilationDatabase &CDB;
};
@@ -206,6 +210,10 @@ class MockDirectoryCompilationDatabase : public MockCompilationDatabase {
void AddFile(StringRef File) { Files.push_back(File.str()); }
+ const tooling::ModuleManager *getModuleManager() const override {
+ return nullptr;
+ }
+
private:
MockDirectoryCompilationDatabase &MCDB;
std::vector<std::string> Files;
@@ -574,7 +582,8 @@ int use() { return a; }
ModulesBuilder Builder(CDB);
- auto UseInfo = Builder.buildPrerequisiteModulesFor(getFullPath("Use.cpp"), FS);
+ auto UseInfo =
+ Builder.buildPrerequisiteModulesFor(getFullPath("Use.cpp"), FS);
ASSERT_TRUE(UseInfo);
HeaderSearchOptions HSOpts;
>From 3220694ec7e981b1c48338d9984f867240eac5eb Mon Sep 17 00:00:00 2001
From: mwasplund <mwasplund at outlook.com>
Date: Sat, 23 May 2026 15:11:36 -0700
Subject: [PATCH 10/10] Missing impl
---
clang/tools/clang-scan-deps/ClangScanDeps.cpp | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/clang/tools/clang-scan-deps/ClangScanDeps.cpp b/clang/tools/clang-scan-deps/ClangScanDeps.cpp
index 5322178a848e2..5bca122599f5b 100644
--- a/clang/tools/clang-scan-deps/ClangScanDeps.cpp
+++ b/clang/tools/clang-scan-deps/ClangScanDeps.cpp
@@ -850,6 +850,10 @@ getCompilationDatabase(int argc, char **argv, std::string &ErrorMessage) {
return {Command};
}
+ const tooling::ModuleManager *getModuleManager() const override {
+ return nullptr;
+ }
+
private:
tooling::CompileCommand Command;
};
More information about the cfe-commits
mailing list