[clang] [clang-tools-extra] Fix issue 503 (PR #192882)
João Távora via cfe-commits
cfe-commits at lists.llvm.org
Sun Apr 19 17:29:22 PDT 2026
https://github.com/joaotavora created https://github.com/llvm/llvm-project/pull/192882
See github.com/clangd/clangd/issues/503 for discussion leading up to this.
Note that the second (topmost) commit is optional. The LSP client I used doesn't need it, but it could presumably be useful to other clients. I don't know where the popular vscode client is maintained or who maintains it, but I think it is affected by github.com/clangd/clangd/issues/503
Anyway, I tested manually this with a small project that lives canonically in /home/$USER/Source/Cpp/proj and is symlinked in `~/proj`. After
```
cd ~/proj
cmake -B build -DCMAKE_EXPORT_COMPILE_COMMANDS=on
ln -sf build/compile_commands.json
```
I start the Emacs vanilla simply with `emacs -Q ~/proj/path/to/some/file.cpp -f eglot`.
With the old `clangd`, indexing doesnt' even start (which I found strange, but maybe that's because that clangd is still version 22). So asking for references fails and considers only `didOpen`'ed files. With this changed version, indexing starts and finishes cleanly, and references considers the index.
Anyway, you'll probably want automated tests for this, but I'm afraid I don't have a clue on how to add them, as these would probably have to be integration tests.
Also sorry if the commit messages are not idiomatic here, did my best and decided to err on the side of verbosity....
>From 4ebfd26d5d7aae7b18229608ad205f8f98b87594 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jo=C3=A3o=20T=C3=A1vora?= <joaotavora at gmail.com>
Date: Mon, 20 Apr 2026 00:37:46 +0100
Subject: [PATCH 1/2] [clang][Tooling] Optionally resolve real paths in
JSONCompilationDatabase
Add ResolveRealPaths flag (default false) to the loadFromFile and
loadFromBuffer factory functions. When set, parse() resolves symlinks
and expands tildes via real_path(), falling back to native path
normalization on failure. The parse() change is ported from Sam
Brightman's https://github.com/sambrightman/llvm-project/commit/bcfd679aeebeb680dc853f2fda14413d45917ac0.
Fixes https://github.com/clangd/clangd/issues/503.
* clang/include/clang/Tooling/JSONCompilationDatabase.h
(loadFromFile, loadFromBuffer): Add ResolveRealPaths parameter.
(JSONCompilationDatabase): Add ResolveRealPaths constructor parameter
and member.
* clang/lib/Tooling/JSONCompilationDatabase.cpp (loadFromFile)
(loadFromBuffer): Accept and pass ResolveRealPaths to constructor.
(parse): Use ResolveRealPaths member to guard real_path() calls.
* clang-tools-extra/clangd/GlobalCompilationDatabase.cpp (parseJSON):
Pass ResolveRealPaths=true to loadFromBuffer.
---
.../clangd/GlobalCompilationDatabase.cpp | 3 +-
.../clang/Tooling/JSONCompilationDatabase.h | 8 ++--
clang/lib/Tooling/JSONCompilationDatabase.cpp | 37 +++++++++++--------
3 files changed, 29 insertions(+), 19 deletions(-)
diff --git a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
index a1d9135111ca8..50b1eb1b8198f 100644
--- a/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
+++ b/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
@@ -250,7 +250,8 @@ DirectoryBasedGlobalCompilationDatabase::DirectoryCache::CachedFile::load(
static std::unique_ptr<tooling::CompilationDatabase>
parseJSON(PathRef Path, llvm::StringRef Data, std::string &Error) {
if (auto CDB = tooling::JSONCompilationDatabase::loadFromBuffer(
- Data, Error, tooling::JSONCommandLineSyntax::AutoDetect)) {
+ Data, Error, tooling::JSONCommandLineSyntax::AutoDetect,
+ /*ResolveRealPaths=*/true)) {
// FS used for expanding response files.
// FIXME: ExpandResponseFilesDatabase appears not to provide the usual
// thread-safety guarantees, as the access to FS is not locked!
diff --git a/clang/include/clang/Tooling/JSONCompilationDatabase.h b/clang/include/clang/Tooling/JSONCompilationDatabase.h
index 96582457c63d5..f97007c8da7f9 100644
--- a/clang/include/clang/Tooling/JSONCompilationDatabase.h
+++ b/clang/include/clang/Tooling/JSONCompilationDatabase.h
@@ -66,14 +66,14 @@ class JSONCompilationDatabase : public CompilationDatabase {
/// loaded from the given file.
static std::unique_ptr<JSONCompilationDatabase>
loadFromFile(StringRef FilePath, std::string &ErrorMessage,
- JSONCommandLineSyntax Syntax);
+ JSONCommandLineSyntax Syntax, bool ResolveRealPaths = false);
/// Loads a JSON compilation database from a data buffer.
///
/// Returns NULL and sets ErrorMessage if the database could not be loaded.
static std::unique_ptr<JSONCompilationDatabase>
loadFromBuffer(StringRef DatabaseString, std::string &ErrorMessage,
- JSONCommandLineSyntax Syntax);
+ JSONCommandLineSyntax Syntax, bool ResolveRealPaths = false);
/// Returns all compile commands in which the specified file was
/// compiled.
@@ -95,8 +95,9 @@ class JSONCompilationDatabase : public CompilationDatabase {
private:
/// Constructs a JSON compilation database on a memory buffer.
JSONCompilationDatabase(std::unique_ptr<llvm::MemoryBuffer> Database,
- JSONCommandLineSyntax Syntax)
+ JSONCommandLineSyntax Syntax, bool ResolveRealPaths)
: Database(std::move(Database)), Syntax(Syntax),
+ ResolveRealPaths(ResolveRealPaths),
YAMLStream(this->Database->getBuffer(), SM) {}
/// Parses the database file and creates the index.
@@ -132,6 +133,7 @@ class JSONCompilationDatabase : public CompilationDatabase {
std::unique_ptr<llvm::MemoryBuffer> Database;
JSONCommandLineSyntax Syntax;
+ bool ResolveRealPaths;
llvm::SourceMgr SM;
llvm::yaml::Stream YAMLStream;
};
diff --git a/clang/lib/Tooling/JSONCompilationDatabase.cpp b/clang/lib/Tooling/JSONCompilationDatabase.cpp
index 0efa75970d986..7e6eb02c8b109 100644
--- a/clang/lib/Tooling/JSONCompilationDatabase.cpp
+++ b/clang/lib/Tooling/JSONCompilationDatabase.cpp
@@ -184,10 +184,9 @@ volatile int JSONAnchorSource = 0;
} // namespace tooling
} // namespace clang
-std::unique_ptr<JSONCompilationDatabase>
-JSONCompilationDatabase::loadFromFile(StringRef FilePath,
- std::string &ErrorMessage,
- JSONCommandLineSyntax Syntax) {
+std::unique_ptr<JSONCompilationDatabase> JSONCompilationDatabase::loadFromFile(
+ StringRef FilePath, std::string &ErrorMessage, JSONCommandLineSyntax Syntax,
+ bool ResolveRealPaths) {
// Don't mmap: if we're a long-lived process, the build system may overwrite.
llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> DatabaseBuffer =
llvm::MemoryBuffer::getFile(FilePath, /*IsText=*/false,
@@ -197,8 +196,8 @@ JSONCompilationDatabase::loadFromFile(StringRef FilePath,
ErrorMessage = "Error while opening JSON database: " + Result.message();
return nullptr;
}
- std::unique_ptr<JSONCompilationDatabase> Database(
- new JSONCompilationDatabase(std::move(*DatabaseBuffer), Syntax));
+ std::unique_ptr<JSONCompilationDatabase> Database(new JSONCompilationDatabase(
+ std::move(*DatabaseBuffer), Syntax, ResolveRealPaths));
if (!Database->parse(ErrorMessage))
return nullptr;
return Database;
@@ -207,11 +206,12 @@ JSONCompilationDatabase::loadFromFile(StringRef FilePath,
std::unique_ptr<JSONCompilationDatabase>
JSONCompilationDatabase::loadFromBuffer(StringRef DatabaseString,
std::string &ErrorMessage,
- JSONCommandLineSyntax Syntax) {
+ JSONCommandLineSyntax Syntax,
+ bool ResolveRealPaths) {
std::unique_ptr<llvm::MemoryBuffer> DatabaseBuffer(
llvm::MemoryBuffer::getMemBufferCopy(DatabaseString));
- std::unique_ptr<JSONCompilationDatabase> Database(
- new JSONCompilationDatabase(std::move(DatabaseBuffer), Syntax));
+ std::unique_ptr<JSONCompilationDatabase> Database(new JSONCompilationDatabase(
+ std::move(DatabaseBuffer), Syntax, ResolveRealPaths));
if (!Database->parse(ErrorMessage))
return nullptr;
return Database;
@@ -411,20 +411,27 @@ bool JSONCompilationDatabase::parse(std::string &ErrorMessage) {
}
SmallString<8> FileStorage;
StringRef FileName = File->getValue(FileStorage);
- SmallString<128> NativeFilePath;
+ SmallString<128> RealFilePath;
if (llvm::sys::path::is_relative(FileName)) {
SmallString<8> DirectoryStorage;
SmallString<128> AbsolutePath(Directory->getValue(DirectoryStorage));
llvm::sys::path::append(AbsolutePath, FileName);
- llvm::sys::path::native(AbsolutePath, NativeFilePath);
+ if (llvm::sys::fs::real_path(AbsolutePath, RealFilePath,
+ /*expand_tilde=*/true)) {
+ llvm::sys::path::native(AbsolutePath, RealFilePath);
+ llvm::sys::path::remove_dots(RealFilePath, /*remove_dot_dot=*/true);
+ }
} else {
- llvm::sys::path::native(FileName, NativeFilePath);
+ if (llvm::sys::fs::real_path(FileName, RealFilePath,
+ /*expand_tilde=*/true)) {
+ llvm::sys::path::native(FileName, RealFilePath);
+ llvm::sys::path::remove_dots(RealFilePath, /*remove_dot_dot=*/true);
+ }
}
- llvm::sys::path::remove_dots(NativeFilePath, /*remove_dot_dot=*/true);
auto Cmd = CompileCommandRef(Directory, File, *Command, Output);
- IndexByFile[NativeFilePath].push_back(Cmd);
+ IndexByFile[RealFilePath].push_back(Cmd);
AllCommands.push_back(Cmd);
- MatchTrie.insert(NativeFilePath);
+ MatchTrie.insert(RealFilePath);
}
return true;
}
>From e2cc29501ddc92168eac124886c41ab34868a14c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jo=C3=A3o=20T=C3=A1vora?= <joaotavora at gmail.com>
Date: Mon, 20 Apr 2026 00:42:02 +0100
Subject: [PATCH 2/2] [clangd] Resolve symlinks server-side for compile command
lookup
Resolve the real path once at file-registration time and reuse it for
all subsequent CDB lookups. This is an optional complement to the
previous commit: a well-behaved LSP client could instead always send
fully-resolved paths in its notifications, making this server-side
resolution unnecessary. The commit is kept separate so it can be
dropped if the upstream consensus favours the client-side approach.
Fixes https://github.com/clangd/clangd/issues/503.
* clang-tools-extra/clangd/Compiler.h: Include SmallString.h.
(ParseInputs): Add RealFilePath field.
* clang-tools-extra/clangd/TUScheduler.cpp: Include SmallString.h.
(ASTWorker::update): Pass RealFilePath to getCompileCommand.
(FileData): Add RealFilePath field.
(TUScheduler::update): Resolve and store real path on first update;
propagate to ParseInputs.
* clang-tools-extra/clangd/index/IndexAction.cpp (toURI): Resolve
symlinks via real_path before creating URI.
---
clang-tools-extra/clangd/Compiler.h | 3 +++
clang-tools-extra/clangd/TUScheduler.cpp | 12 ++++++++++--
clang-tools-extra/clangd/index/IndexAction.cpp | 4 ++++
3 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/clang-tools-extra/clangd/Compiler.h b/clang-tools-extra/clangd/Compiler.h
index 5e5e23d5b9682..58c5476d67447 100644
--- a/clang-tools-extra/clangd/Compiler.h
+++ b/clang-tools-extra/clangd/Compiler.h
@@ -23,6 +23,7 @@
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/PrecompiledPreamble.h"
#include "clang/Tooling/CompilationDatabase.h"
+#include "llvm/ADT/SmallString.h"
#include <memory>
#include <vector>
@@ -65,6 +66,8 @@ struct ParseInputs {
FeatureModuleSet *FeatureModules = nullptr;
// Used to build and manage (C++) modules.
ModulesBuilder *ModulesManager = nullptr;
+ // Real path for extracting the CompileCommand
+ llvm::SmallString<128> RealFilePath;
};
/// Clears \p CI from options that are not supported by clangd, like codegen or
diff --git a/clang-tools-extra/clangd/TUScheduler.cpp b/clang-tools-extra/clangd/TUScheduler.cpp
index 0661ecb58008e..76adfa3624475 100644
--- a/clang-tools-extra/clangd/TUScheduler.cpp
+++ b/clang-tools-extra/clangd/TUScheduler.cpp
@@ -69,6 +69,7 @@
#include "llvm/ADT/FunctionExtras.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/ScopeExit.h"
+#include "llvm/ADT/SmallString.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringExtras.h"
#include "llvm/ADT/StringRef.h"
@@ -869,7 +870,7 @@ void ASTWorker::update(ParseInputs Inputs, WantDiagnostics WantDiags,
// environment to build the file, it would be nice if we could emit a
// "PreparingBuild" status to inform users, it is non-trivial given the
// current implementation.
- auto Cmd = CDB.getCompileCommand(FileName);
+ auto Cmd = CDB.getCompileCommand(Inputs.RealFilePath);
// If we don't have a reliable command for this file, it may be a header.
// Try to find a file that includes it, to borrow its command.
if (!Cmd || !isReliable(*Cmd)) {
@@ -1625,6 +1626,7 @@ struct TUScheduler::FileData {
/// Latest inputs, passed to TUScheduler::update().
std::string Contents;
ASTWorkerHandle Worker;
+ SmallString<128> RealFilePath;
};
TUScheduler::TUScheduler(const GlobalCompilationDatabase &CDB,
@@ -1680,13 +1682,19 @@ bool TUScheduler::update(PathRef File, ParseInputs Inputs,
ASTWorkerHandle Worker = ASTWorker::create(
File, CDB, *IdleASTs, *HeaderIncluders,
WorkerThreads ? &*WorkerThreads : nullptr, Barrier, Opts, *Callbacks);
+ // Attempt to get and store the "real path". Don't worry about
+ // errors/tilde expansion, that naturally happens later when
+ // needed.
+ SmallString<128> RealFilePath{File};
+ llvm::sys::fs::real_path(File, RealFilePath);
FD = std::unique_ptr<FileData>(
- new FileData{Inputs.Contents, std::move(Worker)});
+ new FileData{Inputs.Contents, std::move(Worker), RealFilePath});
ContentChanged = true;
} else if (FD->Contents != Inputs.Contents) {
ContentChanged = true;
FD->Contents = Inputs.Contents;
}
+ Inputs.RealFilePath = FD->RealFilePath;
FD->Worker->update(std::move(Inputs), WantDiags, ContentChanged);
// There might be synthetic update requests, don't change the LastActiveFile
// in such cases.
diff --git a/clang-tools-extra/clangd/index/IndexAction.cpp b/clang-tools-extra/clangd/index/IndexAction.cpp
index 489c61f1ff424..bae320f9d3da1 100644
--- a/clang-tools-extra/clangd/index/IndexAction.cpp
+++ b/clang-tools-extra/clangd/index/IndexAction.cpp
@@ -34,9 +34,13 @@ namespace {
std::optional<std::string> toURI(OptionalFileEntryRef File) {
if (!File)
return std::nullopt;
+ // Does *not* resolve symlinks
auto AbsolutePath = File->getFileEntry().tryGetRealPathName();
if (AbsolutePath.empty())
return std::nullopt;
+ llvm::SmallString<128> RealPathName;
+ if (!llvm::sys::fs::real_path(AbsolutePath, RealPathName))
+ return URI::create(RealPathName).toString();
return URI::create(AbsolutePath).toString();
}
More information about the cfe-commits
mailing list