[clang-tools-extra] cc92959 - [libclang] Add API to override preamble storage path
Aaron Ballman via cfe-commits
cfe-commits at lists.llvm.org
Tue Mar 7 05:25:46 PST 2023
Author: Igor Kushnir
Date: 2023-03-07T08:25:38-05:00
New Revision: cc929590ad305f0d068709c7c7999f5fc6118dc9
URL: https://github.com/llvm/llvm-project/commit/cc929590ad305f0d068709c7c7999f5fc6118dc9
DIFF: https://github.com/llvm/llvm-project/commit/cc929590ad305f0d068709c7c7999f5fc6118dc9.diff
LOG: [libclang] Add API to override preamble storage path
TempPCHFile::create() calls llvm::sys::fs::createTemporaryFile() to
create a file named preamble-*.pch in a system temporary directory. This
commit allows overriding the directory where these often many and large
preamble-*.pch files are stored.
The referenced bug report requests the ability to override the temporary
directory path used by libclang. However, overriding the return value of
llvm::sys::path::system_temp_directory() was rejected during code review
as improper and because it would negatively affect multithreading
performance. Finding all places where libclang uses the temporary
directory is very difficult. Therefore this commit is limited to
override libclang's single known use of the temporary directory.
This commit allows to override the preamble storage path only during
CXIndex construction to avoid multithreading issues and ensure that all
preambles are stored in the same directory. For the same multithreading
and consistency reasons, this commit deprecates
clang_CXIndex_setGlobalOptions() and
clang_CXIndex_setInvocationEmissionPathOption() in favor of specifying
these options during CXIndex construction.
Adding a new CXIndex constructor function each time a new initialization
argument is needed leads to either a large number of function parameters
unneeded by most libclang users or to an exponential number of overloads
that support different usage requirements. Therefore this commit
introduces a new extensible struct CXIndexOptions and a general function
clang_createIndexWithOptions().
A libclang user passes a desired preamble storage path to
clang_createIndexWithOptions(), which stores it in
CIndexer::PreambleStoragePath. Whenever
clang_parseTranslationUnit_Impl() is called, it passes
CIndexer::PreambleStoragePath to ASTUnit::LoadFromCommandLine(), which
stores this argument in ASTUnit::PreambleStoragePath. Whenever
ASTUnit::getMainBufferWithPrecompiledPreamble() is called, it passes
ASTUnit::PreambleStoragePath to PrecompiledPreamble::Build().
PrecompiledPreamble::Build() forwards the corresponding StoragePath
argument to TempPCHFile::create(). If StoragePath is not empty,
TempPCHFile::create() stores the preamble-*.pch file in the directory at
the specified path rather than in the system temporary directory.
The analysis below proves that this passing around of the
PreambleStoragePath string is sufficient to guarantee that the libclang
user override is used in TempPCHFile::create(). The analysis ignores API
uses in test code.
TempPCHFile::create() is called only in PrecompiledPreamble::Build().
PrecompiledPreamble::Build() is called only in two places: one in
clangd, which is not used by libclang, and one in
ASTUnit::getMainBufferWithPrecompiledPreamble().
ASTUnit::getMainBufferWithPrecompiledPreamble() is called in 3 places:
ASTUnit::LoadFromCompilerInvocation() [analyzed below].
ASTUnit::Reparse(), which in turn is called only from
clang_reparseTranslationUnit_Impl(), which in turn is called only from
clang_reparseTranslationUnit(). clang_reparseTranslationUnit() is never
called in LLVM code, but is part of public libclang API. This function's
documentation requires its translation unit argument to have been built
with clang_createTranslationUnitFromSourceFile().
clang_createTranslationUnitFromSourceFile() delegates its work to
clang_parseTranslationUnit(), which delegates to
clang_parseTranslationUnit2(), which delegates to
clang_parseTranslationUnit2FullArgv(), which delegates to
clang_parseTranslationUnit_Impl(), which passes
CIndexer::PreambleStoragePath to the ASTUnit it creates.
ASTUnit::CodeComplete() passes AllowRebuild = false to
ASTUnit::getMainBufferWithPrecompiledPreamble(), which makes it return
nullptr before calling PrecompiledPreamble::Build().
Both ASTUnit::LoadFromCompilerInvocation() overloads (one of which
delegates its work to another) call
ASTUnit::getMainBufferWithPrecompiledPreamble() only if their argument
PrecompilePreambleAfterNParses > 0. LoadFromCompilerInvocation() is
called in:
ASTBuilderAction::runInvocation() keeps the default parameter value
of PrecompilePreambleAfterNParses = 0, meaning that the preamble file is
never created from here.
ASTUnit::LoadFromCommandLine().
ASTUnit::LoadFromCommandLine() is called in two places:
CrossTranslationUnitContext::ASTLoader::loadFromSource() keeps the
default parameter value of PrecompilePreambleAfterNParses = 0, meaning
that the preamble file is never created from here.
clang_parseTranslationUnit_Impl(), which passes
CIndexer::PreambleStoragePath to the ASTUnit it creates.
Therefore, the overridden preamble storage path is always used in
TempPCHFile::create().
TempPCHFile::create() uses PreambleStoragePath in the same way as
LibclangInvocationReporter() uses InvocationEmissionPath. The existing
documentation for clang_CXIndex_setInvocationEmissionPathOption() does
not specify ownership, encoding, separator or relative vs absolute path
requirements. So the documentation for
CXIndexOptions::PreambleStoragePath doesn't either. The assumptions are:
no ownership transfer;
UTF-8 encoding;
native separators.
Both relative and absolute paths are supported.
The added API works as expected in KDevelop:
https://invent.kde.org/kdevelop/kdevelop/-/merge_requests/283
Fixes: https://github.com/llvm/llvm-project/issues/51847
Differential Revision: https://reviews.llvm.org/D143418
Added:
Modified:
clang-tools-extra/clangd/Preamble.cpp
clang/docs/ReleaseNotes.rst
clang/include/clang-c/Index.h
clang/include/clang/Frontend/ASTUnit.h
clang/include/clang/Frontend/PrecompiledPreamble.h
clang/lib/Frontend/ASTUnit.cpp
clang/lib/Frontend/PrecompiledPreamble.cpp
clang/tools/c-index-test/c-index-test.c
clang/tools/libclang/CIndex.cpp
clang/tools/libclang/CIndexer.h
clang/tools/libclang/libclang.map
clang/unittests/Frontend/ASTUnitTest.cpp
clang/unittests/libclang/LibclangTest.cpp
clang/unittests/libclang/TestUtils.h
Removed:
################################################################################
diff --git a/clang-tools-extra/clangd/Preamble.cpp b/clang-tools-extra/clangd/Preamble.cpp
index f50597f9ad209..fceea309710bb 100644
--- a/clang-tools-extra/clangd/Preamble.cpp
+++ b/clang-tools-extra/clangd/Preamble.cpp
@@ -641,7 +641,7 @@ buildPreamble(PathRef FileName, CompilerInvocation CI,
auto BuiltPreamble = PrecompiledPreamble::Build(
CI, ContentsBuffer.get(), Bounds, *PreambleDiagsEngine,
Stats ? TimedFS : StatCacheFS, std::make_shared<PCHContainerOperations>(),
- StoreInMemory, CapturedInfo);
+ StoreInMemory, /*StoragePath=*/StringRef(), CapturedInfo);
PreambleTimer.stopTimer();
// When building the AST for the main file, we do want the function
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index c096133029682..4bc2e54a85998 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -291,6 +291,14 @@ libclang
which identifies whether a constructor or conversion function cursor
was marked with the explicit identifier.
+- Introduced the new ``CXIndex`` constructor function
+ ``clang_createIndexWithOptions``, which allows overriding precompiled preamble
+ storage path.
+
+- Deprecated two functions ``clang_CXIndex_setGlobalOptions`` and
+ ``clang_CXIndex_setInvocationEmissionPathOption`` in favor of the new
+ function ``clang_createIndexWithOptions`` in order to improve thread safety.
+
Static Analyzer
---------------
- Fix incorrect alignment attribute on the this parameter of certain
diff --git a/clang/include/clang-c/Index.h b/clang/include/clang-c/Index.h
index a17a97721de26..152f65c9028e4 100644
--- a/clang/include/clang-c/Index.h
+++ b/clang/include/clang-c/Index.h
@@ -34,7 +34,7 @@
* compatible, thus CINDEX_VERSION_MAJOR is expected to remain stable.
*/
#define CINDEX_VERSION_MAJOR 0
-#define CINDEX_VERSION_MINOR 63
+#define CINDEX_VERSION_MINOR 64
#define CINDEX_VERSION_ENCODE(major, minor) (((major)*10000) + ((minor)*1))
@@ -275,6 +275,22 @@ CINDEX_LINKAGE CXIndex clang_createIndex(int excludeDeclarationsFromPCH,
*/
CINDEX_LINKAGE void clang_disposeIndex(CXIndex index);
+typedef enum {
+ /**
+ * Use the default value of an option that may depend on the process
+ * environment.
+ */
+ CXChoice_Default = 0,
+ /**
+ * Enable the option.
+ */
+ CXChoice_Enabled = 1,
+ /**
+ * Disable the option.
+ */
+ CXChoice_Disabled = 2
+} CXChoice;
+
typedef enum {
/**
* Used to indicate that no special CXIndex options are needed.
@@ -309,9 +325,116 @@ typedef enum {
} CXGlobalOptFlags;
+/**
+ * Index initialization options.
+ *
+ * 0 is the default value of each member of this struct except for Size.
+ * Initialize the struct in one of the following two ways to avoid adapting code
+ * each time a new member is added to it:
+ * \code
+ * CXIndexOptions Opts;
+ * memset(&Opts, 0, sizeof(Opts));
+ * Opts.Size = sizeof(CXIndexOptions);
+ * \endcode
+ * or explicitly initialize the first data member and zero-initialize the rest:
+ * \code
+ * CXIndexOptions Opts = { sizeof(CXIndexOptions) };
+ * \endcode
+ */
+typedef struct CXIndexOptions {
+ /**
+ * The size of struct CXIndexOptions used for option versioning.
+ *
+ * Always initialize this member to sizeof(CXIndexOptions), or assign
+ * sizeof(CXIndexOptions) to it right after creating a CXIndexOptions object.
+ */
+ unsigned Size;
+ /**
+ * A CXChoice enumerator that specifies the indexing priority policy.
+ * \sa CXGlobalOpt_ThreadBackgroundPriorityForIndexing
+ */
+ unsigned char ThreadBackgroundPriorityForIndexing;
+ /**
+ * A CXChoice enumerator that specifies the editing priority policy.
+ * \sa CXGlobalOpt_ThreadBackgroundPriorityForEditing
+ */
+ unsigned char ThreadBackgroundPriorityForEditing;
+ /**
+ * \see clang_createIndex()
+ */
+ int ExcludeDeclarationsFromPCH : 1;
+ /**
+ * \see clang_createIndex()
+ */
+ int DisplayDiagnostics : 1;
+ /**
+ * The path to a directory, in which to store temporary PCH files. If null or
+ * empty, the default system temporary directory is used. These PCH files are
+ * deleted on clean exit but stay on disk if the program crashes or is killed.
+ *
+ * Libclang does not create the directory at the specified path in the file
+ * system. Therefore it must exist, or storing PCH files will fail.
+ */
+ const char *PreambleStoragePath;
+ /**
+ * Specifies a path which will contain log files for certain libclang
+ * invocations. A null value implies that libclang invocations are not logged.
+ */
+ const char *InvocationEmissionPath;
+} CXIndexOptions;
+
+/**
+ * Provides a shared context for creating translation units.
+ *
+ * Call this function instead of clang_createIndex() if you need to configure
+ * the additional options in CXIndexOptions.
+ *
+ * \returns The created index or null in case of error, such as an unsupported
+ * value of options->Size.
+ *
+ * For example:
+ * \code
+ * CXIndex createIndex(const char *ApplicationTemporaryPath) {
+ * const int ExcludeDeclarationsFromPCH = 1;
+ * const int DisplayDiagnostics = 1;
+ * CXIndex Idx;
+ * #if CINDEX_VERSION_MINOR >= 64
+ * CXIndexOptions Opts;
+ * memset(&Opts, 0, sizeof(Opts));
+ * Opts.Size = sizeof(CXIndexOptions);
+ * Opts.ThreadBackgroundPriorityForIndexing = 1;
+ * Opts.ExcludeDeclarationsFromPCH = ExcludeDeclarationsFromPCH;
+ * Opts.DisplayDiagnostics = DisplayDiagnostics;
+ * Opts.PreambleStoragePath = ApplicationTemporaryPath;
+ * Idx = clang_createIndexWithOptions(&Opts);
+ * if (Idx)
+ * return Idx;
+ * fprintf(stderr,
+ * "clang_createIndexWithOptions() failed. "
+ * "CINDEX_VERSION_MINOR = %d, sizeof(CXIndexOptions) = %u\n",
+ * CINDEX_VERSION_MINOR, Opts.Size);
+ * #else
+ * (void)ApplicationTemporaryPath;
+ * #endif
+ * Idx = clang_createIndex(ExcludeDeclarationsFromPCH, DisplayDiagnostics);
+ * clang_CXIndex_setGlobalOptions(
+ * Idx, clang_CXIndex_getGlobalOptions(Idx) |
+ * CXGlobalOpt_ThreadBackgroundPriorityForIndexing);
+ * return Idx;
+ * }
+ * \endcode
+ *
+ * \sa clang_createIndex()
+ */
+CINDEX_LINKAGE CXIndex
+clang_createIndexWithOptions(const CXIndexOptions *options);
+
/**
* Sets general options associated with a CXIndex.
*
+ * This function is DEPRECATED. Set CXIndexOptions::GlobalOptions and call
+ * clang_createIndexWithOptions() instead.
+ *
* For example:
* \code
* CXIndex idx = ...;
@@ -327,6 +450,9 @@ CINDEX_LINKAGE void clang_CXIndex_setGlobalOptions(CXIndex, unsigned options);
/**
* Gets the general options associated with a CXIndex.
*
+ * This function allows to obtain the final option values used by libclang after
+ * specifying the option policies via CXChoice enumerators.
+ *
* \returns A bitmask of options, a bitwise OR of CXGlobalOpt_XXX flags that
* are associated with the given CXIndex object.
*/
@@ -335,6 +461,9 @@ CINDEX_LINKAGE unsigned clang_CXIndex_getGlobalOptions(CXIndex);
/**
* Sets the invocation emission path option in a CXIndex.
*
+ * This function is DEPRECATED. Set CXIndexOptions::InvocationEmissionPath and
+ * call clang_createIndexWithOptions() instead.
+ *
* The invocation emission path specifies a path which will contain log
* files for certain libclang invocations. A null value (default) implies that
* libclang invocations are not logged..
diff --git a/clang/include/clang/Frontend/ASTUnit.h b/clang/include/clang/Frontend/ASTUnit.h
index c792028902728..26f5191737200 100644
--- a/clang/include/clang/Frontend/ASTUnit.h
+++ b/clang/include/clang/Frontend/ASTUnit.h
@@ -124,6 +124,7 @@ class ASTUnit {
std::unique_ptr<ASTWriterData> WriterData;
FileSystemOptions FileSystemOpts;
+ std::string PreambleStoragePath;
/// The AST consumer that received information about the translation
/// unit as it was parsed or loaded.
@@ -802,6 +803,10 @@ class ASTUnit {
///
/// \param ResourceFilesPath - The path to the compiler resource files.
///
+ /// \param PreambleStoragePath - The path to a directory, in which to create
+ /// temporary PCH files. If empty, the default system temporary directory is
+ /// used.
+ ///
/// \param ModuleFormat - If provided, uses the specific module format.
///
/// \param ErrAST - If non-null and parsing failed without any AST to return
@@ -820,7 +825,7 @@ class ASTUnit {
const char **ArgBegin, const char **ArgEnd,
std::shared_ptr<PCHContainerOperations> PCHContainerOps,
IntrusiveRefCntPtr<DiagnosticsEngine> Diags, StringRef ResourceFilesPath,
- bool OnlyLocalDecls = false,
+ StringRef PreambleStoragePath = StringRef(), bool OnlyLocalDecls = false,
CaptureDiagsKind CaptureDiagnostics = CaptureDiagsKind::None,
ArrayRef<RemappedFile> RemappedFiles = std::nullopt,
bool RemappedFilesKeepOriginalName = true,
diff --git a/clang/include/clang/Frontend/PrecompiledPreamble.h b/clang/include/clang/Frontend/PrecompiledPreamble.h
index db9f33ae5961f..798870bf24fe1 100644
--- a/clang/include/clang/Frontend/PrecompiledPreamble.h
+++ b/clang/include/clang/Frontend/PrecompiledPreamble.h
@@ -75,6 +75,10 @@ class PrecompiledPreamble {
/// \param StoreInMemory Store PCH in memory. If false, PCH will be stored in
/// a temporary file.
///
+ /// \param StoragePath The path to a directory, in which to create a temporary
+ /// file to store PCH in. If empty, the default system temporary directory is
+ /// used. This parameter is ignored if \p StoreInMemory is true.
+ ///
/// \param Callbacks A set of callbacks to be executed when building
/// the preamble.
static llvm::ErrorOr<PrecompiledPreamble>
@@ -83,7 +87,8 @@ class PrecompiledPreamble {
DiagnosticsEngine &Diagnostics,
IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS,
std::shared_ptr<PCHContainerOperations> PCHContainerOps,
- bool StoreInMemory, PreambleCallbacks &Callbacks);
+ bool StoreInMemory, StringRef StoragePath,
+ PreambleCallbacks &Callbacks);
PrecompiledPreamble(PrecompiledPreamble &&);
PrecompiledPreamble &operator=(PrecompiledPreamble &&);
diff --git a/clang/lib/Frontend/ASTUnit.cpp b/clang/lib/Frontend/ASTUnit.cpp
index f2939a2e8f0ef..0707f11890375 100644
--- a/clang/lib/Frontend/ASTUnit.cpp
+++ b/clang/lib/Frontend/ASTUnit.cpp
@@ -1397,7 +1397,8 @@ ASTUnit::getMainBufferWithPrecompiledPreamble(
llvm::ErrorOr<PrecompiledPreamble> NewPreamble = PrecompiledPreamble::Build(
PreambleInvocationIn, MainFileBuffer.get(), Bounds, *Diagnostics, VFS,
- PCHContainerOps, /*StoreInMemory=*/false, Callbacks);
+ PCHContainerOps, /*StoreInMemory=*/false, PreambleStoragePath,
+ Callbacks);
PreambleInvocationIn.getFrontendOpts().SkipFunctionBodies =
PreviousSkipFunctionBodies;
@@ -1741,12 +1742,13 @@ ASTUnit *ASTUnit::LoadFromCommandLine(
const char **ArgBegin, const char **ArgEnd,
std::shared_ptr<PCHContainerOperations> PCHContainerOps,
IntrusiveRefCntPtr<DiagnosticsEngine> Diags, StringRef ResourceFilesPath,
- bool OnlyLocalDecls, CaptureDiagsKind CaptureDiagnostics,
- ArrayRef<RemappedFile> RemappedFiles, bool RemappedFilesKeepOriginalName,
- unsigned PrecompilePreambleAfterNParses, TranslationUnitKind TUKind,
- bool CacheCodeCompletionResults, bool IncludeBriefCommentsInCodeCompletion,
- bool AllowPCHWithCompilerErrors, SkipFunctionBodiesScope SkipFunctionBodies,
- bool SingleFileParse, bool UserFilesAreVolatile, bool ForSerialization,
+ StringRef PreambleStoragePath, bool OnlyLocalDecls,
+ CaptureDiagsKind CaptureDiagnostics, ArrayRef<RemappedFile> RemappedFiles,
+ bool RemappedFilesKeepOriginalName, unsigned PrecompilePreambleAfterNParses,
+ TranslationUnitKind TUKind, bool CacheCodeCompletionResults,
+ bool IncludeBriefCommentsInCodeCompletion, bool AllowPCHWithCompilerErrors,
+ SkipFunctionBodiesScope SkipFunctionBodies, bool SingleFileParse,
+ bool UserFilesAreVolatile, bool ForSerialization,
bool RetainExcludedConditionalBlocks, std::optional<StringRef> ModuleFormat,
std::unique_ptr<ASTUnit> *ErrAST,
IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS) {
@@ -1801,6 +1803,7 @@ ASTUnit *ASTUnit::LoadFromCommandLine(
VFS = llvm::vfs::getRealFileSystem();
VFS = createVFSFromCompilerInvocation(*CI, *Diags, VFS);
AST->FileMgr = new FileManager(AST->FileSystemOpts, VFS);
+ AST->PreambleStoragePath = PreambleStoragePath;
AST->ModuleCache = new InMemoryModuleCache;
AST->OnlyLocalDecls = OnlyLocalDecls;
AST->CaptureDiagnostics = CaptureDiagnostics;
diff --git a/clang/lib/Frontend/PrecompiledPreamble.cpp b/clang/lib/Frontend/PrecompiledPreamble.cpp
index 579a0b8b614de..a47d2b0be004c 100644
--- a/clang/lib/Frontend/PrecompiledPreamble.cpp
+++ b/clang/lib/Frontend/PrecompiledPreamble.cpp
@@ -197,20 +197,32 @@ void TemporaryFiles::removeFile(StringRef File) {
class TempPCHFile {
public:
// A main method used to construct TempPCHFile.
- static std::unique_ptr<TempPCHFile> create() {
+ static std::unique_ptr<TempPCHFile> create(StringRef StoragePath) {
// FIXME: This is a hack so that we can override the preamble file during
// crash-recovery testing, which is the only case where the preamble files
// are not necessarily cleaned up.
if (const char *TmpFile = ::getenv("CINDEXTEST_PREAMBLE_FILE"))
return std::unique_ptr<TempPCHFile>(new TempPCHFile(TmpFile));
- llvm::SmallString<64> File;
- // Using a version of createTemporaryFile with a file descriptor guarantees
+ llvm::SmallString<128> File;
+ // Using the versions of createTemporaryFile() and
+ // createUniqueFile() with a file descriptor guarantees
// that we would never get a race condition in a multi-threaded setting
// (i.e., multiple threads getting the same temporary path).
int FD;
- if (auto EC =
- llvm::sys::fs::createTemporaryFile("preamble", "pch", FD, File))
+ std::error_code EC;
+ if (StoragePath.empty())
+ EC = llvm::sys::fs::createTemporaryFile("preamble", "pch", FD, File);
+ else {
+ llvm::SmallString<128> TempPath = StoragePath;
+ // Use the same filename model as fs::createTemporaryFile().
+ llvm::sys::path::append(TempPath, "preamble-%%%%%%.pch");
+ namespace fs = llvm::sys::fs;
+ // Use the same owner-only file permissions as fs::createTemporaryFile().
+ EC = fs::createUniqueFile(TempPath, FD, File, fs::OF_None,
+ fs::owner_read | fs::owner_write);
+ }
+ if (EC)
return nullptr;
// We only needed to make sure the file exists, close the file right away.
llvm::sys::Process::SafelyCloseFileDescriptor(FD);
@@ -403,7 +415,7 @@ llvm::ErrorOr<PrecompiledPreamble> PrecompiledPreamble::Build(
DiagnosticsEngine &Diagnostics,
IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS,
std::shared_ptr<PCHContainerOperations> PCHContainerOps, bool StoreInMemory,
- PreambleCallbacks &Callbacks) {
+ StringRef StoragePath, PreambleCallbacks &Callbacks) {
assert(VFS && "VFS is null");
auto PreambleInvocation = std::make_shared<CompilerInvocation>(Invocation);
@@ -418,7 +430,8 @@ llvm::ErrorOr<PrecompiledPreamble> PrecompiledPreamble::Build(
} else {
// Create a temporary file for the precompiled preamble. In rare
// circumstances, this can fail.
- std::unique_ptr<TempPCHFile> PreamblePCHFile = TempPCHFile::create();
+ std::unique_ptr<TempPCHFile> PreamblePCHFile =
+ TempPCHFile::create(StoragePath);
if (!PreamblePCHFile)
return BuildPreambleError::CouldntCreateTempFile;
Storage = PCHStorage::file(std::move(PreamblePCHFile));
diff --git a/clang/tools/c-index-test/c-index-test.c b/clang/tools/c-index-test/c-index-test.c
index 983738dea1291..448435e83027c 100644
--- a/clang/tools/c-index-test/c-index-test.c
+++ b/clang/tools/c-index-test/c-index-test.c
@@ -67,6 +67,27 @@ extern char *basename(const char *);
extern char *dirname(char *);
#endif
+CXIndex createIndexWithInvocationEmissionPath(int ExcludeDeclarationsFromPCH,
+ int DisplayDiagnostics) {
+ CXIndex Idx;
+
+ CXIndexOptions Opts;
+ memset(&Opts, 0, sizeof(Opts));
+ Opts.Size = sizeof(CXIndexOptions);
+ Opts.ExcludeDeclarationsFromPCH = ExcludeDeclarationsFromPCH;
+ Opts.DisplayDiagnostics = DisplayDiagnostics;
+ Opts.InvocationEmissionPath = getenv("CINDEXTEST_INVOCATION_EMISSION_PATH");
+
+ Idx = clang_createIndexWithOptions(&Opts);
+ if (!Idx) {
+ fprintf(stderr,
+ "clang_createIndexWithOptions() failed. "
+ "CINDEX_VERSION_MINOR = %d, sizeof(CXIndexOptions) = %u\n",
+ CINDEX_VERSION_MINOR, Opts.Size);
+ }
+ return Idx;
+}
+
/** Return the default parsing options. */
static unsigned getDefaultParsingOptions(void) {
unsigned options = CXTranslationUnit_DetailedPreprocessingRecord;
@@ -2046,18 +2067,17 @@ int perform_test_load_source(int argc, const char **argv,
int result;
unsigned Repeats = 0;
unsigned I;
- const char *InvocationPath;
- Idx = clang_createIndex(/* excludeDeclsFromPCH */
- (!strcmp(filter, "local") ||
- !strcmp(filter, "local-display") ||
- !strcmp(filter, "local-pretty"))
- ? 1
- : 0,
- /* displayDiagnostics=*/1);
- InvocationPath = getenv("CINDEXTEST_INVOCATION_EMISSION_PATH");
- if (InvocationPath)
- clang_CXIndex_setInvocationEmissionPathOption(Idx, InvocationPath);
+ Idx =
+ createIndexWithInvocationEmissionPath(/* excludeDeclsFromPCH */
+ (!strcmp(filter, "local") ||
+ !strcmp(filter, "local-display") ||
+ !strcmp(filter, "local-pretty"))
+ ? 1
+ : 0,
+ /* displayDiagnostics=*/1);
+ if (!Idx)
+ return -1;
if ((CommentSchemaFile = parse_comments_schema(argc, argv))) {
argc--;
@@ -2701,7 +2721,6 @@ int perform_code_completion(int argc, const char **argv, int timing_only) {
CXTranslationUnit TU;
unsigned I, Repeats = 1;
unsigned completionOptions = clang_defaultCodeCompleteOptions();
- const char *InvocationPath;
if (getenv("CINDEXTEST_CODE_COMPLETE_PATTERNS"))
completionOptions |= CXCodeComplete_IncludeCodePatterns;
@@ -2724,10 +2743,9 @@ int perform_code_completion(int argc, const char **argv, int timing_only) {
if (parse_remapped_files(argc, argv, 2, &unsaved_files, &num_unsaved_files))
return -1;
- CIdx = clang_createIndex(0, 0);
- InvocationPath = getenv("CINDEXTEST_INVOCATION_EMISSION_PATH");
- if (InvocationPath)
- clang_CXIndex_setInvocationEmissionPathOption(CIdx, InvocationPath);
+ CIdx = createIndexWithInvocationEmissionPath(0, 0);
+ if (!CIdx)
+ return -1;
if (getenv("CINDEXTEST_EDITING"))
Repeats = 5;
@@ -4816,17 +4834,15 @@ static int perform_test_single_symbol_sgf(const char *input, int argc,
int num_unsaved_files = 0;
enum CXErrorCode Err;
int result = 0;
- const char *InvocationPath;
CXString SGF;
const char *usr;
usr = input + strlen("-single-symbol-sgf-for=");
- Idx = clang_createIndex(/* excludeDeclsFromPCH */ 1,
- /* displayDiagnostics=*/0);
- InvocationPath = getenv("CINDEXTEST_INVOCATION_EMISSION_PATH");
- if (InvocationPath)
- clang_CXIndex_setInvocationEmissionPathOption(Idx, InvocationPath);
+ Idx = createIndexWithInvocationEmissionPath(/* excludeDeclsFromPCH */ 1,
+ /* displayDiagnostics=*/0);
+ if (!Idx)
+ return -1;
if (parse_remapped_files(argc, argv, 0, &unsaved_files, &num_unsaved_files)) {
result = -1;
diff --git a/clang/tools/libclang/CIndex.cpp b/clang/tools/libclang/CIndex.cpp
index 46c9cfae202e5..959646707d8f4 100644
--- a/clang/tools/libclang/CIndex.cpp
+++ b/clang/tools/libclang/CIndex.cpp
@@ -3648,8 +3648,10 @@ struct RegisterFatalErrorHandler {
static llvm::ManagedStatic<RegisterFatalErrorHandler>
RegisterFatalErrorHandlerOnce;
-CXIndex clang_createIndex(int excludeDeclarationsFromPCH,
- int displayDiagnostics) {
+static CIndexer *clang_createIndex_Impl(
+ int excludeDeclarationsFromPCH, int displayDiagnostics,
+ unsigned char threadBackgroundPriorityForIndexing = CXChoice_Default,
+ unsigned char threadBackgroundPriorityForEditing = CXChoice_Default) {
// We use crash recovery to make some of our APIs more reliable, implicitly
// enable it.
if (!getenv("LIBCLANG_DISABLE_CRASH_RECOVERY"))
@@ -3673,21 +3675,71 @@ CXIndex clang_createIndex(int excludeDeclarationsFromPCH,
if (displayDiagnostics)
CIdxr->setDisplayDiagnostics();
- if (getenv("LIBCLANG_BGPRIO_INDEX"))
- CIdxr->setCXGlobalOptFlags(CIdxr->getCXGlobalOptFlags() |
- CXGlobalOpt_ThreadBackgroundPriorityForIndexing);
- if (getenv("LIBCLANG_BGPRIO_EDIT"))
- CIdxr->setCXGlobalOptFlags(CIdxr->getCXGlobalOptFlags() |
- CXGlobalOpt_ThreadBackgroundPriorityForEditing);
+ unsigned GlobalOptions = CIdxr->getCXGlobalOptFlags();
+ const auto updateGlobalOption =
+ [&GlobalOptions](unsigned char Policy, CXGlobalOptFlags Flag,
+ const char *EnvironmentVariableName) {
+ switch (Policy) {
+ case CXChoice_Enabled:
+ GlobalOptions |= Flag;
+ break;
+ case CXChoice_Disabled:
+ GlobalOptions &= ~Flag;
+ break;
+ case CXChoice_Default:
+ default: // Fall back to default behavior if Policy is unsupported.
+ if (getenv(EnvironmentVariableName))
+ GlobalOptions |= Flag;
+ }
+ };
+ updateGlobalOption(threadBackgroundPriorityForIndexing,
+ CXGlobalOpt_ThreadBackgroundPriorityForIndexing,
+ "LIBCLANG_BGPRIO_INDEX");
+ updateGlobalOption(threadBackgroundPriorityForEditing,
+ CXGlobalOpt_ThreadBackgroundPriorityForEditing,
+ "LIBCLANG_BGPRIO_EDIT");
+ CIdxr->setCXGlobalOptFlags(GlobalOptions);
return CIdxr;
}
+CXIndex clang_createIndex(int excludeDeclarationsFromPCH,
+ int displayDiagnostics) {
+ return clang_createIndex_Impl(excludeDeclarationsFromPCH, displayDiagnostics);
+}
+
void clang_disposeIndex(CXIndex CIdx) {
if (CIdx)
delete static_cast<CIndexer *>(CIdx);
}
+CXIndex clang_createIndexWithOptions(const CXIndexOptions *options) {
+ // Adding new options to struct CXIndexOptions:
+ // 1. If no other new option has been added in the same libclang version,
+ // sizeof(CXIndexOptions) must increase for versioning purposes.
+ // 2. Options should be added at the end of the struct in order to seamlessly
+ // support older struct versions. If options->Size < sizeof(CXIndexOptions),
+ // don't attempt to read the missing options and rely on the default values of
+ // recently added options being reasonable. For example:
+ // if (options->Size >= offsetof(CXIndexOptions, RecentlyAddedMember))
+ // do_something(options->RecentlyAddedMember);
+
+ // If options->Size > sizeof(CXIndexOptions), the user may have set an option
+ // we can't handle, in which case we return nullptr to report failure.
+ // Replace `!=` with `>` here to support older struct versions. `!=` has the
+ // advantage of catching more usage bugs and no disadvantages while there is a
+ // single supported struct version (the initial version).
+ if (options->Size != sizeof(CXIndexOptions))
+ return nullptr;
+ CIndexer *const CIdxr = clang_createIndex_Impl(
+ options->ExcludeDeclarationsFromPCH, options->DisplayDiagnostics,
+ options->ThreadBackgroundPriorityForIndexing,
+ options->ThreadBackgroundPriorityForEditing);
+ CIdxr->setPreambleStoragePath(options->PreambleStoragePath);
+ CIdxr->setInvocationEmissionPath(options->InvocationEmissionPath);
+ return CIdxr;
+}
+
void clang_CXIndex_setGlobalOptions(CXIndex CIdx, unsigned options) {
if (CIdx)
static_cast<CIndexer *>(CIdx)->setCXGlobalOptFlags(options);
@@ -3897,8 +3949,8 @@ clang_parseTranslationUnit_Impl(CXIndex CIdx, const char *source_filename,
std::unique_ptr<ASTUnit> Unit(ASTUnit::LoadFromCommandLine(
Args->data(), Args->data() + Args->size(),
CXXIdx->getPCHContainerOperations(), Diags,
- CXXIdx->getClangResourcesPath(), CXXIdx->getOnlyLocalDecls(),
- CaptureDiagnostics, *RemappedFiles.get(),
+ CXXIdx->getClangResourcesPath(), CXXIdx->getPreambleStoragePath(),
+ CXXIdx->getOnlyLocalDecls(), CaptureDiagnostics, *RemappedFiles.get(),
/*RemappedFilesKeepOriginalName=*/true, PrecompilePreambleAfterNParses,
TUKind, CacheCodeCompletionResults, IncludeBriefCommentsInCodeCompletion,
/*AllowPCHWithCompilerErrors=*/true, SkipFunctionBodies, SingleFileParse,
diff --git a/clang/tools/libclang/CIndexer.h b/clang/tools/libclang/CIndexer.h
index de15cc34b5e51..c0458326f4be2 100644
--- a/clang/tools/libclang/CIndexer.h
+++ b/clang/tools/libclang/CIndexer.h
@@ -41,6 +41,7 @@ class CIndexer {
std::string ToolchainPath;
+ std::string PreambleStoragePath;
std::string InvocationEmissionPath;
public:
@@ -77,6 +78,12 @@ class CIndexer {
StringRef getClangToolchainPath();
+ void setPreambleStoragePath(StringRef Str) {
+ PreambleStoragePath = Str.str();
+ }
+
+ StringRef getPreambleStoragePath() const { return PreambleStoragePath; }
+
void setInvocationEmissionPath(StringRef Str) {
InvocationEmissionPath = std::string(Str);
}
diff --git a/clang/tools/libclang/libclang.map b/clang/tools/libclang/libclang.map
index 80b244e0fa907..34b1ef1a54514 100644
--- a/clang/tools/libclang/libclang.map
+++ b/clang/tools/libclang/libclang.map
@@ -421,6 +421,7 @@ LLVM_16 {
LLVM_17 {
global:
clang_CXXMethod_isExplicit;
+ clang_createIndexWithOptions;
};
# Example of how to add a new symbol version entry. If you do add a new symbol
diff --git a/clang/unittests/Frontend/ASTUnitTest.cpp b/clang/unittests/Frontend/ASTUnitTest.cpp
index f3859536f368c..0785c6a71cdaf 100644
--- a/clang/unittests/Frontend/ASTUnitTest.cpp
+++ b/clang/unittests/Frontend/ASTUnitTest.cpp
@@ -167,7 +167,7 @@ TEST_F(ASTUnitTest, LoadFromCommandLineEarlyError) {
std::unique_ptr<clang::ASTUnit> ErrUnit;
ASTUnit *AST = ASTUnit::LoadFromCommandLine(
- &Args[0], &Args[4], PCHContainerOps, Diags, "", false,
+ &Args[0], &Args[4], PCHContainerOps, Diags, "", "", false,
CaptureDiagsKind::All, std::nullopt, true, 0, TU_Complete, false, false,
false, SkipFunctionBodiesScope::None, false, true, false, false,
std::nullopt, &ErrUnit, nullptr);
diff --git a/clang/unittests/libclang/LibclangTest.cpp b/clang/unittests/libclang/LibclangTest.cpp
index 11a729fc040d2..18d0fc12a51cf 100644
--- a/clang/unittests/libclang/LibclangTest.cpp
+++ b/clang/unittests/libclang/LibclangTest.cpp
@@ -6,6 +6,7 @@
//
//===----------------------------------------------------------------------===//
+#include "TestUtils.h"
#include "clang-c/Index.h"
#include "clang-c/Rewrite.h"
#include "llvm/ADT/StringRef.h"
@@ -14,7 +15,7 @@
#include "llvm/Support/Path.h"
#include "llvm/Support/raw_ostream.h"
#include "gtest/gtest.h"
-#include "TestUtils.h"
+#include <cstring>
#include <fstream>
#include <functional>
#include <map>
@@ -355,6 +356,168 @@ TEST(libclang, ModuleMapDescriptor) {
clang_ModuleMapDescriptor_dispose(MMD);
}
+TEST_F(LibclangParseTest, GlobalOptions) {
+ EXPECT_EQ(clang_CXIndex_getGlobalOptions(Index), CXGlobalOpt_None);
+}
+
+class LibclangIndexOptionsTest : public LibclangParseTest {
+ virtual void AdjustOptions(CXIndexOptions &Opts) {}
+
+protected:
+ void CreateIndex() override {
+ CXIndexOptions Opts;
+ memset(&Opts, 0, sizeof(Opts));
+ Opts.Size = sizeof(CXIndexOptions);
+ AdjustOptions(Opts);
+ Index = clang_createIndexWithOptions(&Opts);
+ ASSERT_TRUE(Index);
+ }
+};
+
+TEST_F(LibclangIndexOptionsTest, GlobalOptions) {
+ EXPECT_EQ(clang_CXIndex_getGlobalOptions(Index), CXGlobalOpt_None);
+}
+
+class LibclangIndexingEnabledIndexOptionsTest
+ : public LibclangIndexOptionsTest {
+ void AdjustOptions(CXIndexOptions &Opts) override {
+ Opts.ThreadBackgroundPriorityForIndexing = CXChoice_Enabled;
+ }
+};
+
+TEST_F(LibclangIndexingEnabledIndexOptionsTest, GlobalOptions) {
+ EXPECT_EQ(clang_CXIndex_getGlobalOptions(Index),
+ CXGlobalOpt_ThreadBackgroundPriorityForIndexing);
+}
+
+class LibclangIndexingDisabledEditingEnabledIndexOptionsTest
+ : public LibclangIndexOptionsTest {
+ void AdjustOptions(CXIndexOptions &Opts) override {
+ Opts.ThreadBackgroundPriorityForIndexing = CXChoice_Disabled;
+ Opts.ThreadBackgroundPriorityForEditing = CXChoice_Enabled;
+ }
+};
+
+TEST_F(LibclangIndexingDisabledEditingEnabledIndexOptionsTest, GlobalOptions) {
+ EXPECT_EQ(clang_CXIndex_getGlobalOptions(Index),
+ CXGlobalOpt_ThreadBackgroundPriorityForEditing);
+}
+
+class LibclangBothEnabledIndexOptionsTest : public LibclangIndexOptionsTest {
+ void AdjustOptions(CXIndexOptions &Opts) override {
+ Opts.ThreadBackgroundPriorityForIndexing = CXChoice_Enabled;
+ Opts.ThreadBackgroundPriorityForEditing = CXChoice_Enabled;
+ }
+};
+
+TEST_F(LibclangBothEnabledIndexOptionsTest, GlobalOptions) {
+ EXPECT_EQ(clang_CXIndex_getGlobalOptions(Index),
+ CXGlobalOpt_ThreadBackgroundPriorityForAll);
+}
+
+class LibclangPreambleStorageTest : public LibclangParseTest {
+ std::string Main = "main.cpp";
+
+protected:
+ std::string PreambleDir;
+ void InitializePreambleDir() {
+ llvm::SmallString<128> PathBuffer(TestDir);
+ llvm::sys::path::append(PathBuffer, "preambles");
+ namespace fs = llvm::sys::fs;
+ ASSERT_FALSE(fs::create_directory(PathBuffer, false, fs::perms::owner_all));
+
+ PreambleDir = static_cast<std::string>(PathBuffer);
+ FilesAndDirsToRemove.insert(PreambleDir);
+ }
+
+public:
+ void CountPreamblesInPreambleDir(int PreambleCount) {
+ // For some reason, the preamble is not created without '\n' before `int`.
+ WriteFile(Main, "\nint main() {}");
+
+ TUFlags |= CXTranslationUnit_CreatePreambleOnFirstParse;
+ ClangTU = clang_parseTranslationUnit(Index, Main.c_str(), nullptr, 0,
+ nullptr, 0, TUFlags);
+
+ int FileCount = 0;
+
+ namespace fs = llvm::sys::fs;
+ std::error_code EC;
+ for (fs::directory_iterator File(PreambleDir, EC), FileEnd;
+ File != FileEnd && !EC; File.increment(EC)) {
+ ++FileCount;
+
+ EXPECT_EQ(File->type(), fs::file_type::regular_file);
+
+ const auto Filename = llvm::sys::path::filename(File->path());
+ EXPECT_EQ(Filename.size(), std::strlen("preamble-%%%%%%.pch"));
+ EXPECT_TRUE(Filename.startswith("preamble-"));
+ EXPECT_TRUE(Filename.endswith(".pch"));
+
+ const auto Status = File->status();
+ ASSERT_TRUE(Status);
+ if (false) {
+ // The permissions assertion below fails, because the .pch.tmp file is
+ // created with default permissions and replaces the .pch file along
+ // with its permissions. Therefore the permissions set in
+ // TempPCHFile::create() don't matter in the end.
+ EXPECT_EQ(Status->permissions(), fs::owner_read | fs::owner_write);
+ }
+ }
+
+ EXPECT_EQ(FileCount, PreambleCount);
+ }
+};
+
+class LibclangNotOverriddenPreambleStoragePathTest
+ : public LibclangPreambleStorageTest {
+protected:
+ void CreateIndex() override {
+ InitializePreambleDir();
+ LibclangPreambleStorageTest::CreateIndex();
+ }
+};
+
+class LibclangSetPreambleStoragePathTest : public LibclangPreambleStorageTest {
+ virtual const char *PreambleStoragePath() = 0;
+
+protected:
+ void CreateIndex() override {
+ InitializePreambleDir();
+
+ CXIndexOptions Opts = {sizeof(CXIndexOptions)};
+ Opts.PreambleStoragePath = PreambleStoragePath();
+ Index = clang_createIndexWithOptions(&Opts);
+ ASSERT_TRUE(Index);
+ }
+};
+
+class LibclangNullPreambleStoragePathTest
+ : public LibclangSetPreambleStoragePathTest {
+ const char *PreambleStoragePath() override { return nullptr; }
+};
+class LibclangEmptyPreambleStoragePathTest
+ : public LibclangSetPreambleStoragePathTest {
+ const char *PreambleStoragePath() override { return ""; }
+};
+class LibclangPreambleDirPreambleStoragePathTest
+ : public LibclangSetPreambleStoragePathTest {
+ const char *PreambleStoragePath() override { return PreambleDir.c_str(); }
+};
+
+TEST_F(LibclangNotOverriddenPreambleStoragePathTest, CountPreambles) {
+ CountPreamblesInPreambleDir(0);
+}
+TEST_F(LibclangNullPreambleStoragePathTest, CountPreambles) {
+ CountPreamblesInPreambleDir(0);
+}
+TEST_F(LibclangEmptyPreambleStoragePathTest, CountPreambles) {
+ CountPreamblesInPreambleDir(0);
+}
+TEST_F(LibclangPreambleDirPreambleStoragePathTest, CountPreambles) {
+ CountPreamblesInPreambleDir(1);
+}
+
TEST_F(LibclangParseTest, AllSkippedRanges) {
std::string Header = "header.h", Main = "main.cpp";
WriteFile(Header,
diff --git a/clang/unittests/libclang/TestUtils.h b/clang/unittests/libclang/TestUtils.h
index c69746c4408a9..c78351d4afd20 100644
--- a/clang/unittests/libclang/TestUtils.h
+++ b/clang/unittests/libclang/TestUtils.h
@@ -22,11 +22,11 @@
#include <vector>
class LibclangParseTest : public ::testing::Test {
- // std::greater<> to remove files before their parent dirs in TearDown().
- std::set<std::string, std::greater<>> Files;
typedef std::unique_ptr<std::string> fixed_addr_string;
std::map<fixed_addr_string, fixed_addr_string> UnsavedFileContents;
public:
+ // std::greater<> to remove files before their parent dirs in TearDown().
+ std::set<std::string, std::greater<>> FilesAndDirsToRemove;
std::string TestDir;
bool RemoveTestDirRecursivelyDuringTeardown = false;
CXIndex Index;
@@ -40,7 +40,7 @@ class LibclangParseTest : public ::testing::Test {
TestDir = std::string(Dir.str());
TUFlags = CXTranslationUnit_DetailedPreprocessingRecord |
clang_defaultEditingTranslationUnitOptions();
- Index = clang_createIndex(0, 0);
+ CreateIndex();
ClangTU = nullptr;
}
void TearDown() override {
@@ -48,7 +48,7 @@ class LibclangParseTest : public ::testing::Test {
clang_disposeIndex(Index);
namespace fs = llvm::sys::fs;
- for (const std::string &Path : Files)
+ for (const std::string &Path : FilesAndDirsToRemove)
EXPECT_FALSE(fs::remove(Path, /*IgnoreNonExisting=*/false));
if (RemoveTestDirRecursivelyDuringTeardown)
EXPECT_FALSE(fs::remove_directories(TestDir, /*IgnoreErrors=*/false));
@@ -63,7 +63,7 @@ class LibclangParseTest : public ::testing::Test {
FileI != FileEnd; ++FileI) {
ASSERT_NE(*FileI, ".");
path::append(Path, *FileI);
- Files.emplace(Path.str());
+ FilesAndDirsToRemove.emplace(Path.str());
}
Filename = std::string(Path.str());
}
@@ -101,6 +101,9 @@ class LibclangParseTest : public ::testing::Test {
return string;
};
+protected:
+ virtual void CreateIndex() { Index = clang_createIndex(0, 0); }
+
private:
template<typename TState>
static CXChildVisitResult TraverseStateless(CXCursor cx, CXCursor parent,
More information about the cfe-commits
mailing list