r316653 - [Tooling] A new framework for executing clang frontend actions.
Maxim Kuvyrkov via cfe-commits
cfe-commits at lists.llvm.org
Thu Oct 26 06:11:37 PDT 2017
Hi Eric,
This seems to have broken clang-cmake-thumbv7-a15 buildbot [*]. Would you please investigate?
Or did you fix this while I was writing this email in 316661 :-) ?
Thank you,
[*] http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15/builds/12511
--
Maxim Kuvyrkov
www.linaro.org
> On Oct 26, 2017, at 1:38 PM, Eric Liu via cfe-commits <cfe-commits at lists.llvm.org> wrote:
>
> Author: ioeric
> Date: Thu Oct 26 03:38:14 2017
> New Revision: 316653
>
> URL: http://llvm.org/viewvc/llvm-project?rev=316653&view=rev
> Log:
> [Tooling] A new framework for executing clang frontend actions.
>
> Summary:
> This defines a `clang::tooling::ToolExecutor` interface that can be extended to support different execution plans including standalone execution on a given set of TUs or parallel execution on all TUs in a codebase.
>
> In order to enable multiprocessing execution, tool actions are expected to output result into a `ToolResults` interface provided by executors. The `ToolResults` interface abstracts how results are stored e.g. in-memory for standalone executions or on-disk for large-scale execution.
>
> New executors can be registered as `ToolExecutorPlugin`s via the `ToolExecutorPluginRegistry`. CLI tools can use `createExecutorFromCommandLineArgs` to create a specific registered executor according to the command-line arguments.
>
> This patch also implements `StandaloneToolExecutor` which has the same behavior as the current `ClangTool` interface, i.e. execute frontend actions on a given set of TUs. At this point, it's simply a wrapper around `ClangTool` at this point.
>
> This is still experimental but expected to replace the existing `ClangTool` interface so that specific tools would not need to worry about execution.
>
> Reviewers: klimek, arphaman, hokein, sammccall
>
> Reviewed By: klimek
>
> Subscribers: cfe-commits, djasper, mgorny, omtcyfz
>
> Differential Revision: https://reviews.llvm.org/D34272
>
> Added:
> cfe/trunk/include/clang/Tooling/Execution.h
> cfe/trunk/include/clang/Tooling/StandaloneExecution.h
> cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h
> cfe/trunk/lib/Tooling/Execution.cpp
> cfe/trunk/lib/Tooling/StandaloneExecution.cpp
> cfe/trunk/unittests/Tooling/ExecutionTest.cpp
> Modified:
> cfe/trunk/include/clang/Tooling/CommonOptionsParser.h
> cfe/trunk/include/clang/Tooling/Tooling.h
> cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp
> cfe/trunk/lib/Tooling/CMakeLists.txt
> cfe/trunk/lib/Tooling/CommonOptionsParser.cpp
> cfe/trunk/lib/Tooling/Tooling.cpp
> cfe/trunk/unittests/Tooling/CMakeLists.txt
>
> Modified: cfe/trunk/include/clang/Tooling/CommonOptionsParser.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/CommonOptionsParser.h?rev=316653&r1=316652&r2=316653&view=diff
> ==============================================================================
> --- cfe/trunk/include/clang/Tooling/CommonOptionsParser.h (original)
> +++ cfe/trunk/include/clang/Tooling/CommonOptionsParser.h Thu Oct 26 03:38:14 2017
> @@ -109,6 +109,10 @@ public:
> return SourcePathList;
> }
>
> + /// Returns the argument adjuster calculated from "--extra-arg" and
> + //"--extra-arg-before" options.
> + ArgumentsAdjuster getArgumentsAdjuster() { return Adjuster; }
> +
> static const char *const HelpMessage;
>
> private:
> @@ -121,6 +125,7 @@ private:
>
> std::unique_ptr<CompilationDatabase> Compilations;
> std::vector<std::string> SourcePathList;
> + ArgumentsAdjuster Adjuster;
> };
>
> class ArgumentsAdjustingCompilations : public CompilationDatabase {
>
> Added: cfe/trunk/include/clang/Tooling/Execution.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Execution.h?rev=316653&view=auto
> ==============================================================================
> --- cfe/trunk/include/clang/Tooling/Execution.h (added)
> +++ cfe/trunk/include/clang/Tooling/Execution.h Thu Oct 26 03:38:14 2017
> @@ -0,0 +1,168 @@
> +//===--- Execution.h - Executing clang frontend actions -*- C++ ---------*-===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file defines framework for executing clang frontend actions.
> +//
> +// The framework can be extended to support different execution plans including
> +// standalone execution on the given TUs or parallel execution on all TUs in
> +// the codebase.
> +//
> +// In order to enable multiprocessing execution, tool actions are expected to
> +// output result into the ToolResults provided by the executor. The
> +// `ToolResults` is an interface that abstracts how results are stored e.g.
> +// in-memory for standalone execution or on-disk for large-scale execution.
> +//
> +// New executors can be registered as ToolExecutorPlugins via the
> +// `ToolExecutorPluginRegistry`. CLI tools can use
> +// `createExecutorFromCommandLineArgs` to create a specific registered executor
> +// according to the command-line arguments.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef LLVM_CLANG_TOOLING_EXECUTION_H
> +#define LLVM_CLANG_TOOLING_EXECUTION_H
> +
> +#include "clang/Tooling/CommonOptionsParser.h"
> +#include "clang/Tooling/Tooling.h"
> +#include "llvm/Support/Error.h"
> +#include "llvm/Support/Registry.h"
> +
> +namespace clang {
> +namespace tooling {
> +
> +/// \brief An abstraction for the result of a tool execution. For example, the
> +/// underlying result can be in-memory or on-disk.
> +///
> +/// Results should be string key-value pairs. For example, a refactoring tool
> +/// can use source location as key and a replacement in YAML format as value.
> +class ToolResults {
> +public:
> + virtual ~ToolResults() = default;
> + virtual void addResult(StringRef Key, StringRef Value) = 0;
> + virtual std::vector<std::pair<std::string, std::string>> AllKVResults() = 0;
> + virtual void forEachResult(
> + llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) = 0;
> +};
> +
> +class InMemoryToolResults : public ToolResults {
> +public:
> + void addResult(StringRef Key, StringRef Value) override;
> + std::vector<std::pair<std::string, std::string>> AllKVResults() override;
> + void forEachResult(llvm::function_ref<void(StringRef Key, StringRef Value)>
> + Callback) override;
> +
> +private:
> + std::vector<std::pair<std::string, std::string>> KVResults;
> +};
> +
> +/// \brief The context of an execution, including the information about
> +/// compilation and results.
> +class ExecutionContext {
> +public:
> + virtual ~ExecutionContext() {}
> +
> + /// \brief Initializes a context. This does not take ownership of `Results`.
> + explicit ExecutionContext(ToolResults *Results) : Results(Results) {}
> +
> + /// \brief Adds a KV pair to the result container of this execution.
> + void reportResult(StringRef Key, StringRef Value);
> +
> + // Returns the source control system's revision number if applicable.
> + // Otherwise returns an empty string.
> + virtual std::string getRevision() { return ""; }
> +
> + // Returns the corpus being analyzed, e.g. "llvm" for the LLVM codebase, if
> + // applicable.
> + virtual std::string getCorpus() { return ""; }
> +
> + // Returns the currently processed compilation unit if available.
> + virtual std::string getCurrentCompilationUnit() { return ""; }
> +
> +private:
> + ToolResults *Results;
> +};
> +
> +/// \brief Interface for executing clang frontend actions.
> +///
> +/// This can be extended to support running tool actions in different
> +/// execution mode, e.g. on a specific set of TUs or many TUs in parallel.
> +///
> +/// New executors can be registered as ToolExecutorPlugins via the
> +/// `ToolExecutorPluginRegistry`. CLI tools can use
> +/// `createExecutorFromCommandLineArgs` to create a specific registered
> +/// executor according to the command-line arguments.
> +class ToolExecutor {
> +public:
> + virtual ~ToolExecutor() {}
> +
> + /// \brief Returns the name of a specific executor.
> + virtual StringRef getExecutorName() const = 0;
> +
> + /// \brief Executes each action with a corresponding arguments adjuster.
> + virtual llvm::Error
> + execute(llvm::ArrayRef<
> + std::pair<std::unique_ptr<FrontendActionFactory>, ArgumentsAdjuster>>
> + Actions) = 0;
> +
> + /// \brief Convenient functions for the above `execute`.
> + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action);
> + /// Executes an action with an argument adjuster.
> + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action,
> + ArgumentsAdjuster Adjuster);
> +
> + /// \brief Returns a reference to the execution context.
> + ///
> + /// This should be passed to tool callbacks, and tool callbacks should report
> + /// results via the returned context.
> + virtual ExecutionContext *getExecutionContext() = 0;
> +
> + /// \brief Returns a reference to the result container.
> + ///
> + /// NOTE: This should only be used after the execution finishes. Tool
> + /// callbacks should report results via `ExecutionContext` instead.
> + virtual ToolResults *getToolResults() = 0;
> +
> + /// \brief Map a virtual file to be used while running the tool.
> + ///
> + /// \param FilePath The path at which the content will be mapped.
> + /// \param Content A buffer of the file's content.
> + virtual void mapVirtualFile(StringRef FilePath, StringRef Content) = 0;
> +};
> +
> +/// \brief Interface for factories that create specific executors. This is also
> +/// used as a plugin to be registered into ToolExecutorPluginRegistry.
> +class ToolExecutorPlugin {
> +public:
> + virtual ~ToolExecutorPlugin() {}
> +
> + /// \brief Create an `ToolExecutor`.
> + ///
> + /// `OptionsParser` can be consumed (e.g. moved) if the creation succeeds.
> + virtual llvm::Expected<std::unique_ptr<ToolExecutor>>
> + create(CommonOptionsParser &OptionsParser) = 0;
> +};
> +
> +/// \brief This creates a ToolExecutor that is in the global registry based on
> +/// commandline arguments.
> +///
> +/// This picks the right executor based on the `--executor` option. This parses
> +/// the commandline arguments with `CommonOptionsParser`, so caller does not
> +/// need to parse again.
> +///
> +/// By default, this creates a `StandaloneToolExecutor` ("standalone") if
> +/// `--executor` is not provided.
> +llvm::Expected<std::unique_ptr<ToolExecutor>>
> +createExecutorFromCommandLineArgs(int &argc, const char **argv,
> + llvm::cl::OptionCategory &Category,
> + const char *Overview = nullptr);
> +
> +} // end namespace tooling
> +} // end namespace clang
> +
> +#endif // LLVM_CLANG_TOOLING_EXECUTION_H
>
> Added: cfe/trunk/include/clang/Tooling/StandaloneExecution.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/StandaloneExecution.h?rev=316653&view=auto
> ==============================================================================
> --- cfe/trunk/include/clang/Tooling/StandaloneExecution.h (added)
> +++ cfe/trunk/include/clang/Tooling/StandaloneExecution.h Thu Oct 26 03:38:14 2017
> @@ -0,0 +1,97 @@
> +//===--- StandaloneExecution.h - Standalone execution. -*- C++ ----------*-===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file defines standalone execution of clang tools.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
> +#define LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
> +
> +#include "clang/Tooling/ArgumentsAdjusters.h"
> +#include "clang/Tooling/Execution.h"
> +
> +namespace clang {
> +namespace tooling {
> +
> +/// \brief A standalone executor that runs FrontendActions on a given set of
> +/// TUs in sequence.
> +///
> +/// By default, this executor uses the following arguments adjusters (as defined
> +/// in `clang/Tooling/ArgumentsAdjusters.h`):
> +/// - `getClangStripOutputAdjuster()`
> +/// - `getClangSyntaxOnlyAdjuster()`
> +/// - `getClangStripDependencyFileAdjuster()`
> +class StandaloneToolExecutor : public ToolExecutor {
> +public:
> + static const char *ExecutorName;
> +
> + /// \brief Init with \p CompilationDatabase and the paths of all files to be
> + /// proccessed.
> + StandaloneToolExecutor(
> + const CompilationDatabase &Compilations,
> + llvm::ArrayRef<std::string> SourcePaths,
> + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
> + std::make_shared<PCHContainerOperations>());
> +
> + /// \brief Init with \p CommonOptionsParser. This is expected to be used by
> + /// `createExecutorFromCommandLineArgs` based on commandline options.
> + ///
> + /// The executor takes ownership of \p Options.
> + StandaloneToolExecutor(
> + CommonOptionsParser Options,
> + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
> + std::make_shared<PCHContainerOperations>());
> +
> + StringRef getExecutorName() const override { return ExecutorName; }
> +
> + using ToolExecutor::execute;
> +
> + llvm::Error
> + execute(llvm::ArrayRef<
> + std::pair<std::unique_ptr<FrontendActionFactory>, ArgumentsAdjuster>>
> + Actions) override;
> +
> + /// \brief Set a \c DiagnosticConsumer to use during parsing.
> + void setDiagnosticConsumer(DiagnosticConsumer *DiagConsumer) {
> + Tool.setDiagnosticConsumer(DiagConsumer);
> + }
> +
> + ExecutionContext *getExecutionContext() override { return &Context; };
> +
> + ToolResults *getToolResults() override { return &Results; }
> +
> + llvm::ArrayRef<std::string> getSourcePaths() const {
> + return Tool.getSourcePaths();
> + }
> +
> + void mapVirtualFile(StringRef FilePath, StringRef Content) override {
> + Tool.mapVirtualFile(FilePath, Content);
> + }
> +
> + /// \brief Returns the file manager used in the tool.
> + ///
> + /// The file manager is shared between all translation units.
> + FileManager &getFiles() { return Tool.getFiles(); }
> +
> +private:
> + // Used to store the parser when the executor is initialized with parser.
> + llvm::Optional<CommonOptionsParser> OptionsParser;
> + // FIXME: The standalone executor is currently just a wrapper of `ClangTool`.
> + // Merge `ClangTool` implementation into the this.
> + ClangTool Tool;
> + ExecutionContext Context;
> + InMemoryToolResults Results;
> + ArgumentsAdjuster ArgsAdjuster;
> +};
> +
> +} // end namespace tooling
> +} // end namespace clang
> +
> +#endif // LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
>
> Added: cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h?rev=316653&view=auto
> ==============================================================================
> --- cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h (added)
> +++ cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h Thu Oct 26 03:38:14 2017
> @@ -0,0 +1,24 @@
> +//===--- ToolExecutorPluginRegistry.h - -------------------------*- C++ -*-===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
> +#define LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
> +
> +#include "clang/Tooling/Execution.h"
> +#include "llvm/Support/Registry.h"
> +
> +namespace clang {
> +namespace tooling {
> +
> +typedef llvm::Registry<ToolExecutorPlugin> ToolExecutorPluginRegistry;
> +
> +} // end namespace tooling
> +} // end namespace clang
> +
> +#endif // LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
>
> Modified: cfe/trunk/include/clang/Tooling/Tooling.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Tooling.h?rev=316653&r1=316652&r2=316653&view=diff
> ==============================================================================
> --- cfe/trunk/include/clang/Tooling/Tooling.h (original)
> +++ cfe/trunk/include/clang/Tooling/Tooling.h Thu Oct 26 03:38:14 2017
> @@ -31,12 +31,12 @@
> #define LLVM_CLANG_TOOLING_TOOLING_H
>
> #include "clang/AST/ASTConsumer.h"
> -#include "clang/Frontend/PCHContainerOperations.h"
> #include "clang/Basic/Diagnostic.h"
> #include "clang/Basic/FileManager.h"
> #include "clang/Basic/LLVM.h"
> #include "clang/Driver/Util.h"
> #include "clang/Frontend/FrontendAction.h"
> +#include "clang/Frontend/PCHContainerOperations.h"
> #include "clang/Lex/ModuleLoader.h"
> #include "clang/Tooling/ArgumentsAdjusters.h"
> #include "clang/Tooling/CompilationDatabase.h"
> @@ -337,7 +337,9 @@ class ClangTool {
> /// The file manager is shared between all translation units.
> FileManager &getFiles() { return *Files; }
>
> - private:
> + llvm::ArrayRef<std::string> getSourcePaths() const { return SourcePaths; }
> +
> +private:
> const CompilationDatabase &Compilations;
> std::vector<std::string> SourcePaths;
> std::shared_ptr<PCHContainerOperations> PCHContainerOps;
>
> Modified: cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp?rev=316653&r1=316652&r2=316653&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp (original)
> +++ cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp Thu Oct 26 03:38:14 2017
> @@ -96,6 +96,10 @@ ArgumentsAdjuster getInsertArgumentAdjus
>
> ArgumentsAdjuster combineAdjusters(ArgumentsAdjuster First,
> ArgumentsAdjuster Second) {
> + if (!First)
> + return Second;
> + if (!Second)
> + return First;
> return [First, Second](const CommandLineArguments &Args, StringRef File) {
> return Second(First(Args, File), File);
> };
>
> Modified: cfe/trunk/lib/Tooling/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Tooling/CMakeLists.txt (original)
> +++ cfe/trunk/lib/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017
> @@ -11,11 +11,13 @@ add_clang_library(clangTooling
> ArgumentsAdjusters.cpp
> CommonOptionsParser.cpp
> CompilationDatabase.cpp
> + Execution.cpp
> FileMatchTrie.cpp
> FixIt.cpp
> JSONCompilationDatabase.cpp
> Refactoring.cpp
> RefactoringCallbacks.cpp
> + StandaloneExecution.cpp
> Tooling.cpp
>
> DEPENDS
>
> Modified: cfe/trunk/lib/Tooling/CommonOptionsParser.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CommonOptionsParser.cpp?rev=316653&r1=316652&r2=316653&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Tooling/CommonOptionsParser.cpp (original)
> +++ cfe/trunk/lib/Tooling/CommonOptionsParser.cpp Thu Oct 26 03:38:14 2017
> @@ -147,10 +147,12 @@ llvm::Error CommonOptionsParser::init(
> auto AdjustingCompilations =
> llvm::make_unique<ArgumentsAdjustingCompilations>(
> std::move(Compilations));
> - AdjustingCompilations->appendArgumentsAdjuster(
> - getInsertArgumentAdjuster(ArgsBefore, ArgumentInsertPosition::BEGIN));
> - AdjustingCompilations->appendArgumentsAdjuster(
> + Adjuster =
> + getInsertArgumentAdjuster(ArgsBefore, ArgumentInsertPosition::BEGIN);
> + Adjuster = combineAdjusters(
> + std::move(Adjuster),
> getInsertArgumentAdjuster(ArgsAfter, ArgumentInsertPosition::END));
> + AdjustingCompilations->appendArgumentsAdjuster(Adjuster);
> Compilations = std::move(AdjustingCompilations);
> return llvm::Error::success();
> }
>
> Added: cfe/trunk/lib/Tooling/Execution.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Execution.cpp?rev=316653&view=auto
> ==============================================================================
> --- cfe/trunk/lib/Tooling/Execution.cpp (added)
> +++ cfe/trunk/lib/Tooling/Execution.cpp Thu Oct 26 03:38:14 2017
> @@ -0,0 +1,89 @@
> +//===- lib/Tooling/Execution.cpp - Implements tool execution framework. ---===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "clang/Tooling/Execution.h"
> +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
> +#include "clang/Tooling/Tooling.h"
> +
> +LLVM_INSTANTIATE_REGISTRY(clang::tooling::ToolExecutorPluginRegistry)
> +
> +namespace clang {
> +namespace tooling {
> +
> +static llvm::cl::opt<std::string>
> + ExecutorName("executor", llvm::cl::desc("The name of the executor to use."),
> + llvm::cl::init("standalone"));
> +
> +void InMemoryToolResults::addResult(StringRef Key, StringRef Value) {
> + KVResults.push_back({Key.str(), Value.str()});
> +}
> +
> +std::vector<std::pair<std::string, std::string>>
> +InMemoryToolResults::AllKVResults() {
> + return KVResults;
> +}
> +
> +void InMemoryToolResults::forEachResult(
> + llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) {
> + for (const auto &KV : KVResults) {
> + Callback(KV.first, KV.second);
> + }
> +}
> +
> +void ExecutionContext::reportResult(StringRef Key, StringRef Value) {
> + Results->addResult(Key, Value);
> +}
> +
> +llvm::Error
> +ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action) {
> + return execute(std::move(Action), ArgumentsAdjuster());
> +}
> +
> +llvm::Error ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action,
> + ArgumentsAdjuster Adjuster) {
> + std::vector<
> + std::pair<std::unique_ptr<FrontendActionFactory>, ArgumentsAdjuster>>
> + Actions;
> + Actions.emplace_back(std::move(Action), std::move(Adjuster));
> + return execute(Actions);
> +}
> +
> +llvm::Expected<std::unique_ptr<ToolExecutor>>
> +createExecutorFromCommandLineArgs(int &argc, const char **argv,
> + llvm::cl::OptionCategory &Category,
> + const char *Overview) {
> + auto OptionsParser =
> + CommonOptionsParser::create(argc, argv, Category, llvm::cl::ZeroOrMore,
> + /*Overview=*/nullptr);
> + if (!OptionsParser)
> + return OptionsParser.takeError();
> + for (auto I = ToolExecutorPluginRegistry::begin(),
> + E = ToolExecutorPluginRegistry::end();
> + I != E; ++I) {
> + if (I->getName() != ExecutorName) {
> + continue;
> + }
> + std::unique_ptr<ToolExecutorPlugin> Plugin(I->instantiate());
> + llvm::Expected<std::unique_ptr<ToolExecutor>> Executor =
> + Plugin->create(*OptionsParser);
> + if (!Executor) {
> + return llvm::make_error<llvm::StringError>(
> + llvm::Twine("Failed to create '") + I->getName() +
> + "': " + llvm::toString(Executor.takeError()) + "\n",
> + llvm::inconvertibleErrorCode());
> + }
> + return std::move(*Executor);
> + }
> + return llvm::make_error<llvm::StringError>(
> + llvm::Twine("Executor \"") + ExecutorName + "\" is not registered.",
> + llvm::inconvertibleErrorCode());
> +}
> +
> +} // end namespace tooling
> +} // end namespace clang
>
> Added: cfe/trunk/lib/Tooling/StandaloneExecution.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/StandaloneExecution.cpp?rev=316653&view=auto
> ==============================================================================
> --- cfe/trunk/lib/Tooling/StandaloneExecution.cpp (added)
> +++ cfe/trunk/lib/Tooling/StandaloneExecution.cpp Thu Oct 26 03:38:14 2017
> @@ -0,0 +1,91 @@
> +//===- lib/Tooling/Execution.cpp - Standalone clang action execution. -----===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "clang/Tooling/StandaloneExecution.h"
> +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
> +
> +namespace clang {
> +namespace tooling {
> +
> +static llvm::Error make_string_error(const llvm::Twine &Message) {
> + return llvm::make_error<llvm::StringError>(Message,
> + llvm::inconvertibleErrorCode());
> +}
> +
> +const char *StandaloneToolExecutor::ExecutorName = "StandaloneToolExecutor";
> +
> +static ArgumentsAdjuster getDefaultArgumentsAdjusters() {
> + return combineAdjusters(
> + getClangStripOutputAdjuster(),
> + combineAdjusters(getClangSyntaxOnlyAdjuster(),
> + getClangStripDependencyFileAdjuster()));
> +}
> +
> +StandaloneToolExecutor::StandaloneToolExecutor(
> + const CompilationDatabase &Compilations,
> + llvm::ArrayRef<std::string> SourcePaths,
> + std::shared_ptr<PCHContainerOperations> PCHContainerOps)
> + : Tool(Compilations, SourcePaths), Context(&Results),
> + ArgsAdjuster(getDefaultArgumentsAdjusters()) {
> + // Use self-defined default argument adjusters instead of the default
> + // adjusters that come with the old `ClangTool`.
> + Tool.clearArgumentsAdjusters();
> +}
> +
> +StandaloneToolExecutor::StandaloneToolExecutor(
> + CommonOptionsParser Options,
> + std::shared_ptr<PCHContainerOperations> PCHContainerOps)
> + : OptionsParser(std::move(Options)),
> + Tool(OptionsParser->getCompilations(), OptionsParser->getSourcePathList(),
> + PCHContainerOps),
> + Context(&Results), ArgsAdjuster(getDefaultArgumentsAdjusters()) {
> + Tool.clearArgumentsAdjusters();
> +}
> +
> +llvm::Error StandaloneToolExecutor::execute(
> + llvm::ArrayRef<
> + std::pair<std::unique_ptr<FrontendActionFactory>, ArgumentsAdjuster>>
> + Actions) {
> + if (Actions.empty())
> + return make_string_error("No action to execute.");
> +
> + if (Actions.size() != 1)
> + return make_string_error(
> + "Only support executing exactly 1 action at this point.");
> +
> + auto &Action = Actions.front();
> + Tool.appendArgumentsAdjuster(Action.second);
> + Tool.appendArgumentsAdjuster(ArgsAdjuster);
> + if (int Ret = Tool.run(Action.first.get()))
> + return make_string_error("Failed to run action.");
> +
> + return llvm::Error::success();
> +}
> +
> +class StandaloneToolExecutorPlugin : public ToolExecutorPlugin {
> +public:
> + llvm::Expected<std::unique_ptr<ToolExecutor>>
> + create(CommonOptionsParser &OptionsParser) override {
> + if (OptionsParser.getSourcePathList().empty())
> + return make_string_error(
> + "[StandaloneToolExecutorPlugin] No positional argument found.");
> + return llvm::make_unique<StandaloneToolExecutor>(std::move(OptionsParser));
> + }
> +};
> +
> +// This anchor is used to force the linker to link in the generated object file
> +// and thus register the plugin.
> +volatile int ToolExecutorPluginAnchorSource = 0;
> +
> +static ToolExecutorPluginRegistry::Add<StandaloneToolExecutorPlugin>
> + X("standalone", "Runs FrontendActions on a set of files provided "
> + "via positional arguments.");
> +
> +} // end namespace tooling
> +} // end namespace clang
>
> Modified: cfe/trunk/lib/Tooling/Tooling.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Tooling.cpp?rev=316653&r1=316652&r2=316653&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Tooling/Tooling.cpp (original)
> +++ cfe/trunk/lib/Tooling/Tooling.cpp Thu Oct 26 03:38:14 2017
> @@ -29,6 +29,7 @@
> #include "llvm/Config/llvm-config.h"
> #include "llvm/Option/ArgList.h"
> #include "llvm/Option/Option.h"
> +#include "llvm/Support/CommandLine.h"
> #include "llvm/Support/Debug.h"
> #include "llvm/Support/FileSystem.h"
> #include "llvm/Support/Host.h"
> @@ -347,11 +348,7 @@ void ClangTool::mapVirtualFile(StringRef
> }
>
> void ClangTool::appendArgumentsAdjuster(ArgumentsAdjuster Adjuster) {
> - if (ArgsAdjuster)
> - ArgsAdjuster =
> - combineAdjusters(std::move(ArgsAdjuster), std::move(Adjuster));
> - else
> - ArgsAdjuster = std::move(Adjuster);
> + ArgsAdjuster = combineAdjusters(std::move(ArgsAdjuster), std::move(Adjuster));
> }
>
> void ClangTool::clearArgumentsAdjusters() {
>
> Modified: cfe/trunk/unittests/Tooling/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff
> ==============================================================================
> --- cfe/trunk/unittests/Tooling/CMakeLists.txt (original)
> +++ cfe/trunk/unittests/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017
> @@ -16,6 +16,7 @@ add_clang_unittest(ToolingTests
> CommentHandlerTest.cpp
> CompilationDatabaseTest.cpp
> DiagnosticsYamlTest.cpp
> + ExecutionTest.cpp
> FixItTest.cpp
> LexicallyOrderedRecursiveASTVisitorTest.cpp
> LookupTest.cpp
>
> Added: cfe/trunk/unittests/Tooling/ExecutionTest.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/ExecutionTest.cpp?rev=316653&view=auto
> ==============================================================================
> --- cfe/trunk/unittests/Tooling/ExecutionTest.cpp (added)
> +++ cfe/trunk/unittests/Tooling/ExecutionTest.cpp Thu Oct 26 03:38:14 2017
> @@ -0,0 +1,228 @@
> +//===- unittest/Tooling/ExecutionTest.cpp - Tool execution tests. --------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "clang/AST/ASTConsumer.h"
> +#include "clang/AST/DeclCXX.h"
> +#include "clang/AST/RecursiveASTVisitor.h"
> +#include "clang/Frontend/ASTUnit.h"
> +#include "clang/Frontend/FrontendAction.h"
> +#include "clang/Frontend/FrontendActions.h"
> +#include "clang/Tooling/CompilationDatabase.h"
> +#include "clang/Tooling/Execution.h"
> +#include "clang/Tooling/StandaloneExecution.h"
> +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
> +#include "clang/Tooling/Tooling.h"
> +#include "gtest/gtest.h"
> +#include <algorithm>
> +#include <string>
> +
> +namespace clang {
> +namespace tooling {
> +
> +namespace {
> +
> +// This traverses the AST and outputs function name as key and "1" as value for
> +// each function declaration.
> +class ASTConsumerWithResult
> + : public ASTConsumer,
> + public RecursiveASTVisitor<ASTConsumerWithResult> {
> +public:
> + using ASTVisitor = RecursiveASTVisitor<ASTConsumerWithResult>;
> +
> + explicit ASTConsumerWithResult(ExecutionContext *Context) : Context(Context) {
> + assert(Context != nullptr);
> + }
> +
> + void HandleTranslationUnit(clang::ASTContext &Context) override {
> + TraverseDecl(Context.getTranslationUnitDecl());
> + }
> +
> + bool TraverseFunctionDecl(clang::FunctionDecl *Decl) {
> + Context->reportResult(Decl->getNameAsString(), "1");
> + return ASTVisitor::TraverseFunctionDecl(Decl);
> + }
> +
> +private:
> + ExecutionContext *const Context;
> +};
> +
> +class ReportResultAction : public ASTFrontendAction {
> +public:
> + explicit ReportResultAction(ExecutionContext *Context) : Context(Context) {
> + assert(Context != nullptr);
> + }
> +
> +protected:
> + std::unique_ptr<clang::ASTConsumer>
> + CreateASTConsumer(clang::CompilerInstance &compiler,
> + StringRef /* dummy */) override {
> + std::unique_ptr<clang::ASTConsumer> ast_consumer{
> + new ASTConsumerWithResult(Context)};
> + return ast_consumer;
> + }
> +
> +private:
> + ExecutionContext *const Context;
> +};
> +
> +class ReportResultActionFactory : public FrontendActionFactory {
> +public:
> + ReportResultActionFactory(ExecutionContext *Context) : Context(Context) {}
> + FrontendAction *create() override { return new ReportResultAction(Context); }
> +
> +private:
> + ExecutionContext *const Context;
> +};
> +
> +} // namespace
> +
> +class TestToolExecutor : public ToolExecutor {
> +public:
> + static const char *ExecutorName;
> +
> + TestToolExecutor(CommonOptionsParser Options)
> + : OptionsParser(std::move(Options)) {}
> +
> + StringRef getExecutorName() const override { return ExecutorName; }
> +
> + llvm::Error
> + execute(llvm::ArrayRef<std::pair<std::unique_ptr<FrontendActionFactory>,
> + ArgumentsAdjuster>>) override {
> + return llvm::Error::success();
> + }
> +
> + ExecutionContext *getExecutionContext() override { return nullptr; };
> +
> + ToolResults *getToolResults() override { return nullptr; }
> +
> + llvm::ArrayRef<std::string> getSourcePaths() const {
> + return OptionsParser.getSourcePathList();
> + }
> +
> + void mapVirtualFile(StringRef FilePath, StringRef Content) override {
> + VFS[FilePath] = Content;
> + }
> +
> +private:
> + CommonOptionsParser OptionsParser;
> + std::string SourcePaths;
> + std::map<std::string, std::string> VFS;
> +};
> +
> +const char *TestToolExecutor::ExecutorName = "test-executor";
> +
> +class TestToolExecutorPlugin : public ToolExecutorPlugin {
> +public:
> + llvm::Expected<std::unique_ptr<ToolExecutor>>
> + create(CommonOptionsParser &OptionsParser) override {
> + return llvm::make_unique<TestToolExecutor>(std::move(OptionsParser));
> + }
> +};
> +
> +// This anchor is used to force the linker to link in the generated object file
> +// and thus register the plugin.
> +extern volatile int ToolExecutorPluginAnchorSource;
> +
> +static int LLVM_ATTRIBUTE_UNUSED TestToolExecutorPluginAnchorDest =
> + ToolExecutorPluginAnchorSource;
> +
> +static ToolExecutorPluginRegistry::Add<TestToolExecutorPlugin>
> + X("test-executor", "Plugin for TestToolExecutor.");
> +
> +llvm::cl::OptionCategory TestCategory("execution-test options");
> +
> +TEST(CreateToolExecutorTest, FailedCreateExecutorUndefinedFlag) {
> + std::vector<const char *> argv = {"prog", "--fake_flag_no_no_no", "f"};
> + int argc = argv.size();
> + auto Executor =
> + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> + ASSERT_FALSE((bool)Executor);
> + llvm::consumeError(Executor.takeError());
> +}
> +
> +TEST(CreateToolExecutorTest, RegisterFlagsBeforeReset) {
> + llvm::cl::opt<std::string> BeforeReset(
> + "before_reset", llvm::cl::desc("Defined before reset."),
> + llvm::cl::init(""));
> +
> + llvm::cl::ResetAllOptionOccurrences();
> +
> + std::vector<const char *> argv = {"prog", "--before_reset=set", "f"};
> + int argc = argv.size();
> + auto Executor =
> + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> + ASSERT_TRUE((bool)Executor);
> + EXPECT_EQ(BeforeReset, "set");
> + BeforeReset.removeArgument();
> +}
> +
> +TEST(CreateToolExecutorTest, CreateStandaloneToolExecutor) {
> + std::vector<const char *> argv = {"prog", "standalone.cpp"};
> + int argc = argv.size();
> + auto Executor =
> + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> + ASSERT_TRUE((bool)Executor);
> + EXPECT_EQ(Executor->get()->getExecutorName(),
> + StandaloneToolExecutor::ExecutorName);
> +}
> +
> +TEST(CreateToolExecutorTest, CreateTestToolExecutor) {
> + std::vector<const char *> argv = {"prog", "test.cpp",
> + "--executor=test-executor"};
> + int argc = argv.size();
> + auto Executor =
> + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> + ASSERT_TRUE((bool)Executor);
> + EXPECT_EQ(Executor->get()->getExecutorName(), TestToolExecutor::ExecutorName);
> +}
> +
> +TEST(StandaloneToolTest, SynctaxOnlyActionOnSimpleCode) {
> + FixedCompilationDatabase Compilations("/", std::vector<std::string>());
> + StandaloneToolExecutor Executor(Compilations,
> + std::vector<std::string>(1, "/a.cc"));
> + Executor.mapVirtualFile("/a.cc", "int x = 0;");
> +
> + auto Err = Executor.execute(newFrontendActionFactory<SyntaxOnlyAction>(),
> + getClangSyntaxOnlyAdjuster());
> + ASSERT_TRUE(!Err);
> +}
> +
> +TEST(StandaloneToolTest, SimpleAction) {
> + FixedCompilationDatabase Compilations("/", std::vector<std::string>());
> + StandaloneToolExecutor Executor(Compilations,
> + std::vector<std::string>(1, "/a.cc"));
> + Executor.mapVirtualFile("/a.cc", "int x = 0;");
> +
> + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
> + new ReportResultActionFactory(Executor.getExecutionContext())));
> + ASSERT_TRUE(!Err);
> + auto KVs = Executor.getToolResults()->AllKVResults();
> + ASSERT_EQ(KVs.size(), 0u);
> +}
> +
> +TEST(StandaloneToolTest, SimpleActionWithResult) {
> + FixedCompilationDatabase Compilations("/", std::vector<std::string>());
> + StandaloneToolExecutor Executor(Compilations,
> + std::vector<std::string>(1, "/a.cc"));
> + Executor.mapVirtualFile("/a.cc", "int x = 0; void f() {}");
> +
> + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
> + new ReportResultActionFactory(Executor.getExecutionContext())));
> + ASSERT_TRUE(!Err);
> + auto KVs = Executor.getToolResults()->AllKVResults();
> + ASSERT_EQ(KVs.size(), 1u);
> + EXPECT_EQ("f", KVs[0].first);
> + EXPECT_EQ("1", KVs[0].second);
> +
> + Executor.getToolResults()->forEachResult(
> + [](StringRef, StringRef Value) { EXPECT_EQ("1", Value); });
> +}
> +
> +} // end namespace tooling
> +} // end namespace clang
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
More information about the cfe-commits
mailing list