r316653 - [Tooling] A new framework for executing clang frontend actions.
Eric Liu via cfe-commits
cfe-commits at lists.llvm.org
Thu Oct 26 06:16:01 PDT 2017
Hi Maxim,
Thanks for the email! I'll look into this right now.
Thanks,
Eric
On Thu, Oct 26, 2017 at 3:11 PM Maxim Kuvyrkov <maxim.kuvyrkov at linaro.org>
wrote:
> Hi Eric,
>
> This seems to have broken clang-cmake-thumbv7-a15 buildbot [*]. Would you
> please investigate?
>
> Or did you fix this while I was writing this email in 316661 :-) ?
>
> Thank you,
>
> [*] http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15/builds/12511
>
> --
> Maxim Kuvyrkov
> www.linaro.org
>
>
>
> > On Oct 26, 2017, at 1:38 PM, Eric Liu via cfe-commits <
> cfe-commits at lists.llvm.org> wrote:
> >
> > Author: ioeric
> > Date: Thu Oct 26 03:38:14 2017
> > New Revision: 316653
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=316653&view=rev
> > Log:
> > [Tooling] A new framework for executing clang frontend actions.
> >
> > Summary:
> > This defines a `clang::tooling::ToolExecutor` interface that can be
> extended to support different execution plans including standalone
> execution on a given set of TUs or parallel execution on all TUs in a
> codebase.
> >
> > In order to enable multiprocessing execution, tool actions are expected
> to output result into a `ToolResults` interface provided by executors. The
> `ToolResults` interface abstracts how results are stored e.g. in-memory for
> standalone executions or on-disk for large-scale execution.
> >
> > New executors can be registered as `ToolExecutorPlugin`s via the
> `ToolExecutorPluginRegistry`. CLI tools can use
> `createExecutorFromCommandLineArgs` to create a specific registered
> executor according to the command-line arguments.
> >
> > This patch also implements `StandaloneToolExecutor` which has the same
> behavior as the current `ClangTool` interface, i.e. execute frontend
> actions on a given set of TUs. At this point, it's simply a wrapper around
> `ClangTool` at this point.
> >
> > This is still experimental but expected to replace the existing
> `ClangTool` interface so that specific tools would not need to worry about
> execution.
> >
> > Reviewers: klimek, arphaman, hokein, sammccall
> >
> > Reviewed By: klimek
> >
> > Subscribers: cfe-commits, djasper, mgorny, omtcyfz
> >
> > Differential Revision: https://reviews.llvm.org/D34272
> >
> > Added:
> > cfe/trunk/include/clang/Tooling/Execution.h
> > cfe/trunk/include/clang/Tooling/StandaloneExecution.h
> > cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h
> > cfe/trunk/lib/Tooling/Execution.cpp
> > cfe/trunk/lib/Tooling/StandaloneExecution.cpp
> > cfe/trunk/unittests/Tooling/ExecutionTest.cpp
> > Modified:
> > cfe/trunk/include/clang/Tooling/CommonOptionsParser.h
> > cfe/trunk/include/clang/Tooling/Tooling.h
> > cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp
> > cfe/trunk/lib/Tooling/CMakeLists.txt
> > cfe/trunk/lib/Tooling/CommonOptionsParser.cpp
> > cfe/trunk/lib/Tooling/Tooling.cpp
> > cfe/trunk/unittests/Tooling/CMakeLists.txt
> >
> > Modified: cfe/trunk/include/clang/Tooling/CommonOptionsParser.h
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/CommonOptionsParser.h?rev=316653&r1=316652&r2=316653&view=diff
> >
> ==============================================================================
> > --- cfe/trunk/include/clang/Tooling/CommonOptionsParser.h (original)
> > +++ cfe/trunk/include/clang/Tooling/CommonOptionsParser.h Thu Oct 26
> 03:38:14 2017
> > @@ -109,6 +109,10 @@ public:
> > return SourcePathList;
> > }
> >
> > + /// Returns the argument adjuster calculated from "--extra-arg" and
> > + //"--extra-arg-before" options.
> > + ArgumentsAdjuster getArgumentsAdjuster() { return Adjuster; }
> > +
> > static const char *const HelpMessage;
> >
> > private:
> > @@ -121,6 +125,7 @@ private:
> >
> > std::unique_ptr<CompilationDatabase> Compilations;
> > std::vector<std::string> SourcePathList;
> > + ArgumentsAdjuster Adjuster;
> > };
> >
> > class ArgumentsAdjustingCompilations : public CompilationDatabase {
> >
> > Added: cfe/trunk/include/clang/Tooling/Execution.h
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Execution.h?rev=316653&view=auto
> >
> ==============================================================================
> > --- cfe/trunk/include/clang/Tooling/Execution.h (added)
> > +++ cfe/trunk/include/clang/Tooling/Execution.h Thu Oct 26 03:38:14 2017
> > @@ -0,0 +1,168 @@
> > +//===--- Execution.h - Executing clang frontend actions -*- C++
> ---------*-===//
> > +//
> > +// The LLVM Compiler Infrastructure
> > +//
> > +// This file is distributed under the University of Illinois Open Source
> > +// License. See LICENSE.TXT for details.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +//
> > +// This file defines framework for executing clang frontend actions.
> > +//
> > +// The framework can be extended to support different execution plans
> including
> > +// standalone execution on the given TUs or parallel execution on all
> TUs in
> > +// the codebase.
> > +//
> > +// In order to enable multiprocessing execution, tool actions are
> expected to
> > +// output result into the ToolResults provided by the executor. The
> > +// `ToolResults` is an interface that abstracts how results are stored
> e.g.
> > +// in-memory for standalone execution or on-disk for large-scale
> execution.
> > +//
> > +// New executors can be registered as ToolExecutorPlugins via the
> > +// `ToolExecutorPluginRegistry`. CLI tools can use
> > +// `createExecutorFromCommandLineArgs` to create a specific registered
> executor
> > +// according to the command-line arguments.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +
> > +#ifndef LLVM_CLANG_TOOLING_EXECUTION_H
> > +#define LLVM_CLANG_TOOLING_EXECUTION_H
> > +
> > +#include "clang/Tooling/CommonOptionsParser.h"
> > +#include "clang/Tooling/Tooling.h"
> > +#include "llvm/Support/Error.h"
> > +#include "llvm/Support/Registry.h"
> > +
> > +namespace clang {
> > +namespace tooling {
> > +
> > +/// \brief An abstraction for the result of a tool execution. For
> example, the
> > +/// underlying result can be in-memory or on-disk.
> > +///
> > +/// Results should be string key-value pairs. For example, a
> refactoring tool
> > +/// can use source location as key and a replacement in YAML format as
> value.
> > +class ToolResults {
> > +public:
> > + virtual ~ToolResults() = default;
> > + virtual void addResult(StringRef Key, StringRef Value) = 0;
> > + virtual std::vector<std::pair<std::string, std::string>>
> AllKVResults() = 0;
> > + virtual void forEachResult(
> > + llvm::function_ref<void(StringRef Key, StringRef Value)>
> Callback) = 0;
> > +};
> > +
> > +class InMemoryToolResults : public ToolResults {
> > +public:
> > + void addResult(StringRef Key, StringRef Value) override;
> > + std::vector<std::pair<std::string, std::string>> AllKVResults()
> override;
> > + void forEachResult(llvm::function_ref<void(StringRef Key, StringRef
> Value)>
> > + Callback) override;
> > +
> > +private:
> > + std::vector<std::pair<std::string, std::string>> KVResults;
> > +};
> > +
> > +/// \brief The context of an execution, including the information about
> > +/// compilation and results.
> > +class ExecutionContext {
> > +public:
> > + virtual ~ExecutionContext() {}
> > +
> > + /// \brief Initializes a context. This does not take ownership of
> `Results`.
> > + explicit ExecutionContext(ToolResults *Results) : Results(Results) {}
> > +
> > + /// \brief Adds a KV pair to the result container of this execution.
> > + void reportResult(StringRef Key, StringRef Value);
> > +
> > + // Returns the source control system's revision number if applicable.
> > + // Otherwise returns an empty string.
> > + virtual std::string getRevision() { return ""; }
> > +
> > + // Returns the corpus being analyzed, e.g. "llvm" for the LLVM
> codebase, if
> > + // applicable.
> > + virtual std::string getCorpus() { return ""; }
> > +
> > + // Returns the currently processed compilation unit if available.
> > + virtual std::string getCurrentCompilationUnit() { return ""; }
> > +
> > +private:
> > + ToolResults *Results;
> > +};
> > +
> > +/// \brief Interface for executing clang frontend actions.
> > +///
> > +/// This can be extended to support running tool actions in different
> > +/// execution mode, e.g. on a specific set of TUs or many TUs in
> parallel.
> > +///
> > +/// New executors can be registered as ToolExecutorPlugins via the
> > +/// `ToolExecutorPluginRegistry`. CLI tools can use
> > +/// `createExecutorFromCommandLineArgs` to create a specific registered
> > +/// executor according to the command-line arguments.
> > +class ToolExecutor {
> > +public:
> > + virtual ~ToolExecutor() {}
> > +
> > + /// \brief Returns the name of a specific executor.
> > + virtual StringRef getExecutorName() const = 0;
> > +
> > + /// \brief Executes each action with a corresponding arguments
> adjuster.
> > + virtual llvm::Error
> > + execute(llvm::ArrayRef<
> > + std::pair<std::unique_ptr<FrontendActionFactory>,
> ArgumentsAdjuster>>
> > + Actions) = 0;
> > +
> > + /// \brief Convenient functions for the above `execute`.
> > + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action);
> > + /// Executes an action with an argument adjuster.
> > + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action,
> > + ArgumentsAdjuster Adjuster);
> > +
> > + /// \brief Returns a reference to the execution context.
> > + ///
> > + /// This should be passed to tool callbacks, and tool callbacks
> should report
> > + /// results via the returned context.
> > + virtual ExecutionContext *getExecutionContext() = 0;
> > +
> > + /// \brief Returns a reference to the result container.
> > + ///
> > + /// NOTE: This should only be used after the execution finishes. Tool
> > + /// callbacks should report results via `ExecutionContext` instead.
> > + virtual ToolResults *getToolResults() = 0;
> > +
> > + /// \brief Map a virtual file to be used while running the tool.
> > + ///
> > + /// \param FilePath The path at which the content will be mapped.
> > + /// \param Content A buffer of the file's content.
> > + virtual void mapVirtualFile(StringRef FilePath, StringRef Content) =
> 0;
> > +};
> > +
> > +/// \brief Interface for factories that create specific executors. This
> is also
> > +/// used as a plugin to be registered into ToolExecutorPluginRegistry.
> > +class ToolExecutorPlugin {
> > +public:
> > + virtual ~ToolExecutorPlugin() {}
> > +
> > + /// \brief Create an `ToolExecutor`.
> > + ///
> > + /// `OptionsParser` can be consumed (e.g. moved) if the creation
> succeeds.
> > + virtual llvm::Expected<std::unique_ptr<ToolExecutor>>
> > + create(CommonOptionsParser &OptionsParser) = 0;
> > +};
> > +
> > +/// \brief This creates a ToolExecutor that is in the global registry
> based on
> > +/// commandline arguments.
> > +///
> > +/// This picks the right executor based on the `--executor` option.
> This parses
> > +/// the commandline arguments with `CommonOptionsParser`, so caller
> does not
> > +/// need to parse again.
> > +///
> > +/// By default, this creates a `StandaloneToolExecutor` ("standalone")
> if
> > +/// `--executor` is not provided.
> > +llvm::Expected<std::unique_ptr<ToolExecutor>>
> > +createExecutorFromCommandLineArgs(int &argc, const char **argv,
> > + llvm::cl::OptionCategory &Category,
> > + const char *Overview = nullptr);
> > +
> > +} // end namespace tooling
> > +} // end namespace clang
> > +
> > +#endif // LLVM_CLANG_TOOLING_EXECUTION_H
> >
> > Added: cfe/trunk/include/clang/Tooling/StandaloneExecution.h
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/StandaloneExecution.h?rev=316653&view=auto
> >
> ==============================================================================
> > --- cfe/trunk/include/clang/Tooling/StandaloneExecution.h (added)
> > +++ cfe/trunk/include/clang/Tooling/StandaloneExecution.h Thu Oct 26
> 03:38:14 2017
> > @@ -0,0 +1,97 @@
> > +//===--- StandaloneExecution.h - Standalone execution. -*- C++
> ----------*-===//
> > +//
> > +// The LLVM Compiler Infrastructure
> > +//
> > +// This file is distributed under the University of Illinois Open Source
> > +// License. See LICENSE.TXT for details.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +//
> > +// This file defines standalone execution of clang tools.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +
> > +#ifndef LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
> > +#define LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
> > +
> > +#include "clang/Tooling/ArgumentsAdjusters.h"
> > +#include "clang/Tooling/Execution.h"
> > +
> > +namespace clang {
> > +namespace tooling {
> > +
> > +/// \brief A standalone executor that runs FrontendActions on a given
> set of
> > +/// TUs in sequence.
> > +///
> > +/// By default, this executor uses the following arguments adjusters
> (as defined
> > +/// in `clang/Tooling/ArgumentsAdjusters.h`):
> > +/// - `getClangStripOutputAdjuster()`
> > +/// - `getClangSyntaxOnlyAdjuster()`
> > +/// - `getClangStripDependencyFileAdjuster()`
> > +class StandaloneToolExecutor : public ToolExecutor {
> > +public:
> > + static const char *ExecutorName;
> > +
> > + /// \brief Init with \p CompilationDatabase and the paths of all
> files to be
> > + /// proccessed.
> > + StandaloneToolExecutor(
> > + const CompilationDatabase &Compilations,
> > + llvm::ArrayRef<std::string> SourcePaths,
> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
> > + std::make_shared<PCHContainerOperations>());
> > +
> > + /// \brief Init with \p CommonOptionsParser. This is expected to be
> used by
> > + /// `createExecutorFromCommandLineArgs` based on commandline options.
> > + ///
> > + /// The executor takes ownership of \p Options.
> > + StandaloneToolExecutor(
> > + CommonOptionsParser Options,
> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
> > + std::make_shared<PCHContainerOperations>());
> > +
> > + StringRef getExecutorName() const override { return ExecutorName; }
> > +
> > + using ToolExecutor::execute;
> > +
> > + llvm::Error
> > + execute(llvm::ArrayRef<
> > + std::pair<std::unique_ptr<FrontendActionFactory>,
> ArgumentsAdjuster>>
> > + Actions) override;
> > +
> > + /// \brief Set a \c DiagnosticConsumer to use during parsing.
> > + void setDiagnosticConsumer(DiagnosticConsumer *DiagConsumer) {
> > + Tool.setDiagnosticConsumer(DiagConsumer);
> > + }
> > +
> > + ExecutionContext *getExecutionContext() override { return &Context; };
> > +
> > + ToolResults *getToolResults() override { return &Results; }
> > +
> > + llvm::ArrayRef<std::string> getSourcePaths() const {
> > + return Tool.getSourcePaths();
> > + }
> > +
> > + void mapVirtualFile(StringRef FilePath, StringRef Content) override {
> > + Tool.mapVirtualFile(FilePath, Content);
> > + }
> > +
> > + /// \brief Returns the file manager used in the tool.
> > + ///
> > + /// The file manager is shared between all translation units.
> > + FileManager &getFiles() { return Tool.getFiles(); }
> > +
> > +private:
> > + // Used to store the parser when the executor is initialized with
> parser.
> > + llvm::Optional<CommonOptionsParser> OptionsParser;
> > + // FIXME: The standalone executor is currently just a wrapper of
> `ClangTool`.
> > + // Merge `ClangTool` implementation into the this.
> > + ClangTool Tool;
> > + ExecutionContext Context;
> > + InMemoryToolResults Results;
> > + ArgumentsAdjuster ArgsAdjuster;
> > +};
> > +
> > +} // end namespace tooling
> > +} // end namespace clang
> > +
> > +#endif // LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
> >
> > Added: cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h?rev=316653&view=auto
> >
> ==============================================================================
> > --- cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h (added)
> > +++ cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h Thu Oct
> 26 03:38:14 2017
> > @@ -0,0 +1,24 @@
> > +//===--- ToolExecutorPluginRegistry.h - -------------------------*- C++
> -*-===//
> > +//
> > +// The LLVM Compiler Infrastructure
> > +//
> > +// This file is distributed under the University of Illinois Open Source
> > +// License. See LICENSE.TXT for details.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +
> > +#ifndef LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
> > +#define LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
> > +
> > +#include "clang/Tooling/Execution.h"
> > +#include "llvm/Support/Registry.h"
> > +
> > +namespace clang {
> > +namespace tooling {
> > +
> > +typedef llvm::Registry<ToolExecutorPlugin> ToolExecutorPluginRegistry;
> > +
> > +} // end namespace tooling
> > +} // end namespace clang
> > +
> > +#endif // LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
> >
> > Modified: cfe/trunk/include/clang/Tooling/Tooling.h
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Tooling.h?rev=316653&r1=316652&r2=316653&view=diff
> >
> ==============================================================================
> > --- cfe/trunk/include/clang/Tooling/Tooling.h (original)
> > +++ cfe/trunk/include/clang/Tooling/Tooling.h Thu Oct 26 03:38:14 2017
> > @@ -31,12 +31,12 @@
> > #define LLVM_CLANG_TOOLING_TOOLING_H
> >
> > #include "clang/AST/ASTConsumer.h"
> > -#include "clang/Frontend/PCHContainerOperations.h"
> > #include "clang/Basic/Diagnostic.h"
> > #include "clang/Basic/FileManager.h"
> > #include "clang/Basic/LLVM.h"
> > #include "clang/Driver/Util.h"
> > #include "clang/Frontend/FrontendAction.h"
> > +#include "clang/Frontend/PCHContainerOperations.h"
> > #include "clang/Lex/ModuleLoader.h"
> > #include "clang/Tooling/ArgumentsAdjusters.h"
> > #include "clang/Tooling/CompilationDatabase.h"
> > @@ -337,7 +337,9 @@ class ClangTool {
> > /// The file manager is shared between all translation units.
> > FileManager &getFiles() { return *Files; }
> >
> > - private:
> > + llvm::ArrayRef<std::string> getSourcePaths() const { return
> SourcePaths; }
> > +
> > +private:
> > const CompilationDatabase &Compilations;
> > std::vector<std::string> SourcePaths;
> > std::shared_ptr<PCHContainerOperations> PCHContainerOps;
> >
> > Modified: cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp?rev=316653&r1=316652&r2=316653&view=diff
> >
> ==============================================================================
> > --- cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp (original)
> > +++ cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp Thu Oct 26 03:38:14 2017
> > @@ -96,6 +96,10 @@ ArgumentsAdjuster getInsertArgumentAdjus
> >
> > ArgumentsAdjuster combineAdjusters(ArgumentsAdjuster First,
> > ArgumentsAdjuster Second) {
> > + if (!First)
> > + return Second;
> > + if (!Second)
> > + return First;
> > return [First, Second](const CommandLineArguments &Args, StringRef
> File) {
> > return Second(First(Args, File), File);
> > };
> >
> > Modified: cfe/trunk/lib/Tooling/CMakeLists.txt
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff
> >
> ==============================================================================
> > --- cfe/trunk/lib/Tooling/CMakeLists.txt (original)
> > +++ cfe/trunk/lib/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017
> > @@ -11,11 +11,13 @@ add_clang_library(clangTooling
> > ArgumentsAdjusters.cpp
> > CommonOptionsParser.cpp
> > CompilationDatabase.cpp
> > + Execution.cpp
> > FileMatchTrie.cpp
> > FixIt.cpp
> > JSONCompilationDatabase.cpp
> > Refactoring.cpp
> > RefactoringCallbacks.cpp
> > + StandaloneExecution.cpp
> > Tooling.cpp
> >
> > DEPENDS
> >
> > Modified: cfe/trunk/lib/Tooling/CommonOptionsParser.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CommonOptionsParser.cpp?rev=316653&r1=316652&r2=316653&view=diff
> >
> ==============================================================================
> > --- cfe/trunk/lib/Tooling/CommonOptionsParser.cpp (original)
> > +++ cfe/trunk/lib/Tooling/CommonOptionsParser.cpp Thu Oct 26 03:38:14
> 2017
> > @@ -147,10 +147,12 @@ llvm::Error CommonOptionsParser::init(
> > auto AdjustingCompilations =
> > llvm::make_unique<ArgumentsAdjustingCompilations>(
> > std::move(Compilations));
> > - AdjustingCompilations->appendArgumentsAdjuster(
> > - getInsertArgumentAdjuster(ArgsBefore,
> ArgumentInsertPosition::BEGIN));
> > - AdjustingCompilations->appendArgumentsAdjuster(
> > + Adjuster =
> > + getInsertArgumentAdjuster(ArgsBefore,
> ArgumentInsertPosition::BEGIN);
> > + Adjuster = combineAdjusters(
> > + std::move(Adjuster),
> > getInsertArgumentAdjuster(ArgsAfter, ArgumentInsertPosition::END));
> > + AdjustingCompilations->appendArgumentsAdjuster(Adjuster);
> > Compilations = std::move(AdjustingCompilations);
> > return llvm::Error::success();
> > }
> >
> > Added: cfe/trunk/lib/Tooling/Execution.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Execution.cpp?rev=316653&view=auto
> >
> ==============================================================================
> > --- cfe/trunk/lib/Tooling/Execution.cpp (added)
> > +++ cfe/trunk/lib/Tooling/Execution.cpp Thu Oct 26 03:38:14 2017
> > @@ -0,0 +1,89 @@
> > +//===- lib/Tooling/Execution.cpp - Implements tool execution framework.
> ---===//
> > +//
> > +// The LLVM Compiler Infrastructure
> > +//
> > +// This file is distributed under the University of Illinois Open Source
> > +// License. See LICENSE.TXT for details.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +
> > +#include "clang/Tooling/Execution.h"
> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
> > +#include "clang/Tooling/Tooling.h"
> > +
> > +LLVM_INSTANTIATE_REGISTRY(clang::tooling::ToolExecutorPluginRegistry)
> > +
> > +namespace clang {
> > +namespace tooling {
> > +
> > +static llvm::cl::opt<std::string>
> > + ExecutorName("executor", llvm::cl::desc("The name of the executor
> to use."),
> > + llvm::cl::init("standalone"));
> > +
> > +void InMemoryToolResults::addResult(StringRef Key, StringRef Value) {
> > + KVResults.push_back({Key.str(), Value.str()});
> > +}
> > +
> > +std::vector<std::pair<std::string, std::string>>
> > +InMemoryToolResults::AllKVResults() {
> > + return KVResults;
> > +}
> > +
> > +void InMemoryToolResults::forEachResult(
> > + llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) {
> > + for (const auto &KV : KVResults) {
> > + Callback(KV.first, KV.second);
> > + }
> > +}
> > +
> > +void ExecutionContext::reportResult(StringRef Key, StringRef Value) {
> > + Results->addResult(Key, Value);
> > +}
> > +
> > +llvm::Error
> > +ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action) {
> > + return execute(std::move(Action), ArgumentsAdjuster());
> > +}
> > +
> > +llvm::Error
> ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action,
> > + ArgumentsAdjuster Adjuster) {
> > + std::vector<
> > + std::pair<std::unique_ptr<FrontendActionFactory>,
> ArgumentsAdjuster>>
> > + Actions;
> > + Actions.emplace_back(std::move(Action), std::move(Adjuster));
> > + return execute(Actions);
> > +}
> > +
> > +llvm::Expected<std::unique_ptr<ToolExecutor>>
> > +createExecutorFromCommandLineArgs(int &argc, const char **argv,
> > + llvm::cl::OptionCategory &Category,
> > + const char *Overview) {
> > + auto OptionsParser =
> > + CommonOptionsParser::create(argc, argv, Category,
> llvm::cl::ZeroOrMore,
> > + /*Overview=*/nullptr);
> > + if (!OptionsParser)
> > + return OptionsParser.takeError();
> > + for (auto I = ToolExecutorPluginRegistry::begin(),
> > + E = ToolExecutorPluginRegistry::end();
> > + I != E; ++I) {
> > + if (I->getName() != ExecutorName) {
> > + continue;
> > + }
> > + std::unique_ptr<ToolExecutorPlugin> Plugin(I->instantiate());
> > + llvm::Expected<std::unique_ptr<ToolExecutor>> Executor =
> > + Plugin->create(*OptionsParser);
> > + if (!Executor) {
> > + return llvm::make_error<llvm::StringError>(
> > + llvm::Twine("Failed to create '") + I->getName() +
> > + "': " + llvm::toString(Executor.takeError()) + "\n",
> > + llvm::inconvertibleErrorCode());
> > + }
> > + return std::move(*Executor);
> > + }
> > + return llvm::make_error<llvm::StringError>(
> > + llvm::Twine("Executor \"") + ExecutorName + "\" is not
> registered.",
> > + llvm::inconvertibleErrorCode());
> > +}
> > +
> > +} // end namespace tooling
> > +} // end namespace clang
> >
> > Added: cfe/trunk/lib/Tooling/StandaloneExecution.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/StandaloneExecution.cpp?rev=316653&view=auto
> >
> ==============================================================================
> > --- cfe/trunk/lib/Tooling/StandaloneExecution.cpp (added)
> > +++ cfe/trunk/lib/Tooling/StandaloneExecution.cpp Thu Oct 26 03:38:14
> 2017
> > @@ -0,0 +1,91 @@
> > +//===- lib/Tooling/Execution.cpp - Standalone clang action execution.
> -----===//
> > +//
> > +// The LLVM Compiler Infrastructure
> > +//
> > +// This file is distributed under the University of Illinois Open Source
> > +// License. See LICENSE.TXT for details.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +
> > +#include "clang/Tooling/StandaloneExecution.h"
> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
> > +
> > +namespace clang {
> > +namespace tooling {
> > +
> > +static llvm::Error make_string_error(const llvm::Twine &Message) {
> > + return llvm::make_error<llvm::StringError>(Message,
> > +
> llvm::inconvertibleErrorCode());
> > +}
> > +
> > +const char *StandaloneToolExecutor::ExecutorName =
> "StandaloneToolExecutor";
> > +
> > +static ArgumentsAdjuster getDefaultArgumentsAdjusters() {
> > + return combineAdjusters(
> > + getClangStripOutputAdjuster(),
> > + combineAdjusters(getClangSyntaxOnlyAdjuster(),
> > + getClangStripDependencyFileAdjuster()));
> > +}
> > +
> > +StandaloneToolExecutor::StandaloneToolExecutor(
> > + const CompilationDatabase &Compilations,
> > + llvm::ArrayRef<std::string> SourcePaths,
> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps)
> > + : Tool(Compilations, SourcePaths), Context(&Results),
> > + ArgsAdjuster(getDefaultArgumentsAdjusters()) {
> > + // Use self-defined default argument adjusters instead of the default
> > + // adjusters that come with the old `ClangTool`.
> > + Tool.clearArgumentsAdjusters();
> > +}
> > +
> > +StandaloneToolExecutor::StandaloneToolExecutor(
> > + CommonOptionsParser Options,
> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps)
> > + : OptionsParser(std::move(Options)),
> > + Tool(OptionsParser->getCompilations(),
> OptionsParser->getSourcePathList(),
> > + PCHContainerOps),
> > + Context(&Results), ArgsAdjuster(getDefaultArgumentsAdjusters()) {
> > + Tool.clearArgumentsAdjusters();
> > +}
> > +
> > +llvm::Error StandaloneToolExecutor::execute(
> > + llvm::ArrayRef<
> > + std::pair<std::unique_ptr<FrontendActionFactory>,
> ArgumentsAdjuster>>
> > + Actions) {
> > + if (Actions.empty())
> > + return make_string_error("No action to execute.");
> > +
> > + if (Actions.size() != 1)
> > + return make_string_error(
> > + "Only support executing exactly 1 action at this point.");
> > +
> > + auto &Action = Actions.front();
> > + Tool.appendArgumentsAdjuster(Action.second);
> > + Tool.appendArgumentsAdjuster(ArgsAdjuster);
> > + if (int Ret = Tool.run(Action.first.get()))
> > + return make_string_error("Failed to run action.");
> > +
> > + return llvm::Error::success();
> > +}
> > +
> > +class StandaloneToolExecutorPlugin : public ToolExecutorPlugin {
> > +public:
> > + llvm::Expected<std::unique_ptr<ToolExecutor>>
> > + create(CommonOptionsParser &OptionsParser) override {
> > + if (OptionsParser.getSourcePathList().empty())
> > + return make_string_error(
> > + "[StandaloneToolExecutorPlugin] No positional argument
> found.");
> > + return
> llvm::make_unique<StandaloneToolExecutor>(std::move(OptionsParser));
> > + }
> > +};
> > +
> > +// This anchor is used to force the linker to link in the generated
> object file
> > +// and thus register the plugin.
> > +volatile int ToolExecutorPluginAnchorSource = 0;
> > +
> > +static ToolExecutorPluginRegistry::Add<StandaloneToolExecutorPlugin>
> > + X("standalone", "Runs FrontendActions on a set of files provided "
> > + "via positional arguments.");
> > +
> > +} // end namespace tooling
> > +} // end namespace clang
> >
> > Modified: cfe/trunk/lib/Tooling/Tooling.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Tooling.cpp?rev=316653&r1=316652&r2=316653&view=diff
> >
> ==============================================================================
> > --- cfe/trunk/lib/Tooling/Tooling.cpp (original)
> > +++ cfe/trunk/lib/Tooling/Tooling.cpp Thu Oct 26 03:38:14 2017
> > @@ -29,6 +29,7 @@
> > #include "llvm/Config/llvm-config.h"
> > #include "llvm/Option/ArgList.h"
> > #include "llvm/Option/Option.h"
> > +#include "llvm/Support/CommandLine.h"
> > #include "llvm/Support/Debug.h"
> > #include "llvm/Support/FileSystem.h"
> > #include "llvm/Support/Host.h"
> > @@ -347,11 +348,7 @@ void ClangTool::mapVirtualFile(StringRef
> > }
> >
> > void ClangTool::appendArgumentsAdjuster(ArgumentsAdjuster Adjuster) {
> > - if (ArgsAdjuster)
> > - ArgsAdjuster =
> > - combineAdjusters(std::move(ArgsAdjuster), std::move(Adjuster));
> > - else
> > - ArgsAdjuster = std::move(Adjuster);
> > + ArgsAdjuster = combineAdjusters(std::move(ArgsAdjuster),
> std::move(Adjuster));
> > }
> >
> > void ClangTool::clearArgumentsAdjusters() {
> >
> > Modified: cfe/trunk/unittests/Tooling/CMakeLists.txt
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff
> >
> ==============================================================================
> > --- cfe/trunk/unittests/Tooling/CMakeLists.txt (original)
> > +++ cfe/trunk/unittests/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017
> > @@ -16,6 +16,7 @@ add_clang_unittest(ToolingTests
> > CommentHandlerTest.cpp
> > CompilationDatabaseTest.cpp
> > DiagnosticsYamlTest.cpp
> > + ExecutionTest.cpp
> > FixItTest.cpp
> > LexicallyOrderedRecursiveASTVisitorTest.cpp
> > LookupTest.cpp
> >
> > Added: cfe/trunk/unittests/Tooling/ExecutionTest.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/ExecutionTest.cpp?rev=316653&view=auto
> >
> ==============================================================================
> > --- cfe/trunk/unittests/Tooling/ExecutionTest.cpp (added)
> > +++ cfe/trunk/unittests/Tooling/ExecutionTest.cpp Thu Oct 26 03:38:14
> 2017
> > @@ -0,0 +1,228 @@
> > +//===- unittest/Tooling/ExecutionTest.cpp - Tool execution tests.
> --------===//
> > +//
> > +// The LLVM Compiler Infrastructure
> > +//
> > +// This file is distributed under the University of Illinois Open Source
> > +// License. See LICENSE.TXT for details.
> > +//
> >
> +//===----------------------------------------------------------------------===//
> > +
> > +#include "clang/AST/ASTConsumer.h"
> > +#include "clang/AST/DeclCXX.h"
> > +#include "clang/AST/RecursiveASTVisitor.h"
> > +#include "clang/Frontend/ASTUnit.h"
> > +#include "clang/Frontend/FrontendAction.h"
> > +#include "clang/Frontend/FrontendActions.h"
> > +#include "clang/Tooling/CompilationDatabase.h"
> > +#include "clang/Tooling/Execution.h"
> > +#include "clang/Tooling/StandaloneExecution.h"
> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
> > +#include "clang/Tooling/Tooling.h"
> > +#include "gtest/gtest.h"
> > +#include <algorithm>
> > +#include <string>
> > +
> > +namespace clang {
> > +namespace tooling {
> > +
> > +namespace {
> > +
> > +// This traverses the AST and outputs function name as key and "1" as
> value for
> > +// each function declaration.
> > +class ASTConsumerWithResult
> > + : public ASTConsumer,
> > + public RecursiveASTVisitor<ASTConsumerWithResult> {
> > +public:
> > + using ASTVisitor = RecursiveASTVisitor<ASTConsumerWithResult>;
> > +
> > + explicit ASTConsumerWithResult(ExecutionContext *Context) :
> Context(Context) {
> > + assert(Context != nullptr);
> > + }
> > +
> > + void HandleTranslationUnit(clang::ASTContext &Context) override {
> > + TraverseDecl(Context.getTranslationUnitDecl());
> > + }
> > +
> > + bool TraverseFunctionDecl(clang::FunctionDecl *Decl) {
> > + Context->reportResult(Decl->getNameAsString(), "1");
> > + return ASTVisitor::TraverseFunctionDecl(Decl);
> > + }
> > +
> > +private:
> > + ExecutionContext *const Context;
> > +};
> > +
> > +class ReportResultAction : public ASTFrontendAction {
> > +public:
> > + explicit ReportResultAction(ExecutionContext *Context) :
> Context(Context) {
> > + assert(Context != nullptr);
> > + }
> > +
> > +protected:
> > + std::unique_ptr<clang::ASTConsumer>
> > + CreateASTConsumer(clang::CompilerInstance &compiler,
> > + StringRef /* dummy */) override {
> > + std::unique_ptr<clang::ASTConsumer> ast_consumer{
> > + new ASTConsumerWithResult(Context)};
> > + return ast_consumer;
> > + }
> > +
> > +private:
> > + ExecutionContext *const Context;
> > +};
> > +
> > +class ReportResultActionFactory : public FrontendActionFactory {
> > +public:
> > + ReportResultActionFactory(ExecutionContext *Context) :
> Context(Context) {}
> > + FrontendAction *create() override { return new
> ReportResultAction(Context); }
> > +
> > +private:
> > + ExecutionContext *const Context;
> > +};
> > +
> > +} // namespace
> > +
> > +class TestToolExecutor : public ToolExecutor {
> > +public:
> > + static const char *ExecutorName;
> > +
> > + TestToolExecutor(CommonOptionsParser Options)
> > + : OptionsParser(std::move(Options)) {}
> > +
> > + StringRef getExecutorName() const override { return ExecutorName; }
> > +
> > + llvm::Error
> > +
> execute(llvm::ArrayRef<std::pair<std::unique_ptr<FrontendActionFactory>,
> > + ArgumentsAdjuster>>) override {
> > + return llvm::Error::success();
> > + }
> > +
> > + ExecutionContext *getExecutionContext() override { return nullptr; };
> > +
> > + ToolResults *getToolResults() override { return nullptr; }
> > +
> > + llvm::ArrayRef<std::string> getSourcePaths() const {
> > + return OptionsParser.getSourcePathList();
> > + }
> > +
> > + void mapVirtualFile(StringRef FilePath, StringRef Content) override {
> > + VFS[FilePath] = Content;
> > + }
> > +
> > +private:
> > + CommonOptionsParser OptionsParser;
> > + std::string SourcePaths;
> > + std::map<std::string, std::string> VFS;
> > +};
> > +
> > +const char *TestToolExecutor::ExecutorName = "test-executor";
> > +
> > +class TestToolExecutorPlugin : public ToolExecutorPlugin {
> > +public:
> > + llvm::Expected<std::unique_ptr<ToolExecutor>>
> > + create(CommonOptionsParser &OptionsParser) override {
> > + return
> llvm::make_unique<TestToolExecutor>(std::move(OptionsParser));
> > + }
> > +};
> > +
> > +// This anchor is used to force the linker to link in the generated
> object file
> > +// and thus register the plugin.
> > +extern volatile int ToolExecutorPluginAnchorSource;
> > +
> > +static int LLVM_ATTRIBUTE_UNUSED TestToolExecutorPluginAnchorDest =
> > + ToolExecutorPluginAnchorSource;
> > +
> > +static ToolExecutorPluginRegistry::Add<TestToolExecutorPlugin>
> > + X("test-executor", "Plugin for TestToolExecutor.");
> > +
> > +llvm::cl::OptionCategory TestCategory("execution-test options");
> > +
> > +TEST(CreateToolExecutorTest, FailedCreateExecutorUndefinedFlag) {
> > + std::vector<const char *> argv = {"prog", "--fake_flag_no_no_no",
> "f"};
> > + int argc = argv.size();
> > + auto Executor =
> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> > + ASSERT_FALSE((bool)Executor);
> > + llvm::consumeError(Executor.takeError());
> > +}
> > +
> > +TEST(CreateToolExecutorTest, RegisterFlagsBeforeReset) {
> > + llvm::cl::opt<std::string> BeforeReset(
> > + "before_reset", llvm::cl::desc("Defined before reset."),
> > + llvm::cl::init(""));
> > +
> > + llvm::cl::ResetAllOptionOccurrences();
> > +
> > + std::vector<const char *> argv = {"prog", "--before_reset=set", "f"};
> > + int argc = argv.size();
> > + auto Executor =
> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> > + ASSERT_TRUE((bool)Executor);
> > + EXPECT_EQ(BeforeReset, "set");
> > + BeforeReset.removeArgument();
> > +}
> > +
> > +TEST(CreateToolExecutorTest, CreateStandaloneToolExecutor) {
> > + std::vector<const char *> argv = {"prog", "standalone.cpp"};
> > + int argc = argv.size();
> > + auto Executor =
> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> > + ASSERT_TRUE((bool)Executor);
> > + EXPECT_EQ(Executor->get()->getExecutorName(),
> > + StandaloneToolExecutor::ExecutorName);
> > +}
> > +
> > +TEST(CreateToolExecutorTest, CreateTestToolExecutor) {
> > + std::vector<const char *> argv = {"prog", "test.cpp",
> > + "--executor=test-executor"};
> > + int argc = argv.size();
> > + auto Executor =
> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
> > + ASSERT_TRUE((bool)Executor);
> > + EXPECT_EQ(Executor->get()->getExecutorName(),
> TestToolExecutor::ExecutorName);
> > +}
> > +
> > +TEST(StandaloneToolTest, SynctaxOnlyActionOnSimpleCode) {
> > + FixedCompilationDatabase Compilations("/",
> std::vector<std::string>());
> > + StandaloneToolExecutor Executor(Compilations,
> > + std::vector<std::string>(1, "/a.cc"));
> > + Executor.mapVirtualFile("/a.cc", "int x = 0;");
> > +
> > + auto Err =
> Executor.execute(newFrontendActionFactory<SyntaxOnlyAction>(),
> > + getClangSyntaxOnlyAdjuster());
> > + ASSERT_TRUE(!Err);
> > +}
> > +
> > +TEST(StandaloneToolTest, SimpleAction) {
> > + FixedCompilationDatabase Compilations("/",
> std::vector<std::string>());
> > + StandaloneToolExecutor Executor(Compilations,
> > + std::vector<std::string>(1, "/a.cc"));
> > + Executor.mapVirtualFile("/a.cc", "int x = 0;");
> > +
> > + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
> > + new ReportResultActionFactory(Executor.getExecutionContext())));
> > + ASSERT_TRUE(!Err);
> > + auto KVs = Executor.getToolResults()->AllKVResults();
> > + ASSERT_EQ(KVs.size(), 0u);
> > +}
> > +
> > +TEST(StandaloneToolTest, SimpleActionWithResult) {
> > + FixedCompilationDatabase Compilations("/",
> std::vector<std::string>());
> > + StandaloneToolExecutor Executor(Compilations,
> > + std::vector<std::string>(1, "/a.cc"));
> > + Executor.mapVirtualFile("/a.cc", "int x = 0; void f() {}");
> > +
> > + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
> > + new ReportResultActionFactory(Executor.getExecutionContext())));
> > + ASSERT_TRUE(!Err);
> > + auto KVs = Executor.getToolResults()->AllKVResults();
> > + ASSERT_EQ(KVs.size(), 1u);
> > + EXPECT_EQ("f", KVs[0].first);
> > + EXPECT_EQ("1", KVs[0].second);
> > +
> > + Executor.getToolResults()->forEachResult(
> > + [](StringRef, StringRef Value) { EXPECT_EQ("1", Value); });
> > +}
> > +
> > +} // end namespace tooling
> > +} // end namespace clang
> >
> >
> > _______________________________________________
> > cfe-commits mailing list
> > cfe-commits at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20171026/8192d7d0/attachment-0001.html>
More information about the cfe-commits
mailing list