r316653 - [Tooling] A new framework for executing clang frontend actions.
Eric Liu via cfe-commits
cfe-commits at lists.llvm.org
Thu Oct 26 07:04:12 PDT 2017
It's not intended, but r316661 indeed fixed
http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15 ;)
On Thu, Oct 26, 2017 at 3:16 PM Eric Liu <ioeric at google.com> wrote:
> Hi Maxim,
>
> Thanks for the email! I'll look into this right now.
>
> Thanks,
> Eric
>
> On Thu, Oct 26, 2017 at 3:11 PM Maxim Kuvyrkov <maxim.kuvyrkov at linaro.org>
> wrote:
>
>> Hi Eric,
>>
>> This seems to have broken clang-cmake-thumbv7-a15 buildbot [*]. Would
>> you please investigate?
>>
>> Or did you fix this while I was writing this email in 316661 :-) ?
>>
>> Thank you,
>>
>> [*]
>> http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15/builds/12511
>>
>> --
>> Maxim Kuvyrkov
>> www.linaro.org
>>
>>
>>
>> > On Oct 26, 2017, at 1:38 PM, Eric Liu via cfe-commits <
>> cfe-commits at lists.llvm.org> wrote:
>> >
>> > Author: ioeric
>> > Date: Thu Oct 26 03:38:14 2017
>> > New Revision: 316653
>> >
>> > URL: http://llvm.org/viewvc/llvm-project?rev=316653&view=rev
>> > Log:
>> > [Tooling] A new framework for executing clang frontend actions.
>> >
>> > Summary:
>> > This defines a `clang::tooling::ToolExecutor` interface that can be
>> extended to support different execution plans including standalone
>> execution on a given set of TUs or parallel execution on all TUs in a
>> codebase.
>> >
>> > In order to enable multiprocessing execution, tool actions are expected
>> to output result into a `ToolResults` interface provided by executors. The
>> `ToolResults` interface abstracts how results are stored e.g. in-memory for
>> standalone executions or on-disk for large-scale execution.
>> >
>> > New executors can be registered as `ToolExecutorPlugin`s via the
>> `ToolExecutorPluginRegistry`. CLI tools can use
>> `createExecutorFromCommandLineArgs` to create a specific registered
>> executor according to the command-line arguments.
>> >
>> > This patch also implements `StandaloneToolExecutor` which has the same
>> behavior as the current `ClangTool` interface, i.e. execute frontend
>> actions on a given set of TUs. At this point, it's simply a wrapper around
>> `ClangTool` at this point.
>> >
>> > This is still experimental but expected to replace the existing
>> `ClangTool` interface so that specific tools would not need to worry about
>> execution.
>> >
>> > Reviewers: klimek, arphaman, hokein, sammccall
>> >
>> > Reviewed By: klimek
>> >
>> > Subscribers: cfe-commits, djasper, mgorny, omtcyfz
>> >
>> > Differential Revision: https://reviews.llvm.org/D34272
>> >
>> > Added:
>> > cfe/trunk/include/clang/Tooling/Execution.h
>> > cfe/trunk/include/clang/Tooling/StandaloneExecution.h
>> > cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h
>> > cfe/trunk/lib/Tooling/Execution.cpp
>> > cfe/trunk/lib/Tooling/StandaloneExecution.cpp
>> > cfe/trunk/unittests/Tooling/ExecutionTest.cpp
>> > Modified:
>> > cfe/trunk/include/clang/Tooling/CommonOptionsParser.h
>> > cfe/trunk/include/clang/Tooling/Tooling.h
>> > cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp
>> > cfe/trunk/lib/Tooling/CMakeLists.txt
>> > cfe/trunk/lib/Tooling/CommonOptionsParser.cpp
>> > cfe/trunk/lib/Tooling/Tooling.cpp
>> > cfe/trunk/unittests/Tooling/CMakeLists.txt
>> >
>> > Modified: cfe/trunk/include/clang/Tooling/CommonOptionsParser.h
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/CommonOptionsParser.h?rev=316653&r1=316652&r2=316653&view=diff
>> >
>> ==============================================================================
>> > --- cfe/trunk/include/clang/Tooling/CommonOptionsParser.h (original)
>> > +++ cfe/trunk/include/clang/Tooling/CommonOptionsParser.h Thu Oct 26
>> 03:38:14 2017
>> > @@ -109,6 +109,10 @@ public:
>> > return SourcePathList;
>> > }
>> >
>> > + /// Returns the argument adjuster calculated from "--extra-arg" and
>> > + //"--extra-arg-before" options.
>> > + ArgumentsAdjuster getArgumentsAdjuster() { return Adjuster; }
>> > +
>> > static const char *const HelpMessage;
>> >
>> > private:
>> > @@ -121,6 +125,7 @@ private:
>> >
>> > std::unique_ptr<CompilationDatabase> Compilations;
>> > std::vector<std::string> SourcePathList;
>> > + ArgumentsAdjuster Adjuster;
>> > };
>> >
>> > class ArgumentsAdjustingCompilations : public CompilationDatabase {
>> >
>> > Added: cfe/trunk/include/clang/Tooling/Execution.h
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Execution.h?rev=316653&view=auto
>> >
>> ==============================================================================
>> > --- cfe/trunk/include/clang/Tooling/Execution.h (added)
>> > +++ cfe/trunk/include/clang/Tooling/Execution.h Thu Oct 26 03:38:14 2017
>> > @@ -0,0 +1,168 @@
>> > +//===--- Execution.h - Executing clang frontend actions -*- C++
>> ---------*-===//
>> > +//
>> > +// The LLVM Compiler Infrastructure
>> > +//
>> > +// This file is distributed under the University of Illinois Open
>> Source
>> > +// License. See LICENSE.TXT for details.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +//
>> > +// This file defines framework for executing clang frontend actions.
>> > +//
>> > +// The framework can be extended to support different execution plans
>> including
>> > +// standalone execution on the given TUs or parallel execution on all
>> TUs in
>> > +// the codebase.
>> > +//
>> > +// In order to enable multiprocessing execution, tool actions are
>> expected to
>> > +// output result into the ToolResults provided by the executor. The
>> > +// `ToolResults` is an interface that abstracts how results are
>> stored e.g.
>> > +// in-memory for standalone execution or on-disk for large-scale
>> execution.
>> > +//
>> > +// New executors can be registered as ToolExecutorPlugins via the
>> > +// `ToolExecutorPluginRegistry`. CLI tools can use
>> > +// `createExecutorFromCommandLineArgs` to create a specific
>> registered executor
>> > +// according to the command-line arguments.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +
>> > +#ifndef LLVM_CLANG_TOOLING_EXECUTION_H
>> > +#define LLVM_CLANG_TOOLING_EXECUTION_H
>> > +
>> > +#include "clang/Tooling/CommonOptionsParser.h"
>> > +#include "clang/Tooling/Tooling.h"
>> > +#include "llvm/Support/Error.h"
>> > +#include "llvm/Support/Registry.h"
>> > +
>> > +namespace clang {
>> > +namespace tooling {
>> > +
>> > +/// \brief An abstraction for the result of a tool execution. For
>> example, the
>> > +/// underlying result can be in-memory or on-disk.
>> > +///
>> > +/// Results should be string key-value pairs. For example, a
>> refactoring tool
>> > +/// can use source location as key and a replacement in YAML format as
>> value.
>> > +class ToolResults {
>> > +public:
>> > + virtual ~ToolResults() = default;
>> > + virtual void addResult(StringRef Key, StringRef Value) = 0;
>> > + virtual std::vector<std::pair<std::string, std::string>>
>> AllKVResults() = 0;
>> > + virtual void forEachResult(
>> > + llvm::function_ref<void(StringRef Key, StringRef Value)>
>> Callback) = 0;
>> > +};
>> > +
>> > +class InMemoryToolResults : public ToolResults {
>> > +public:
>> > + void addResult(StringRef Key, StringRef Value) override;
>> > + std::vector<std::pair<std::string, std::string>> AllKVResults()
>> override;
>> > + void forEachResult(llvm::function_ref<void(StringRef Key, StringRef
>> Value)>
>> > + Callback) override;
>> > +
>> > +private:
>> > + std::vector<std::pair<std::string, std::string>> KVResults;
>> > +};
>> > +
>> > +/// \brief The context of an execution, including the information about
>> > +/// compilation and results.
>> > +class ExecutionContext {
>> > +public:
>> > + virtual ~ExecutionContext() {}
>> > +
>> > + /// \brief Initializes a context. This does not take ownership of
>> `Results`.
>> > + explicit ExecutionContext(ToolResults *Results) : Results(Results) {}
>> > +
>> > + /// \brief Adds a KV pair to the result container of this execution.
>> > + void reportResult(StringRef Key, StringRef Value);
>> > +
>> > + // Returns the source control system's revision number if applicable.
>> > + // Otherwise returns an empty string.
>> > + virtual std::string getRevision() { return ""; }
>> > +
>> > + // Returns the corpus being analyzed, e.g. "llvm" for the LLVM
>> codebase, if
>> > + // applicable.
>> > + virtual std::string getCorpus() { return ""; }
>> > +
>> > + // Returns the currently processed compilation unit if available.
>> > + virtual std::string getCurrentCompilationUnit() { return ""; }
>> > +
>> > +private:
>> > + ToolResults *Results;
>> > +};
>> > +
>> > +/// \brief Interface for executing clang frontend actions.
>> > +///
>> > +/// This can be extended to support running tool actions in different
>> > +/// execution mode, e.g. on a specific set of TUs or many TUs in
>> parallel.
>> > +///
>> > +/// New executors can be registered as ToolExecutorPlugins via the
>> > +/// `ToolExecutorPluginRegistry`. CLI tools can use
>> > +/// `createExecutorFromCommandLineArgs` to create a specific
>> registered
>> > +/// executor according to the command-line arguments.
>> > +class ToolExecutor {
>> > +public:
>> > + virtual ~ToolExecutor() {}
>> > +
>> > + /// \brief Returns the name of a specific executor.
>> > + virtual StringRef getExecutorName() const = 0;
>> > +
>> > + /// \brief Executes each action with a corresponding arguments
>> adjuster.
>> > + virtual llvm::Error
>> > + execute(llvm::ArrayRef<
>> > + std::pair<std::unique_ptr<FrontendActionFactory>,
>> ArgumentsAdjuster>>
>> > + Actions) = 0;
>> > +
>> > + /// \brief Convenient functions for the above `execute`.
>> > + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action);
>> > + /// Executes an action with an argument adjuster.
>> > + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action,
>> > + ArgumentsAdjuster Adjuster);
>> > +
>> > + /// \brief Returns a reference to the execution context.
>> > + ///
>> > + /// This should be passed to tool callbacks, and tool callbacks
>> should report
>> > + /// results via the returned context.
>> > + virtual ExecutionContext *getExecutionContext() = 0;
>> > +
>> > + /// \brief Returns a reference to the result container.
>> > + ///
>> > + /// NOTE: This should only be used after the execution finishes. Tool
>> > + /// callbacks should report results via `ExecutionContext` instead.
>> > + virtual ToolResults *getToolResults() = 0;
>> > +
>> > + /// \brief Map a virtual file to be used while running the tool.
>> > + ///
>> > + /// \param FilePath The path at which the content will be mapped.
>> > + /// \param Content A buffer of the file's content.
>> > + virtual void mapVirtualFile(StringRef FilePath, StringRef Content) =
>> 0;
>> > +};
>> > +
>> > +/// \brief Interface for factories that create specific executors.
>> This is also
>> > +/// used as a plugin to be registered into ToolExecutorPluginRegistry.
>> > +class ToolExecutorPlugin {
>> > +public:
>> > + virtual ~ToolExecutorPlugin() {}
>> > +
>> > + /// \brief Create an `ToolExecutor`.
>> > + ///
>> > + /// `OptionsParser` can be consumed (e.g. moved) if the creation
>> succeeds.
>> > + virtual llvm::Expected<std::unique_ptr<ToolExecutor>>
>> > + create(CommonOptionsParser &OptionsParser) = 0;
>> > +};
>> > +
>> > +/// \brief This creates a ToolExecutor that is in the global registry
>> based on
>> > +/// commandline arguments.
>> > +///
>> > +/// This picks the right executor based on the `--executor` option.
>> This parses
>> > +/// the commandline arguments with `CommonOptionsParser`, so caller
>> does not
>> > +/// need to parse again.
>> > +///
>> > +/// By default, this creates a `StandaloneToolExecutor` ("standalone")
>> if
>> > +/// `--executor` is not provided.
>> > +llvm::Expected<std::unique_ptr<ToolExecutor>>
>> > +createExecutorFromCommandLineArgs(int &argc, const char **argv,
>> > + llvm::cl::OptionCategory &Category,
>> > + const char *Overview = nullptr);
>> > +
>> > +} // end namespace tooling
>> > +} // end namespace clang
>> > +
>> > +#endif // LLVM_CLANG_TOOLING_EXECUTION_H
>> >
>> > Added: cfe/trunk/include/clang/Tooling/StandaloneExecution.h
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/StandaloneExecution.h?rev=316653&view=auto
>> >
>> ==============================================================================
>> > --- cfe/trunk/include/clang/Tooling/StandaloneExecution.h (added)
>> > +++ cfe/trunk/include/clang/Tooling/StandaloneExecution.h Thu Oct 26
>> 03:38:14 2017
>> > @@ -0,0 +1,97 @@
>> > +//===--- StandaloneExecution.h - Standalone execution. -*- C++
>> ----------*-===//
>> > +//
>> > +// The LLVM Compiler Infrastructure
>> > +//
>> > +// This file is distributed under the University of Illinois Open
>> Source
>> > +// License. See LICENSE.TXT for details.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +//
>> > +// This file defines standalone execution of clang tools.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +
>> > +#ifndef LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
>> > +#define LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
>> > +
>> > +#include "clang/Tooling/ArgumentsAdjusters.h"
>> > +#include "clang/Tooling/Execution.h"
>> > +
>> > +namespace clang {
>> > +namespace tooling {
>> > +
>> > +/// \brief A standalone executor that runs FrontendActions on a given
>> set of
>> > +/// TUs in sequence.
>> > +///
>> > +/// By default, this executor uses the following arguments adjusters
>> (as defined
>> > +/// in `clang/Tooling/ArgumentsAdjusters.h`):
>> > +/// - `getClangStripOutputAdjuster()`
>> > +/// - `getClangSyntaxOnlyAdjuster()`
>> > +/// - `getClangStripDependencyFileAdjuster()`
>> > +class StandaloneToolExecutor : public ToolExecutor {
>> > +public:
>> > + static const char *ExecutorName;
>> > +
>> > + /// \brief Init with \p CompilationDatabase and the paths of all
>> files to be
>> > + /// proccessed.
>> > + StandaloneToolExecutor(
>> > + const CompilationDatabase &Compilations,
>> > + llvm::ArrayRef<std::string> SourcePaths,
>> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
>> > + std::make_shared<PCHContainerOperations>());
>> > +
>> > + /// \brief Init with \p CommonOptionsParser. This is expected to be
>> used by
>> > + /// `createExecutorFromCommandLineArgs` based on commandline options.
>> > + ///
>> > + /// The executor takes ownership of \p Options.
>> > + StandaloneToolExecutor(
>> > + CommonOptionsParser Options,
>> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps =
>> > + std::make_shared<PCHContainerOperations>());
>> > +
>> > + StringRef getExecutorName() const override { return ExecutorName; }
>> > +
>> > + using ToolExecutor::execute;
>> > +
>> > + llvm::Error
>> > + execute(llvm::ArrayRef<
>> > + std::pair<std::unique_ptr<FrontendActionFactory>,
>> ArgumentsAdjuster>>
>> > + Actions) override;
>> > +
>> > + /// \brief Set a \c DiagnosticConsumer to use during parsing.
>> > + void setDiagnosticConsumer(DiagnosticConsumer *DiagConsumer) {
>> > + Tool.setDiagnosticConsumer(DiagConsumer);
>> > + }
>> > +
>> > + ExecutionContext *getExecutionContext() override { return &Context;
>> };
>> > +
>> > + ToolResults *getToolResults() override { return &Results; }
>> > +
>> > + llvm::ArrayRef<std::string> getSourcePaths() const {
>> > + return Tool.getSourcePaths();
>> > + }
>> > +
>> > + void mapVirtualFile(StringRef FilePath, StringRef Content) override {
>> > + Tool.mapVirtualFile(FilePath, Content);
>> > + }
>> > +
>> > + /// \brief Returns the file manager used in the tool.
>> > + ///
>> > + /// The file manager is shared between all translation units.
>> > + FileManager &getFiles() { return Tool.getFiles(); }
>> > +
>> > +private:
>> > + // Used to store the parser when the executor is initialized with
>> parser.
>> > + llvm::Optional<CommonOptionsParser> OptionsParser;
>> > + // FIXME: The standalone executor is currently just a wrapper of
>> `ClangTool`.
>> > + // Merge `ClangTool` implementation into the this.
>> > + ClangTool Tool;
>> > + ExecutionContext Context;
>> > + InMemoryToolResults Results;
>> > + ArgumentsAdjuster ArgsAdjuster;
>> > +};
>> > +
>> > +} // end namespace tooling
>> > +} // end namespace clang
>> > +
>> > +#endif // LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
>> >
>> > Added: cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h?rev=316653&view=auto
>> >
>> ==============================================================================
>> > --- cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h (added)
>> > +++ cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h Thu
>> Oct 26 03:38:14 2017
>> > @@ -0,0 +1,24 @@
>> > +//===--- ToolExecutorPluginRegistry.h - -------------------------*-
>> C++ -*-===//
>> > +//
>> > +// The LLVM Compiler Infrastructure
>> > +//
>> > +// This file is distributed under the University of Illinois Open
>> Source
>> > +// License. See LICENSE.TXT for details.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +
>> > +#ifndef LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
>> > +#define LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
>> > +
>> > +#include "clang/Tooling/Execution.h"
>> > +#include "llvm/Support/Registry.h"
>> > +
>> > +namespace clang {
>> > +namespace tooling {
>> > +
>> > +typedef llvm::Registry<ToolExecutorPlugin> ToolExecutorPluginRegistry;
>> > +
>> > +} // end namespace tooling
>> > +} // end namespace clang
>> > +
>> > +#endif // LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
>> >
>> > Modified: cfe/trunk/include/clang/Tooling/Tooling.h
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Tooling.h?rev=316653&r1=316652&r2=316653&view=diff
>> >
>> ==============================================================================
>> > --- cfe/trunk/include/clang/Tooling/Tooling.h (original)
>> > +++ cfe/trunk/include/clang/Tooling/Tooling.h Thu Oct 26 03:38:14 2017
>> > @@ -31,12 +31,12 @@
>> > #define LLVM_CLANG_TOOLING_TOOLING_H
>> >
>> > #include "clang/AST/ASTConsumer.h"
>> > -#include "clang/Frontend/PCHContainerOperations.h"
>> > #include "clang/Basic/Diagnostic.h"
>> > #include "clang/Basic/FileManager.h"
>> > #include "clang/Basic/LLVM.h"
>> > #include "clang/Driver/Util.h"
>> > #include "clang/Frontend/FrontendAction.h"
>> > +#include "clang/Frontend/PCHContainerOperations.h"
>> > #include "clang/Lex/ModuleLoader.h"
>> > #include "clang/Tooling/ArgumentsAdjusters.h"
>> > #include "clang/Tooling/CompilationDatabase.h"
>> > @@ -337,7 +337,9 @@ class ClangTool {
>> > /// The file manager is shared between all translation units.
>> > FileManager &getFiles() { return *Files; }
>> >
>> > - private:
>> > + llvm::ArrayRef<std::string> getSourcePaths() const { return
>> SourcePaths; }
>> > +
>> > +private:
>> > const CompilationDatabase &Compilations;
>> > std::vector<std::string> SourcePaths;
>> > std::shared_ptr<PCHContainerOperations> PCHContainerOps;
>> >
>> > Modified: cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp?rev=316653&r1=316652&r2=316653&view=diff
>> >
>> ==============================================================================
>> > --- cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp (original)
>> > +++ cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp Thu Oct 26 03:38:14
>> 2017
>> > @@ -96,6 +96,10 @@ ArgumentsAdjuster getInsertArgumentAdjus
>> >
>> > ArgumentsAdjuster combineAdjusters(ArgumentsAdjuster First,
>> > ArgumentsAdjuster Second) {
>> > + if (!First)
>> > + return Second;
>> > + if (!Second)
>> > + return First;
>> > return [First, Second](const CommandLineArguments &Args, StringRef
>> File) {
>> > return Second(First(Args, File), File);
>> > };
>> >
>> > Modified: cfe/trunk/lib/Tooling/CMakeLists.txt
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff
>> >
>> ==============================================================================
>> > --- cfe/trunk/lib/Tooling/CMakeLists.txt (original)
>> > +++ cfe/trunk/lib/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017
>> > @@ -11,11 +11,13 @@ add_clang_library(clangTooling
>> > ArgumentsAdjusters.cpp
>> > CommonOptionsParser.cpp
>> > CompilationDatabase.cpp
>> > + Execution.cpp
>> > FileMatchTrie.cpp
>> > FixIt.cpp
>> > JSONCompilationDatabase.cpp
>> > Refactoring.cpp
>> > RefactoringCallbacks.cpp
>> > + StandaloneExecution.cpp
>> > Tooling.cpp
>> >
>> > DEPENDS
>> >
>> > Modified: cfe/trunk/lib/Tooling/CommonOptionsParser.cpp
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CommonOptionsParser.cpp?rev=316653&r1=316652&r2=316653&view=diff
>> >
>> ==============================================================================
>> > --- cfe/trunk/lib/Tooling/CommonOptionsParser.cpp (original)
>> > +++ cfe/trunk/lib/Tooling/CommonOptionsParser.cpp Thu Oct 26 03:38:14
>> 2017
>> > @@ -147,10 +147,12 @@ llvm::Error CommonOptionsParser::init(
>> > auto AdjustingCompilations =
>> > llvm::make_unique<ArgumentsAdjustingCompilations>(
>> > std::move(Compilations));
>> > - AdjustingCompilations->appendArgumentsAdjuster(
>> > - getInsertArgumentAdjuster(ArgsBefore,
>> ArgumentInsertPosition::BEGIN));
>> > - AdjustingCompilations->appendArgumentsAdjuster(
>> > + Adjuster =
>> > + getInsertArgumentAdjuster(ArgsBefore,
>> ArgumentInsertPosition::BEGIN);
>> > + Adjuster = combineAdjusters(
>> > + std::move(Adjuster),
>> > getInsertArgumentAdjuster(ArgsAfter,
>> ArgumentInsertPosition::END));
>> > + AdjustingCompilations->appendArgumentsAdjuster(Adjuster);
>> > Compilations = std::move(AdjustingCompilations);
>> > return llvm::Error::success();
>> > }
>> >
>> > Added: cfe/trunk/lib/Tooling/Execution.cpp
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Execution.cpp?rev=316653&view=auto
>> >
>> ==============================================================================
>> > --- cfe/trunk/lib/Tooling/Execution.cpp (added)
>> > +++ cfe/trunk/lib/Tooling/Execution.cpp Thu Oct 26 03:38:14 2017
>> > @@ -0,0 +1,89 @@
>> > +//===- lib/Tooling/Execution.cpp - Implements tool execution
>> framework. ---===//
>> > +//
>> > +// The LLVM Compiler Infrastructure
>> > +//
>> > +// This file is distributed under the University of Illinois Open
>> Source
>> > +// License. See LICENSE.TXT for details.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +
>> > +#include "clang/Tooling/Execution.h"
>> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
>> > +#include "clang/Tooling/Tooling.h"
>> > +
>> > +LLVM_INSTANTIATE_REGISTRY(clang::tooling::ToolExecutorPluginRegistry)
>> > +
>> > +namespace clang {
>> > +namespace tooling {
>> > +
>> > +static llvm::cl::opt<std::string>
>> > + ExecutorName("executor", llvm::cl::desc("The name of the executor
>> to use."),
>> > + llvm::cl::init("standalone"));
>> > +
>> > +void InMemoryToolResults::addResult(StringRef Key, StringRef Value) {
>> > + KVResults.push_back({Key.str(), Value.str()});
>> > +}
>> > +
>> > +std::vector<std::pair<std::string, std::string>>
>> > +InMemoryToolResults::AllKVResults() {
>> > + return KVResults;
>> > +}
>> > +
>> > +void InMemoryToolResults::forEachResult(
>> > + llvm::function_ref<void(StringRef Key, StringRef Value)> Callback)
>> {
>> > + for (const auto &KV : KVResults) {
>> > + Callback(KV.first, KV.second);
>> > + }
>> > +}
>> > +
>> > +void ExecutionContext::reportResult(StringRef Key, StringRef Value) {
>> > + Results->addResult(Key, Value);
>> > +}
>> > +
>> > +llvm::Error
>> > +ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action) {
>> > + return execute(std::move(Action), ArgumentsAdjuster());
>> > +}
>> > +
>> > +llvm::Error
>> ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action,
>> > + ArgumentsAdjuster Adjuster) {
>> > + std::vector<
>> > + std::pair<std::unique_ptr<FrontendActionFactory>,
>> ArgumentsAdjuster>>
>> > + Actions;
>> > + Actions.emplace_back(std::move(Action), std::move(Adjuster));
>> > + return execute(Actions);
>> > +}
>> > +
>> > +llvm::Expected<std::unique_ptr<ToolExecutor>>
>> > +createExecutorFromCommandLineArgs(int &argc, const char **argv,
>> > + llvm::cl::OptionCategory &Category,
>> > + const char *Overview) {
>> > + auto OptionsParser =
>> > + CommonOptionsParser::create(argc, argv, Category,
>> llvm::cl::ZeroOrMore,
>> > + /*Overview=*/nullptr);
>> > + if (!OptionsParser)
>> > + return OptionsParser.takeError();
>> > + for (auto I = ToolExecutorPluginRegistry::begin(),
>> > + E = ToolExecutorPluginRegistry::end();
>> > + I != E; ++I) {
>> > + if (I->getName() != ExecutorName) {
>> > + continue;
>> > + }
>> > + std::unique_ptr<ToolExecutorPlugin> Plugin(I->instantiate());
>> > + llvm::Expected<std::unique_ptr<ToolExecutor>> Executor =
>> > + Plugin->create(*OptionsParser);
>> > + if (!Executor) {
>> > + return llvm::make_error<llvm::StringError>(
>> > + llvm::Twine("Failed to create '") + I->getName() +
>> > + "': " + llvm::toString(Executor.takeError()) + "\n",
>> > + llvm::inconvertibleErrorCode());
>> > + }
>> > + return std::move(*Executor);
>> > + }
>> > + return llvm::make_error<llvm::StringError>(
>> > + llvm::Twine("Executor \"") + ExecutorName + "\" is not
>> registered.",
>> > + llvm::inconvertibleErrorCode());
>> > +}
>> > +
>> > +} // end namespace tooling
>> > +} // end namespace clang
>> >
>> > Added: cfe/trunk/lib/Tooling/StandaloneExecution.cpp
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/StandaloneExecution.cpp?rev=316653&view=auto
>> >
>> ==============================================================================
>> > --- cfe/trunk/lib/Tooling/StandaloneExecution.cpp (added)
>> > +++ cfe/trunk/lib/Tooling/StandaloneExecution.cpp Thu Oct 26 03:38:14
>> 2017
>> > @@ -0,0 +1,91 @@
>> > +//===- lib/Tooling/Execution.cpp - Standalone clang action execution.
>> -----===//
>> > +//
>> > +// The LLVM Compiler Infrastructure
>> > +//
>> > +// This file is distributed under the University of Illinois Open
>> Source
>> > +// License. See LICENSE.TXT for details.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +
>> > +#include "clang/Tooling/StandaloneExecution.h"
>> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
>> > +
>> > +namespace clang {
>> > +namespace tooling {
>> > +
>> > +static llvm::Error make_string_error(const llvm::Twine &Message) {
>> > + return llvm::make_error<llvm::StringError>(Message,
>> > +
>> llvm::inconvertibleErrorCode());
>> > +}
>> > +
>> > +const char *StandaloneToolExecutor::ExecutorName =
>> "StandaloneToolExecutor";
>> > +
>> > +static ArgumentsAdjuster getDefaultArgumentsAdjusters() {
>> > + return combineAdjusters(
>> > + getClangStripOutputAdjuster(),
>> > + combineAdjusters(getClangSyntaxOnlyAdjuster(),
>> > + getClangStripDependencyFileAdjuster()));
>> > +}
>> > +
>> > +StandaloneToolExecutor::StandaloneToolExecutor(
>> > + const CompilationDatabase &Compilations,
>> > + llvm::ArrayRef<std::string> SourcePaths,
>> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps)
>> > + : Tool(Compilations, SourcePaths), Context(&Results),
>> > + ArgsAdjuster(getDefaultArgumentsAdjusters()) {
>> > + // Use self-defined default argument adjusters instead of the default
>> > + // adjusters that come with the old `ClangTool`.
>> > + Tool.clearArgumentsAdjusters();
>> > +}
>> > +
>> > +StandaloneToolExecutor::StandaloneToolExecutor(
>> > + CommonOptionsParser Options,
>> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps)
>> > + : OptionsParser(std::move(Options)),
>> > + Tool(OptionsParser->getCompilations(),
>> OptionsParser->getSourcePathList(),
>> > + PCHContainerOps),
>> > + Context(&Results), ArgsAdjuster(getDefaultArgumentsAdjusters()) {
>> > + Tool.clearArgumentsAdjusters();
>> > +}
>> > +
>> > +llvm::Error StandaloneToolExecutor::execute(
>> > + llvm::ArrayRef<
>> > + std::pair<std::unique_ptr<FrontendActionFactory>,
>> ArgumentsAdjuster>>
>> > + Actions) {
>> > + if (Actions.empty())
>> > + return make_string_error("No action to execute.");
>> > +
>> > + if (Actions.size() != 1)
>> > + return make_string_error(
>> > + "Only support executing exactly 1 action at this point.");
>> > +
>> > + auto &Action = Actions.front();
>> > + Tool.appendArgumentsAdjuster(Action.second);
>> > + Tool.appendArgumentsAdjuster(ArgsAdjuster);
>> > + if (int Ret = Tool.run(Action.first.get()))
>> > + return make_string_error("Failed to run action.");
>> > +
>> > + return llvm::Error::success();
>> > +}
>> > +
>> > +class StandaloneToolExecutorPlugin : public ToolExecutorPlugin {
>> > +public:
>> > + llvm::Expected<std::unique_ptr<ToolExecutor>>
>> > + create(CommonOptionsParser &OptionsParser) override {
>> > + if (OptionsParser.getSourcePathList().empty())
>> > + return make_string_error(
>> > + "[StandaloneToolExecutorPlugin] No positional argument
>> found.");
>> > + return
>> llvm::make_unique<StandaloneToolExecutor>(std::move(OptionsParser));
>> > + }
>> > +};
>> > +
>> > +// This anchor is used to force the linker to link in the generated
>> object file
>> > +// and thus register the plugin.
>> > +volatile int ToolExecutorPluginAnchorSource = 0;
>> > +
>> > +static ToolExecutorPluginRegistry::Add<StandaloneToolExecutorPlugin>
>> > + X("standalone", "Runs FrontendActions on a set of files provided "
>> > + "via positional arguments.");
>> > +
>> > +} // end namespace tooling
>> > +} // end namespace clang
>> >
>> > Modified: cfe/trunk/lib/Tooling/Tooling.cpp
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Tooling.cpp?rev=316653&r1=316652&r2=316653&view=diff
>> >
>> ==============================================================================
>> > --- cfe/trunk/lib/Tooling/Tooling.cpp (original)
>> > +++ cfe/trunk/lib/Tooling/Tooling.cpp Thu Oct 26 03:38:14 2017
>> > @@ -29,6 +29,7 @@
>> > #include "llvm/Config/llvm-config.h"
>> > #include "llvm/Option/ArgList.h"
>> > #include "llvm/Option/Option.h"
>> > +#include "llvm/Support/CommandLine.h"
>> > #include "llvm/Support/Debug.h"
>> > #include "llvm/Support/FileSystem.h"
>> > #include "llvm/Support/Host.h"
>> > @@ -347,11 +348,7 @@ void ClangTool::mapVirtualFile(StringRef
>> > }
>> >
>> > void ClangTool::appendArgumentsAdjuster(ArgumentsAdjuster Adjuster) {
>> > - if (ArgsAdjuster)
>> > - ArgsAdjuster =
>> > - combineAdjusters(std::move(ArgsAdjuster), std::move(Adjuster));
>> > - else
>> > - ArgsAdjuster = std::move(Adjuster);
>> > + ArgsAdjuster = combineAdjusters(std::move(ArgsAdjuster),
>> std::move(Adjuster));
>> > }
>> >
>> > void ClangTool::clearArgumentsAdjusters() {
>> >
>> > Modified: cfe/trunk/unittests/Tooling/CMakeLists.txt
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff
>> >
>> ==============================================================================
>> > --- cfe/trunk/unittests/Tooling/CMakeLists.txt (original)
>> > +++ cfe/trunk/unittests/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017
>> > @@ -16,6 +16,7 @@ add_clang_unittest(ToolingTests
>> > CommentHandlerTest.cpp
>> > CompilationDatabaseTest.cpp
>> > DiagnosticsYamlTest.cpp
>> > + ExecutionTest.cpp
>> > FixItTest.cpp
>> > LexicallyOrderedRecursiveASTVisitorTest.cpp
>> > LookupTest.cpp
>> >
>> > Added: cfe/trunk/unittests/Tooling/ExecutionTest.cpp
>> > URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/ExecutionTest.cpp?rev=316653&view=auto
>> >
>> ==============================================================================
>> > --- cfe/trunk/unittests/Tooling/ExecutionTest.cpp (added)
>> > +++ cfe/trunk/unittests/Tooling/ExecutionTest.cpp Thu Oct 26 03:38:14
>> 2017
>> > @@ -0,0 +1,228 @@
>> > +//===- unittest/Tooling/ExecutionTest.cpp - Tool execution tests.
>> --------===//
>> > +//
>> > +// The LLVM Compiler Infrastructure
>> > +//
>> > +// This file is distributed under the University of Illinois Open
>> Source
>> > +// License. See LICENSE.TXT for details.
>> > +//
>> >
>> +//===----------------------------------------------------------------------===//
>> > +
>> > +#include "clang/AST/ASTConsumer.h"
>> > +#include "clang/AST/DeclCXX.h"
>> > +#include "clang/AST/RecursiveASTVisitor.h"
>> > +#include "clang/Frontend/ASTUnit.h"
>> > +#include "clang/Frontend/FrontendAction.h"
>> > +#include "clang/Frontend/FrontendActions.h"
>> > +#include "clang/Tooling/CompilationDatabase.h"
>> > +#include "clang/Tooling/Execution.h"
>> > +#include "clang/Tooling/StandaloneExecution.h"
>> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h"
>> > +#include "clang/Tooling/Tooling.h"
>> > +#include "gtest/gtest.h"
>> > +#include <algorithm>
>> > +#include <string>
>> > +
>> > +namespace clang {
>> > +namespace tooling {
>> > +
>> > +namespace {
>> > +
>> > +// This traverses the AST and outputs function name as key and "1" as
>> value for
>> > +// each function declaration.
>> > +class ASTConsumerWithResult
>> > + : public ASTConsumer,
>> > + public RecursiveASTVisitor<ASTConsumerWithResult> {
>> > +public:
>> > + using ASTVisitor = RecursiveASTVisitor<ASTConsumerWithResult>;
>> > +
>> > + explicit ASTConsumerWithResult(ExecutionContext *Context) :
>> Context(Context) {
>> > + assert(Context != nullptr);
>> > + }
>> > +
>> > + void HandleTranslationUnit(clang::ASTContext &Context) override {
>> > + TraverseDecl(Context.getTranslationUnitDecl());
>> > + }
>> > +
>> > + bool TraverseFunctionDecl(clang::FunctionDecl *Decl) {
>> > + Context->reportResult(Decl->getNameAsString(), "1");
>> > + return ASTVisitor::TraverseFunctionDecl(Decl);
>> > + }
>> > +
>> > +private:
>> > + ExecutionContext *const Context;
>> > +};
>> > +
>> > +class ReportResultAction : public ASTFrontendAction {
>> > +public:
>> > + explicit ReportResultAction(ExecutionContext *Context) :
>> Context(Context) {
>> > + assert(Context != nullptr);
>> > + }
>> > +
>> > +protected:
>> > + std::unique_ptr<clang::ASTConsumer>
>> > + CreateASTConsumer(clang::CompilerInstance &compiler,
>> > + StringRef /* dummy */) override {
>> > + std::unique_ptr<clang::ASTConsumer> ast_consumer{
>> > + new ASTConsumerWithResult(Context)};
>> > + return ast_consumer;
>> > + }
>> > +
>> > +private:
>> > + ExecutionContext *const Context;
>> > +};
>> > +
>> > +class ReportResultActionFactory : public FrontendActionFactory {
>> > +public:
>> > + ReportResultActionFactory(ExecutionContext *Context) :
>> Context(Context) {}
>> > + FrontendAction *create() override { return new
>> ReportResultAction(Context); }
>> > +
>> > +private:
>> > + ExecutionContext *const Context;
>> > +};
>> > +
>> > +} // namespace
>> > +
>> > +class TestToolExecutor : public ToolExecutor {
>> > +public:
>> > + static const char *ExecutorName;
>> > +
>> > + TestToolExecutor(CommonOptionsParser Options)
>> > + : OptionsParser(std::move(Options)) {}
>> > +
>> > + StringRef getExecutorName() const override { return ExecutorName; }
>> > +
>> > + llvm::Error
>> > +
>> execute(llvm::ArrayRef<std::pair<std::unique_ptr<FrontendActionFactory>,
>> > + ArgumentsAdjuster>>) override {
>> > + return llvm::Error::success();
>> > + }
>> > +
>> > + ExecutionContext *getExecutionContext() override { return nullptr; };
>> > +
>> > + ToolResults *getToolResults() override { return nullptr; }
>> > +
>> > + llvm::ArrayRef<std::string> getSourcePaths() const {
>> > + return OptionsParser.getSourcePathList();
>> > + }
>> > +
>> > + void mapVirtualFile(StringRef FilePath, StringRef Content) override {
>> > + VFS[FilePath] = Content;
>> > + }
>> > +
>> > +private:
>> > + CommonOptionsParser OptionsParser;
>> > + std::string SourcePaths;
>> > + std::map<std::string, std::string> VFS;
>> > +};
>> > +
>> > +const char *TestToolExecutor::ExecutorName = "test-executor";
>> > +
>> > +class TestToolExecutorPlugin : public ToolExecutorPlugin {
>> > +public:
>> > + llvm::Expected<std::unique_ptr<ToolExecutor>>
>> > + create(CommonOptionsParser &OptionsParser) override {
>> > + return
>> llvm::make_unique<TestToolExecutor>(std::move(OptionsParser));
>> > + }
>> > +};
>> > +
>> > +// This anchor is used to force the linker to link in the generated
>> object file
>> > +// and thus register the plugin.
>> > +extern volatile int ToolExecutorPluginAnchorSource;
>> > +
>> > +static int LLVM_ATTRIBUTE_UNUSED TestToolExecutorPluginAnchorDest =
>> > + ToolExecutorPluginAnchorSource;
>> > +
>> > +static ToolExecutorPluginRegistry::Add<TestToolExecutorPlugin>
>> > + X("test-executor", "Plugin for TestToolExecutor.");
>> > +
>> > +llvm::cl::OptionCategory TestCategory("execution-test options");
>> > +
>> > +TEST(CreateToolExecutorTest, FailedCreateExecutorUndefinedFlag) {
>> > + std::vector<const char *> argv = {"prog", "--fake_flag_no_no_no",
>> "f"};
>> > + int argc = argv.size();
>> > + auto Executor =
>> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
>> > + ASSERT_FALSE((bool)Executor);
>> > + llvm::consumeError(Executor.takeError());
>> > +}
>> > +
>> > +TEST(CreateToolExecutorTest, RegisterFlagsBeforeReset) {
>> > + llvm::cl::opt<std::string> BeforeReset(
>> > + "before_reset", llvm::cl::desc("Defined before reset."),
>> > + llvm::cl::init(""));
>> > +
>> > + llvm::cl::ResetAllOptionOccurrences();
>> > +
>> > + std::vector<const char *> argv = {"prog", "--before_reset=set", "f"};
>> > + int argc = argv.size();
>> > + auto Executor =
>> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
>> > + ASSERT_TRUE((bool)Executor);
>> > + EXPECT_EQ(BeforeReset, "set");
>> > + BeforeReset.removeArgument();
>> > +}
>> > +
>> > +TEST(CreateToolExecutorTest, CreateStandaloneToolExecutor) {
>> > + std::vector<const char *> argv = {"prog", "standalone.cpp"};
>> > + int argc = argv.size();
>> > + auto Executor =
>> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
>> > + ASSERT_TRUE((bool)Executor);
>> > + EXPECT_EQ(Executor->get()->getExecutorName(),
>> > + StandaloneToolExecutor::ExecutorName);
>> > +}
>> > +
>> > +TEST(CreateToolExecutorTest, CreateTestToolExecutor) {
>> > + std::vector<const char *> argv = {"prog", "test.cpp",
>> > + "--executor=test-executor"};
>> > + int argc = argv.size();
>> > + auto Executor =
>> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
>> > + ASSERT_TRUE((bool)Executor);
>> > + EXPECT_EQ(Executor->get()->getExecutorName(),
>> TestToolExecutor::ExecutorName);
>> > +}
>> > +
>> > +TEST(StandaloneToolTest, SynctaxOnlyActionOnSimpleCode) {
>> > + FixedCompilationDatabase Compilations("/",
>> std::vector<std::string>());
>> > + StandaloneToolExecutor Executor(Compilations,
>> > + std::vector<std::string>(1,
>> "/a.cc"));
>> > + Executor.mapVirtualFile("/a.cc", "int x = 0;");
>> > +
>> > + auto Err =
>> Executor.execute(newFrontendActionFactory<SyntaxOnlyAction>(),
>> > + getClangSyntaxOnlyAdjuster());
>> > + ASSERT_TRUE(!Err);
>> > +}
>> > +
>> > +TEST(StandaloneToolTest, SimpleAction) {
>> > + FixedCompilationDatabase Compilations("/",
>> std::vector<std::string>());
>> > + StandaloneToolExecutor Executor(Compilations,
>> > + std::vector<std::string>(1,
>> "/a.cc"));
>> > + Executor.mapVirtualFile("/a.cc", "int x = 0;");
>> > +
>> > + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
>> > + new ReportResultActionFactory(Executor.getExecutionContext())));
>> > + ASSERT_TRUE(!Err);
>> > + auto KVs = Executor.getToolResults()->AllKVResults();
>> > + ASSERT_EQ(KVs.size(), 0u);
>> > +}
>> > +
>> > +TEST(StandaloneToolTest, SimpleActionWithResult) {
>> > + FixedCompilationDatabase Compilations("/",
>> std::vector<std::string>());
>> > + StandaloneToolExecutor Executor(Compilations,
>> > + std::vector<std::string>(1,
>> "/a.cc"));
>> > + Executor.mapVirtualFile("/a.cc", "int x = 0; void f() {}");
>> > +
>> > + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
>> > + new ReportResultActionFactory(Executor.getExecutionContext())));
>> > + ASSERT_TRUE(!Err);
>> > + auto KVs = Executor.getToolResults()->AllKVResults();
>> > + ASSERT_EQ(KVs.size(), 1u);
>> > + EXPECT_EQ("f", KVs[0].first);
>> > + EXPECT_EQ("1", KVs[0].second);
>> > +
>> > + Executor.getToolResults()->forEachResult(
>> > + [](StringRef, StringRef Value) { EXPECT_EQ("1", Value); });
>> > +}
>> > +
>> > +} // end namespace tooling
>> > +} // end namespace clang
>> >
>> >
>> > _______________________________________________
>> > cfe-commits mailing list
>> > cfe-commits at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20171026/5fd05545/attachment-0001.html>
More information about the cfe-commits
mailing list