[llvm] 258f055 - [Orc][examples] Add LLJITWithRemoteDebugging example
Stefan Gränitz via llvm-commits
llvm-commits at lists.llvm.org
Wed Mar 31 01:33:59 PDT 2021
Hi David, thanks for working on it!
I ran the test suite on Linux and macOS with examples enabled and it
worked well. Also, I didn't get a message from buildmaster for this. Do
we have a bot for your setup and is it public?
On 30/03/2021 22:05, David Blaikie wrote:
> This broke the build with examples enabled (LLVM_BUILD_EXAMPLES=ON) I
> think. My second attempt at fixing this is
> in ae217bf1f3277b8c14590a130ee5e63cc1664443
>
> On Sun, Mar 28, 2021 at 8:25 AM Stefan Gränitz via llvm-commits
> <llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>> wrote:
>
>
> Author: Stefan Gränitz
> Date: 2021-03-28T17:25:09+02:00
> New Revision: 258f055ed93661900bc568350e09f467c0950486
>
> URL:
> https://github.com/llvm/llvm-project/commit/258f055ed93661900bc568350e09f467c0950486
> <https://github.com/llvm/llvm-project/commit/258f055ed93661900bc568350e09f467c0950486>
> DIFF:
> https://github.com/llvm/llvm-project/commit/258f055ed93661900bc568350e09f467c0950486.diff
> <https://github.com/llvm/llvm-project/commit/258f055ed93661900bc568350e09f467c0950486.diff>
>
> LOG: [Orc][examples] Add LLJITWithRemoteDebugging example
>
> Added:
>
> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>
> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>
> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>
> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
> llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
> llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
> llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>
> Modified:
> llvm/examples/OrcV2Examples/CMakeLists.txt
> llvm/examples/OrcV2Examples/ExampleModules.h
>
> Removed:
>
>
>
> ################################################################################
> diff --git a/llvm/examples/OrcV2Examples/CMakeLists.txt
> b/llvm/examples/OrcV2Examples/CMakeLists.txt
> index bed277e59e0b..f6a11d6f0ef2 100644
> --- a/llvm/examples/OrcV2Examples/CMakeLists.txt
> +++ b/llvm/examples/OrcV2Examples/CMakeLists.txt
> @@ -12,3 +12,7 @@ add_subdirectory(OrcV2CBindingsAddObjectFile)
> add_subdirectory(OrcV2CBindingsBasicUsage)
> add_subdirectory(OrcV2CBindingsReflectProcessSymbols)
> add_subdirectory(OrcV2CBindingsRemovableCode)
> +
> +if(CMAKE_HOST_UNIX)
> + add_subdirectory(LLJITWithRemoteDebugging)
> +endif()
>
> diff --git a/llvm/examples/OrcV2Examples/ExampleModules.h
> b/llvm/examples/OrcV2Examples/ExampleModules.h
> index c88609fae769..53da756e15f7 100644
> --- a/llvm/examples/OrcV2Examples/ExampleModules.h
> +++ b/llvm/examples/OrcV2Examples/ExampleModules.h
> @@ -52,4 +52,16 @@ parseExampleModule(llvm::StringRef Source,
> llvm::StringRef Name) {
> return createSMDiagnosticError(Err);
> }
>
> +inline llvm::Expected<llvm::orc::ThreadSafeModule>
> +parseExampleModuleFromFile(llvm::StringRef FileName) {
> + using namespace llvm;
> + auto Ctx = std::make_unique<LLVMContext>();
> + SMDiagnostic Err;
> +
> + if (auto M = parseIRFile(FileName, Err, *Ctx))
> + return orc::ThreadSafeModule(std::move(M), std::move(Ctx));
> +
> + return createSMDiagnosticError(Err);
> +}
> +
> #endif // LLVM_EXAMPLES_ORCV2EXAMPLES_EXAMPLEMODULES_H
>
> diff --git
> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
> new file mode 100644
> index 000000000000..b93dd4924b05
> --- /dev/null
> +++
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
> @@ -0,0 +1,18 @@
> +set(LLVM_LINK_COMPONENTS
> + Core
> + ExecutionEngine
> + IRReader
> + JITLink
> + OrcJIT
> + OrcTargetProcess
> + Support
> + nativecodegen
> + )
> +
> +add_llvm_example(LLJITWithRemoteDebugging
> + LLJITWithRemoteDebugging.cpp
> + RemoteJITUtils.cpp
> +
> + DEPENDS
> + llvm-jitlink-executor
> + )
>
> diff --git
> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
> new file mode 100644
> index 000000000000..6da30f608206
> --- /dev/null
> +++
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
> @@ -0,0 +1,258 @@
> +//===--- LLJITWithRemoteDebugging.cpp - LLJIT targeting a child
> process ---===//
> +//
> +// Part of the LLVM Project, under the Apache License v2.0 with
> LLVM Exceptions.
> +// See https://llvm.org/LICENSE.txt
> <https://llvm.org/LICENSE.txt> for license information.
> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This example shows how to use LLJIT and JITLink for
> out-of-process execution
> +// with debug support. A few notes beforehand:
> +//
> +// * Debuggers must implement the GDB JIT interface (gdb, udb,
> lldb 12+).
> +// * Debug support is currently limited to ELF on x86-64
> platforms that run
> +// Unix-like systems.
> +// * There is a test for this example and it ships an IR file
> that is prepared
> +// for the instructions below.
> +//
> +//
> +// The following command line session provides a complete
> walkthrough of the
> +// feature using LLDB 12:
> +//
> +// [Terminal 1] Prepare a debuggable out-of-process JIT session:
> +//
> +// > cd llvm-project/build
> +// > ninja LLJITWithRemoteDebugging llvm-jitlink-executor
> +// > cp
> ../llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll .
> +// > bin/LLJITWithRemoteDebugging --wait-for-debugger
> argc_sub1_elf.ll
> +// Found out-of-process executor: bin/llvm-jitlink-executor
> +// Launched executor in subprocess: 65535
> +// Attach a debugger and press any key to continue.
> +//
> +//
> +// [Terminal 2] Attach a debugger to the child process:
> +//
> +// (lldb) log enable lldb jit
> +// (lldb) settings set plugin.jit-loader.gdb.enable on
> +// (lldb) settings set target.source-map Inputs/ \
> +//
> /path/to/llvm-project/llvm/test/Examples/OrcV2Examples/Inputs/
> +// (lldb) attach -p 65535
> +// JITLoaderGDB::SetJITBreakpoint looking for JIT register hook
> +// JITLoaderGDB::SetJITBreakpoint setting JIT breakpoint
> +// Process 65535 stopped
> +// (lldb) b sub1
> +// Breakpoint 1: no locations (pending).
> +// WARNING: Unable to resolve breakpoint to any actual locations.
> +// (lldb) c
> +// Process 65535 resuming
> +//
> +//
> +// [Terminal 1] Press a key to start code generation and execution:
> +//
> +// Parsed input IR code from: argc_sub1_elf.ll
> +// Initialized LLJIT for remote executor
> +// Running: argc_sub1_elf.ll
> +//
> +//
> +// [Terminal 2] Breakpoint hits; we change the argc value from 1
> to 42:
> +//
> +// (lldb) JITLoaderGDB::JITDebugBreakpointHit hit JIT breakpoint
> +// JITLoaderGDB::ReadJITDescriptorImpl registering JIT entry
> at 0x106b34000
> +// 1 location added to breakpoint 1
> +// Process 65535 stopped
> +// * thread #1, queue = 'com.apple.main-thread', stop reason =
> breakpoint 1.1
> +// frame #0: JIT(0x106b34000)`sub1(x=1) at argc_sub1.c:1:28
> +// -> 1 int sub1(int x) { return x - 1; }
> +// 2 int main(int argc, char **argv) { return sub1(argc); }
> +// (lldb) p x
> +// (int) $0 = 1
> +// (lldb) expr x = 42
> +// (int) $1 = 42
> +// (lldb) c
> +//
> +//
> +// [Terminal 1] Example output reflects the modified value:
> +//
> +// Exit code: 41
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
> +#include "llvm/ExecutionEngine/Orc/LLJIT.h"
> +#include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
> +#include "llvm/Support/CommandLine.h"
> +#include "llvm/Support/Error.h"
> +#include "llvm/Support/FormatVariadic.h"
> +#include "llvm/Support/InitLLVM.h"
> +#include "llvm/Support/TargetSelect.h"
> +#include "llvm/Support/raw_ostream.h"
> +
> +#include "../ExampleModules.h"
> +#include "RemoteJITUtils.h"
> +
> +#include <memory>
> +#include <string>
> +
> +using namespace llvm;
> +using namespace llvm::orc;
> +
> +// The LLVM IR file to run.
> +static cl::list<std::string> InputFiles(cl::Positional,
> cl::OneOrMore,
> + cl::desc("<input files>"));
> +
> +// Command line arguments to pass to the JITed main function.
> +static cl::list<std::string> InputArgv("args", cl::Positional,
> + cl::desc("<program
> arguments>..."),
> + cl::ZeroOrMore,
> cl::PositionalEatsArgs);
> +
> +// Given paths must exist on the remote target.
> +static cl::list<std::string>
> + Dylibs("dlopen", cl::desc("Dynamic libraries to load before
> linking"),
> + cl::value_desc("filename"), cl::ZeroOrMore);
> +
> +// File path of the executable to launch for execution in a child
> process.
> +// Inter-process communication will go through stdin/stdout pipes.
> +static cl::opt<std::string>
> + OOPExecutor("executor", cl::desc("Set the out-of-process
> executor"),
> + cl::value_desc("filename"));
> +
> +// Network address of a running executor process that we can
> connected through a
> +// TCP socket. It may run locally or on a remote machine.
> +static cl::opt<std::string> OOPExecutorConnect(
> + "connect",
> + cl::desc("Connect to an out-of-process executor through a TCP
> socket"),
> + cl::value_desc("<hostname>:<port>"));
> +
> +// Give the user a chance to connect a debugger. Once we
> connected the executor
> +// process, wait for the user to press a key (and print out its
> PID if it's a
> +// child process).
> +static cl::opt<bool>
> + WaitForDebugger("wait-for-debugger",
> + cl::desc("Wait for user input before entering
> JITed code"),
> + cl::init(false));
> +
> +ExitOnError ExitOnErr;
> +
> +static std::unique_ptr<JITLinkExecutor> connectExecutor(const
> char *Argv0,
> +
> ExecutionSession &ES) {
> + // Connect to a running out-of-process executor through a TCP
> socket.
> + if (!OOPExecutorConnect.empty()) {
> + std::unique_ptr<TCPSocketJITLinkExecutor> Exec =
> +
> ExitOnErr(JITLinkExecutor::ConnectTCPSocket(OOPExecutorConnect, ES));
> +
> + outs() << "Connected to executor at " << OOPExecutorConnect
> << "\n";
> + if (WaitForDebugger) {
> + outs() << "Attach a debugger and press any key to continue.\n";
> + fflush(stdin);
> + getchar();
> + }
> +
> + return std::move(Exec);
> + }
> +
> + // Launch a out-of-process executor locally in a child process.
> + std::unique_ptr<ChildProcessJITLinkExecutor> Exec = ExitOnErr(
> + OOPExecutor.empty() ? JITLinkExecutor::FindLocal(Argv0)
> + :
> JITLinkExecutor::CreateLocal(OOPExecutor));
> +
> + outs() << "Found out-of-process executor: " << Exec->getPath()
> << "\n";
> +
> + ExitOnErr(Exec->launch(ES));
> + if (WaitForDebugger) {
> + outs() << "Launched executor in subprocess: " <<
> Exec->getPID() << "\n"
> + << "Attach a debugger and press any key to continue.\n";
> + fflush(stdin);
> + getchar();
> + }
> +
> + return std::move(Exec);
> +}
> +
> +int main(int argc, char *argv[]) {
> + InitLLVM X(argc, argv);
> +
> + InitializeNativeTarget();
> + InitializeNativeTargetAsmPrinter();
> +
> + ExitOnErr.setBanner(std::string(argv[0]) + ": ");
> + cl::ParseCommandLineOptions(argc, argv,
> "LLJITWithRemoteDebugging");
> +
> + auto ES = std::make_unique<ExecutionSession>();
> + ES->setErrorReporter([&](Error Err) {
> ExitOnErr(std::move(Err)); });
> +
> + // Launch/connect the out-of-process executor.
> + std::unique_ptr<JITLinkExecutor> Executor =
> connectExecutor(argv[0], *ES);
> +
> + // Load the given IR files.
> + std::vector<ThreadSafeModule> TSMs;
> + for (const std::string &Path : InputFiles) {
> + outs() << "Parsing input IR code from: " << Path << "\n";
> + TSMs.push_back(ExitOnErr(parseExampleModuleFromFile(Path)));
> + }
> +
> + StringRef TT;
> + StringRef MainModuleName;
> + TSMs.front().withModuleDo([&MainModuleName, &TT](Module &M) {
> + MainModuleName = M.getName();
> + TT = M.getTargetTriple();
> + });
> +
> + for (const ThreadSafeModule &TSM : TSMs)
> + ExitOnErr(TSM.withModuleDo([TT, MainModuleName](Module &M) ->
> Error {
> + if (M.getTargetTriple() != TT)
> + return make_error<StringError>(
> + formatv("Different target triples in input files:\n"
> + " '{0}' in '{1}'\n '{2}' in '{3}'",
> + TT, MainModuleName, M.getTargetTriple(),
> M.getName()),
> + inconvertibleErrorCode());
> + return Error::success();
> + }));
> +
> + // Create a target machine that matches the input triple.
> + JITTargetMachineBuilder JTMB((Triple(TT)));
> + JTMB.setCodeModel(CodeModel::Small);
> + JTMB.setRelocationModel(Reloc::PIC_);
> +
> + // Create LLJIT and destroy it before disconnecting the target
> process.
> + {
> + outs() << "Initializing LLJIT for remote executor\n";
> + auto J = ExitOnErr(LLJITBuilder()
> + .setExecutionSession(std::move(ES))
> +
> .setJITTargetMachineBuilder(std::move(JTMB))
> +
> .setObjectLinkingLayerCreator(std::ref(*Executor))
> + .create());
> +
> + // Add plugin for debug support.
> + ExitOnErr(Executor->addDebugSupport(J->getObjLinkingLayer()));
> +
> + // Load required shared libraries on the remote target and
> add a generator
> + // for each of it, so the compiler can lookup their symbols.
> + for (const std::string &Path : Dylibs)
> +
> J->getMainJITDylib().addGenerator(ExitOnErr(Executor->loadDylib(Path)));
> +
> + // Add the loaded IR module to the JIT. This will set up
> symbol tables and
> + // prepare for materialization.
> + for (ThreadSafeModule &TSM : TSMs)
> + ExitOnErr(J->addIRModule(std::move(TSM)));
> +
> + // The example uses a non-lazy JIT for simplicity. Thus,
> looking up the main
> + // function will materialize all reachable code. It also
> triggers debug
> + // registration in the remote target process.
> + JITEvaluatedSymbol MainFn = ExitOnErr(J->lookup("main"));
> +
> + outs() << "Running: main(";
> + int Pos = 0;
> + for (const std::string &Arg : InputArgv)
> + outs() << (Pos++ == 0 ? "" : ", ") << Arg;
> + outs() << ")\n";
> +
> + // Execute the code in the remote target process and dump the
> result. With
> + // the debugger attached to the target, it should be possible
> to inspect the
> + // JITed code as if it was compiled statically.
> + int Result = ExitOnErr(Executor->runAsMain(MainFn, InputArgv));
> + outs() << "Exit code: " << Result << "\n";
> + }
> +
> + ExitOnErr(Executor->disconnect());
> + return 0;
> +}
>
> diff --git
> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
> new file mode 100644
> index 000000000000..abce14de0fe0
> --- /dev/null
> +++
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
> @@ -0,0 +1,347 @@
> +//===-- RemoteJITUtils.cpp - Utilities for remote-JITing
> --------*- C++ -*-===//
> +//
> +// Part of the LLVM Project, under the Apache License v2.0 with
> LLVM Exceptions.
> +// See https://llvm.org/LICENSE.txt
> <https://llvm.org/LICENSE.txt> for license information.
> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "RemoteJITUtils.h"
> +
> +#include "llvm/ExecutionEngine/Orc/DebugObjectManagerPlugin.h"
> +#include "llvm/ExecutionEngine/Orc/OrcRPCTargetProcessControl.h"
> +#include "llvm/ExecutionEngine/Orc/Shared/RPCUtils.h"
> +#include "llvm/ExecutionEngine/Orc/TPCDebugObjectRegistrar.h"
> +#include
> "llvm/ExecutionEngine/Orc/TPCDynamicLibrarySearchGenerator.h"
> +#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"
> +#include "llvm/Support/FileSystem.h"
> +#include "llvm/Support/Path.h"
> +#include "llvm/Support/ToolOutputFile.h"
> +
> +#ifdef LLVM_ON_UNIX
> +#include <netdb.h>
> +#include <netinet/in.h>
> +#include <sys/socket.h>
> +#include <unistd.h>
> +#endif // LLVM_ON_UNIX
> +
> +using namespace llvm;
> +using namespace llvm::orc;
> +
> +namespace llvm {
> +namespace orc {
> +
> +class RemoteTargetProcessControl
> + : public OrcRPCTargetProcessControlBase<
> +
> shared::MultiThreadedRPCEndpoint<JITLinkExecutor::RPCChannel>> {
> +public:
> + using RPCChannel = JITLinkExecutor::RPCChannel;
> + using RPCEndpoint = shared::MultiThreadedRPCEndpoint<RPCChannel>;
> +
> +private:
> + using ThisT = RemoteTargetProcessControl;
> + using BaseT = OrcRPCTargetProcessControlBase<RPCEndpoint>;
> + using MemoryAccess = OrcRPCTPCMemoryAccess<ThisT>;
> + using MemoryManager = OrcRPCTPCJITLinkMemoryManager<ThisT>;
> +
> +public:
> + using BaseT::initializeORCRPCTPCBase;
> +
> + RemoteTargetProcessControl(ExecutionSession &ES,
> + std::unique_ptr<RPCChannel> Channel,
> + std::unique_ptr<RPCEndpoint> Endpoint);
> +
> + void initializeMemoryManagement();
> + Error disconnect() override;
> +
> +private:
> + std::unique_ptr<RPCChannel> Channel;
> + std::unique_ptr<RPCEndpoint> Endpoint;
> + std::unique_ptr<MemoryAccess> OwnedMemAccess;
> + std::unique_ptr<MemoryManager> OwnedMemMgr;
> + std::atomic<bool> Finished{false};
> + std::thread ListenerThread;
> +};
> +
> +RemoteTargetProcessControl::RemoteTargetProcessControl(
> + ExecutionSession &ES, std::unique_ptr<RPCChannel> Channel,
> + std::unique_ptr<RPCEndpoint> Endpoint)
> + : BaseT(ES.getSymbolStringPool(), *Endpoint,
> + [&ES](Error Err) { ES.reportError(std::move(Err)); }),
> + Channel(std::move(Channel)), Endpoint(std::move(Endpoint)) {
> +
> + ListenerThread = std::thread([&]() {
> + while (!Finished) {
> + if (auto Err = this->Endpoint->handleOne()) {
> + reportError(std::move(Err));
> + return;
> + }
> + }
> + });
> +}
> +
> +void RemoteTargetProcessControl::initializeMemoryManagement() {
> + OwnedMemAccess = std::make_unique<MemoryAccess>(*this);
> + OwnedMemMgr = std::make_unique<MemoryManager>(*this);
> +
> + // Base class needs non-owning access.
> + MemAccess = OwnedMemAccess.get();
> + MemMgr = OwnedMemMgr.get();
> +}
> +
> +Error RemoteTargetProcessControl::disconnect() {
> + std::promise<MSVCPError> P;
> + auto F = P.get_future();
> + auto Err = closeConnection([&](Error Err) -> Error {
> + P.set_value(std::move(Err));
> + Finished = true;
> + return Error::success();
> + });
> + ListenerThread.join();
> + return joinErrors(std::move(Err), F.get());
> +}
> +
> +} // namespace orc
> +} // namespace llvm
> +
> +JITLinkExecutor::JITLinkExecutor() = default;
> +JITLinkExecutor::~JITLinkExecutor() = default;
> +
> +Expected<std::unique_ptr<ObjectLayer>>
> +JITLinkExecutor::operator()(ExecutionSession &ES, const Triple &TT) {
> + return std::make_unique<ObjectLinkingLayer>(ES, TPC->getMemMgr());
> +}
> +
> +Error JITLinkExecutor::addDebugSupport(ObjectLayer &ObjLayer) {
> + auto Registrar = createJITLoaderGDBRegistrar(*TPC);
> + if (!Registrar)
> + return Registrar.takeError();
> +
> + cast<ObjectLinkingLayer>(&ObjLayer)->addPlugin(
> +
> std::make_unique<DebugObjectManagerPlugin>(ObjLayer.getExecutionSession(),
> +
> std::move(*Registrar)));
> +
> + return Error::success();
> +}
> +
> +Expected<std::unique_ptr<DefinitionGenerator>>
> +JITLinkExecutor::loadDylib(StringRef RemotePath) {
> + if (auto Handle = TPC->loadDylib(RemotePath.data()))
> + return
> std::make_unique<TPCDynamicLibrarySearchGenerator>(*TPC, *Handle);
> + else
> + return Handle.takeError();
> +}
> +
> +Expected<int> JITLinkExecutor::runAsMain(JITEvaluatedSymbol MainSym,
> + ArrayRef<std::string>
> Args) {
> + return TPC->runAsMain(MainSym.getAddress(), Args);
> +}
> +
> +Error JITLinkExecutor::disconnect() { return TPC->disconnect(); }
> +
> +static std::string defaultPath(const char *HostArgv0, StringRef
> ExecutorName) {
> + // This just needs to be some symbol in the binary; C++ doesn't
> + // allow taking the address of ::main however.
> + void *P = (void *)(intptr_t)defaultPath;
> + SmallString<256> FullName(sys::fs::getMainExecutable(HostArgv0,
> P));
> + sys::path::remove_filename(FullName);
> + sys::path::append(FullName, ExecutorName);
> + return FullName.str().str();
> +}
> +
> +Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
> +JITLinkExecutor::FindLocal(const char *HostArgv) {
> + std::string BestGuess = defaultPath(HostArgv,
> "llvm-jitlink-executor");
> + auto Executor = CreateLocal(BestGuess);
> + if (!Executor) {
> + consumeError(Executor.takeError());
> + return make_error<StringError>(
> + formatv("Unable to find usable executor: {0}", BestGuess),
> + inconvertibleErrorCode());
> + }
> + return Executor;
> +}
> +
> +Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
> +JITLinkExecutor::CreateLocal(std::string ExecutablePath) {
> + if (!sys::fs::can_execute(ExecutablePath))
> + return make_error<StringError>(
> + formatv("Specified executor invalid: {0}", ExecutablePath),
> + inconvertibleErrorCode());
> + return std::unique_ptr<ChildProcessJITLinkExecutor>(
> + new ChildProcessJITLinkExecutor(std::move(ExecutablePath)));
> +}
> +
> +TCPSocketJITLinkExecutor::TCPSocketJITLinkExecutor(
> + std::unique_ptr<RemoteTargetProcessControl> TPC) {
> + this->TPC = std::move(TPC);
> +}
> +
> +#ifndef LLVM_ON_UNIX
> +
> +// FIXME: Add support for Windows.
> +Error ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
> + return make_error<StringError>(
> + "Remote JITing not yet supported on non-unix platforms",
> + inconvertibleErrorCode());
> +}
> +
> +// FIXME: Add support for Windows.
> +Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
> +JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
> + ExecutionSession &ES) {
> + return make_error<StringError>(
> + "Remote JITing not yet supported on non-unix platforms",
> + inconvertibleErrorCode());
> +}
> +
> +#else
> +
> +Error ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
> + constexpr int ReadEnd = 0;
> + constexpr int WriteEnd = 1;
> +
> + // Pipe FDs.
> + int ToExecutor[2];
> + int FromExecutor[2];
> +
> + // Create pipes to/from the executor..
> + if (pipe(ToExecutor) != 0 || pipe(FromExecutor) != 0)
> + return make_error<StringError>("Unable to create pipe for
> executor",
> + inconvertibleErrorCode());
> +
> + ProcessID = fork();
> + if (ProcessID == 0) {
> + // In the child...
> +
> + // Close the parent ends of the pipes
> + close(ToExecutor[WriteEnd]);
> + close(FromExecutor[ReadEnd]);
> +
> + // Execute the child process.
> + std::unique_ptr<char[]> ExecPath, FDSpecifier;
> + {
> + ExecPath = std::make_unique<char[]>(ExecutablePath.size() + 1);
> + strcpy(ExecPath.get(), ExecutablePath.data());
> +
> + std::string FDSpecifierStr("filedescs=");
> + FDSpecifierStr += utostr(ToExecutor[ReadEnd]);
> + FDSpecifierStr += ',';
> + FDSpecifierStr += utostr(FromExecutor[WriteEnd]);
> + FDSpecifier =
> std::make_unique<char[]>(FDSpecifierStr.size() + 1);
> + strcpy(FDSpecifier.get(), FDSpecifierStr.c_str());
> + }
> +
> + char *const Args[] = {ExecPath.get(), FDSpecifier.get(),
> nullptr};
> + int RC = execvp(ExecPath.get(), Args);
> + if (RC != 0)
> + return make_error<StringError>(
> + "Unable to launch out-of-process executor '" +
> ExecutablePath + "'\n",
> + inconvertibleErrorCode());
> +
> + llvm_unreachable("Fork won't return in success case");
> + }
> + // else we're the parent...
> +
> + // Close the child ends of the pipes
> + close(ToExecutor[ReadEnd]);
> + close(FromExecutor[WriteEnd]);
> +
> + auto Channel =
> + std::make_unique<RPCChannel>(FromExecutor[ReadEnd],
> ToExecutor[WriteEnd]);
> + auto Endpoint =
> +
> std::make_unique<RemoteTargetProcessControl::RPCEndpoint>(*Channel,
> true);
> +
> + TPC = std::make_unique<RemoteTargetProcessControl>(ES,
> std::move(Channel),
> +
> std::move(Endpoint));
> +
> + if (auto Err = TPC->initializeORCRPCTPCBase())
> + return joinErrors(std::move(Err), TPC->disconnect());
> +
> + TPC->initializeMemoryManagement();
> +
> + shared::registerStringError<RPCChannel>();
> + return Error::success();
> +}
> +
> +static Expected<int> connectTCPSocketImpl(std::string Host,
> + std::string PortStr) {
> + addrinfo *AI;
> + addrinfo Hints{};
> + Hints.ai_family = AF_INET;
> + Hints.ai_socktype = SOCK_STREAM;
> + Hints.ai_flags = AI_NUMERICSERV;
> +
> + if (int EC = getaddrinfo(Host.c_str(), PortStr.c_str(), &Hints,
> &AI))
> + return make_error<StringError>(
> + formatv("address resolution failed ({0})", gai_strerror(EC)),
> + inconvertibleErrorCode());
> +
> + // Cycle through the returned addrinfo structures and connect
> to the first
> + // reachable endpoint.
> + int SockFD;
> + addrinfo *Server;
> + for (Server = AI; Server != nullptr; Server = Server->ai_next) {
> + // If socket fails, maybe it's because the address family is
> not supported.
> + // Skip to the next addrinfo structure.
> + if ((SockFD = socket(AI->ai_family, AI->ai_socktype,
> AI->ai_protocol)) < 0)
> + continue;
> +
> + // If connect works, we exit the loop with a working socket.
> + if (connect(SockFD, Server->ai_addr, Server->ai_addrlen) == 0)
> + break;
> +
> + close(SockFD);
> + }
> + freeaddrinfo(AI);
> +
> + // Did we reach the end of the loop without connecting to a
> valid endpoint?
> + if (Server == nullptr)
> + return make_error<StringError>("invalid hostname",
> + inconvertibleErrorCode());
> +
> + return SockFD;
> +}
> +
> +Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
> +JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
> + ExecutionSession &ES) {
> + auto CreateErr = [NetworkAddress](StringRef Details) {
> + return make_error<StringError>(
> + formatv("Failed to connect TCP socket '{0}': {1}",
> NetworkAddress,
> + Details),
> + inconvertibleErrorCode());
> + };
> +
> + StringRef Host, PortStr;
> + std::tie(Host, PortStr) = NetworkAddress.split(':');
> + if (Host.empty())
> + return CreateErr("host name cannot be empty");
> + if (PortStr.empty())
> + return CreateErr("port cannot be empty");
> + int Port = 0;
> + if (PortStr.getAsInteger(10, Port))
> + return CreateErr("port number is not a valid integer");
> +
> + Expected<int> SockFD = connectTCPSocketImpl(Host.str(),
> PortStr.str());
> + if (!SockFD)
> + return CreateErr(toString(SockFD.takeError()));
> +
> + auto Channel = std::make_unique<RPCChannel>(*SockFD, *SockFD);
> + auto Endpoint =
> +
> std::make_unique<RemoteTargetProcessControl::RPCEndpoint>(*Channel,
> true);
> +
> + auto TPC = std::make_unique<RemoteTargetProcessControl>(
> + ES, std::move(Channel), std::move(Endpoint));
> +
> + if (auto Err = TPC->initializeORCRPCTPCBase())
> + return joinErrors(std::move(Err), TPC->disconnect());
> +
> + TPC->initializeMemoryManagement();
> + shared::registerStringError<RPCChannel>();
> +
> + return std::unique_ptr<TCPSocketJITLinkExecutor>(
> + new TCPSocketJITLinkExecutor(std::move(TPC)));
> +}
> +
> +#endif
>
> diff --git
> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
> new file mode 100644
> index 000000000000..e629c0e036f5
> --- /dev/null
> +++
> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
> @@ -0,0 +1,111 @@
> +//===-- RemoteJITUtils.h - Utilities for remote-JITing
> ----------*- C++ -*-===//
> +//
> +// Part of the LLVM Project, under the Apache License v2.0 with
> LLVM Exceptions.
> +// See https://llvm.org/LICENSE.txt
> <https://llvm.org/LICENSE.txt> for license information.
> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// Utilities for TargetProcessControl-based remote JITing with
> Orc and JITLink.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef
> LLVM_EXAMPLES_ORCV2EXAMPLES_LLJITWITHREMOTEDEBUGGING_REMOTEJITUTILS_H
> +#define
> LLVM_EXAMPLES_ORCV2EXAMPLES_LLJITWITHREMOTEDEBUGGING_REMOTEJITUTILS_H
> +
> +#include "llvm/ADT/ArrayRef.h"
> +#include "llvm/ADT/StringRef.h"
> +#include "llvm/ADT/Triple.h"
> +#include "llvm/ExecutionEngine/JITSymbol.h"
> +#include "llvm/ExecutionEngine/Orc/Core.h"
> +#include "llvm/ExecutionEngine/Orc/Layer.h"
> +#include "llvm/ExecutionEngine/Orc/ObjectTransformLayer.h"
> +#include "llvm/ExecutionEngine/Orc/Shared/FDRawByteChannel.h"
> +#include "llvm/Support/Error.h"
> +
> +#include <memory>
> +#include <string>
> +
> +#if !defined(_MSC_VER) && !defined(__MINGW32__)
> +#include <unistd.h>
> +#else
> +#include <io.h>
> +#endif
> +
> +namespace llvm {
> +namespace orc {
> +
> +class ChildProcessJITLinkExecutor;
> +class RemoteTargetProcessControl;
> +class TCPSocketJITLinkExecutor;
> +
> +class JITLinkExecutor {
> +public:
> + using RPCChannel = shared::FDRawByteChannel;
> +
> + /// Create a JITLinkExecutor for the given exectuable on disk.
> + static Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
> + CreateLocal(std::string ExecutablePath);
> +
> + /// Find the default exectuable on disk and create a
> JITLinkExecutor for it.
> + static Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
> + FindLocal(const char *JITArgv0);
> +
> + /// Create a JITLinkExecutor that connects to the given network
> address
> + /// through a TCP socket. A valid NetworkAddress provides
> hostname and port,
> + /// e.g. localhost:20000.
> + static Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
> + ConnectTCPSocket(StringRef NetworkAddress, ExecutionSession &ES);
> +
> + // Implement ObjectLinkingLayerCreator
> + Expected<std::unique_ptr<ObjectLayer>>
> operator()(ExecutionSession &,
> + const Triple &);
> +
> + Error addDebugSupport(ObjectLayer &ObjLayer);
> +
> + Expected<std::unique_ptr<DefinitionGenerator>>
> + loadDylib(StringRef RemotePath);
> +
> + Expected<int> runAsMain(JITEvaluatedSymbol MainSym,
> + ArrayRef<std::string> Args);
> + Error disconnect();
> +
> + virtual ~JITLinkExecutor();
> +
> +protected:
> + std::unique_ptr<RemoteTargetProcessControl> TPC;
> +
> + JITLinkExecutor();
> +};
> +
> +/// JITLinkExecutor that runs in a child process on the local
> machine.
> +class ChildProcessJITLinkExecutor : public JITLinkExecutor {
> +public:
> + Error launch(ExecutionSession &ES);
> +
> + pid_t getPID() const { return ProcessID; }
> + StringRef getPath() const { return ExecutablePath; }
> +
> +private:
> + std::string ExecutablePath;
> + pid_t ProcessID;
> +
> + ChildProcessJITLinkExecutor(std::string ExecutablePath)
> + : ExecutablePath(std::move(ExecutablePath)) {}
> +
> + static std::string defaultPath(const char *HostArgv0, StringRef
> ExecutorName);
> + friend class JITLinkExecutor;
> +};
> +
> +/// JITLinkExecutor connected through a TCP socket.
> +class TCPSocketJITLinkExecutor : public JITLinkExecutor {
> +private:
> +
> TCPSocketJITLinkExecutor(std::unique_ptr<RemoteTargetProcessControl>
> TPC);
> +
> + friend class JITLinkExecutor;
> +};
> +
> +} // namespace orc
> +} // namespace llvm
> +
> +#endif
>
> diff --git a/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
> b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
> new file mode 100644
> index 000000000000..3a5c2bcefee1
> --- /dev/null
> +++ b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
> @@ -0,0 +1,2 @@
> +int sub1(int x) { return x - 1; }
> +int main(int argc, char **argv) { return sub1(argc); }
>
> diff --git
> a/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
> b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
> new file mode 100644
> index 000000000000..659dbe109ec6
> --- /dev/null
> +++ b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
> @@ -0,0 +1,52 @@
> +; ModuleID = 'argc_sub1.c'
> +target triple = "x86_64-unknown-unknown-elf"
> +
> +define i32 @sub1(i32) !dbg !8 {
> + call void @llvm.dbg.value(metadata i32 %0, metadata !13,
> metadata !DIExpression()), !dbg !14
> + %2 = add nsw i32 %0, -1, !dbg !15
> + ret i32 %2, !dbg !16
> +}
> +
> +define i32 @main(i32, i8** nocapture readnone) !dbg !17 {
> + call void @llvm.dbg.value(metadata i32 %0, metadata !24,
> metadata !DIExpression()), !dbg !26
> + call void @llvm.dbg.value(metadata i8** %1, metadata !25,
> metadata !DIExpression()), !dbg !27
> + %3 = tail call i32 @sub1(i32 %0), !dbg !28
> + ret i32 %3, !dbg !29
> +}
> +
> +declare void @llvm.dbg.value(metadata, metadata, metadata)
> +
> +!llvm.dbg.cu <http://llvm.dbg.cu> = !{!0}
> +!llvm.module.flags = !{!3, !4, !5, !6}
> +!llvm.ident = !{!7}
> +
> +!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1,
> producer: "clang version 7.0.1-8+deb10u2
> (tags/RELEASE_701/final)", isOptimized: true, runtimeVersion: 0,
> emissionKind: FullDebug, enums: !2)
> +!1 = !DIFile(filename: "argc_sub1.c", directory: "Inputs/")
> +!2 = !{}
> +!3 = !{i32 2, !"Dwarf Version", i32 4}
> +!4 = !{i32 2, !"Debug Info Version", i32 3}
> +!5 = !{i32 1, !"wchar_size", i32 4}
> +!6 = !{i32 7, !"PIC Level", i32 2}
> +!7 = !{!"clang version 7.0.1-8+deb10u2 (tags/RELEASE_701/final)"}
> +!8 = distinct !DISubprogram(name: "sub1", scope: !1, file: !1,
> line: 1, type: !9, isLocal: false, isDefinition: true, scopeLine:
> 1, flags: DIFlagPrototyped, isOptimized: true, unit: !0,
> retainedNodes: !12)
> +!9 = !DISubroutineType(types: !10)
> +!10 = !{!11, !11}
> +!11 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
> +!12 = !{!13}
> +!13 = !DILocalVariable(name: "x", arg: 1, scope: !8, file: !1,
> line: 1, type: !11)
> +!14 = !DILocation(line: 1, column: 14, scope: !8)
> +!15 = !DILocation(line: 1, column: 28, scope: !8)
> +!16 = !DILocation(line: 1, column: 19, scope: !8)
> +!17 = distinct !DISubprogram(name: "main", scope: !1, file: !1,
> line: 2, type: !18, isLocal: false, isDefinition: true, scopeLine:
> 2, flags: DIFlagPrototyped, isOptimized: true, unit: !0,
> retainedNodes: !23)
> +!18 = !DISubroutineType(types: !19)
> +!19 = !{!11, !11, !20}
> +!20 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !21,
> size: 64)
> +!21 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !22,
> size: 64)
> +!22 = !DIBasicType(name: "char", size: 8, encoding:
> DW_ATE_signed_char)
> +!23 = !{!24, !25}
> +!24 = !DILocalVariable(name: "argc", arg: 1, scope: !17, file:
> !1, line: 2, type: !11)
> +!25 = !DILocalVariable(name: "argv", arg: 2, scope: !17, file:
> !1, line: 2, type: !20)
> +!26 = !DILocation(line: 2, column: 14, scope: !17)
> +!27 = !DILocation(line: 2, column: 27, scope: !17)
> +!28 = !DILocation(line: 2, column: 42, scope: !17)
> +!29 = !DILocation(line: 2, column: 35, scope: !17)
>
> diff --git
> a/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
> b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
> new file mode 100644
> index 000000000000..a9c3946b55e2
> --- /dev/null
> +++
> b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
> @@ -0,0 +1,10 @@
> +# This test makes sure that the example builds and executes as
> expected.
> +# Instructions for debugging can be found in
> LLJITWithRemoteDebugging.cpp
> +
> +# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll
> +# CHECK: Running: {{.*}}/Inputs/argc_sub1_elf.ll
> +# CHECK: Exit code: 0
> +
> +# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll --args
> 2nd 3rd 4th
> +# CHECK: Running: {{.*}}/Inputs/argc_sub1_elf.ll 2nd 3rd 4th
> +# CHECK: Exit code: 3
>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits>
>
--
https://flowcrypt.com/pub/stefan.graenitz@gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210331/81eddde5/attachment.html>
More information about the llvm-commits
mailing list