[llvm] 258f055 - [Orc][examples] Add LLJITWithRemoteDebugging example
David Blaikie via llvm-commits
llvm-commits at lists.llvm.org
Sat Apr 3 00:30:40 PDT 2021
On Wed, Mar 31, 2021 at 1:34 AM Stefan Gränitz <stefan.graenitz at gmail.com>
wrote:
>
> Hi David, thanks for working on it!
>
> I ran the test suite on Linux and macOS with examples enabled and it
worked well.
Did you modify an existing build directory that had already built the
examples from some other invocation?
If you do these things:
1) enable examples in your CMakeCache.txt:
LLVM_BUILD_EXAMPLES:BOOL=ON
LLVM_INCLUDE_EXAMPLES:BOOL=ON
2) Delete any already built version of the jit binary:
build$ rm -f ./bin/LLJITWithRemoteDebugging
3) Undo ae217bf1f3277b8c14590a130ee5e63cc1664443 by making this change:
diff --git llvm/test/CMakeLists.txt llvm/test/CMakeLists.txt
index 81107be862de..7c4fa2e9033a 100644
--- llvm/test/CMakeLists.txt
+++ llvm/test/CMakeLists.txt
@@ -158,11 +158,6 @@ if(LLVM_BUILD_EXAMPLES)
Kaleidoscope-Ch7
LLJITWithThinLTOSummaries
)
- if(CMAKE_HOST_UNIX)
- list(APPEND LLVM_TEST_DEPENDS
- LLJITWithRemoteDebugging
- )
- endif()
if (NOT WIN32)
list(APPEND LLVM_TEST_DEPENDS
Bye
then run `ninja check-llvm-examples` (or make, etc - though perhaps ninja
is part of the necessary repro steps, I guess - I haven't tried with make)
- does it fail for you?
> Also, I didn't get a message from buildmaster for this. Do we have a bot
for your setup and is it public?
Not sure - I guess probably not. We probably should have something building
with the examples enabled.
>
> On 30/03/2021 22:05, David Blaikie wrote:
>
> This broke the build with examples enabled (LLVM_BUILD_EXAMPLES=ON) I
think. My second attempt at fixing this is in
ae217bf1f3277b8c14590a130ee5e63cc1664443
>
> On Sun, Mar 28, 2021 at 8:25 AM Stefan Gränitz via llvm-commits <
llvm-commits at lists.llvm.org> wrote:
>>
>>
>> Author: Stefan Gränitz
>> Date: 2021-03-28T17:25:09+02:00
>> New Revision: 258f055ed93661900bc568350e09f467c0950486
>>
>> URL:
https://github.com/llvm/llvm-project/commit/258f055ed93661900bc568350e09f467c0950486
>> DIFF:
https://github.com/llvm/llvm-project/commit/258f055ed93661900bc568350e09f467c0950486.diff
>>
>> LOG: [Orc][examples] Add LLJITWithRemoteDebugging example
>>
>> Added:
>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>
llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>
llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>> llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
>> llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
>> llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>
>> Modified:
>> llvm/examples/OrcV2Examples/CMakeLists.txt
>> llvm/examples/OrcV2Examples/ExampleModules.h
>>
>> Removed:
>>
>>
>>
>>
################################################################################
>> diff --git a/llvm/examples/OrcV2Examples/CMakeLists.txt
b/llvm/examples/OrcV2Examples/CMakeLists.txt
>> index bed277e59e0b..f6a11d6f0ef2 100644
>> --- a/llvm/examples/OrcV2Examples/CMakeLists.txt
>> +++ b/llvm/examples/OrcV2Examples/CMakeLists.txt
>> @@ -12,3 +12,7 @@ add_subdirectory(OrcV2CBindingsAddObjectFile)
>> add_subdirectory(OrcV2CBindingsBasicUsage)
>> add_subdirectory(OrcV2CBindingsReflectProcessSymbols)
>> add_subdirectory(OrcV2CBindingsRemovableCode)
>> +
>> +if(CMAKE_HOST_UNIX)
>> + add_subdirectory(LLJITWithRemoteDebugging)
>> +endif()
>>
>> diff --git a/llvm/examples/OrcV2Examples/ExampleModules.h
b/llvm/examples/OrcV2Examples/ExampleModules.h
>> index c88609fae769..53da756e15f7 100644
>> --- a/llvm/examples/OrcV2Examples/ExampleModules.h
>> +++ b/llvm/examples/OrcV2Examples/ExampleModules.h
>> @@ -52,4 +52,16 @@ parseExampleModule(llvm::StringRef Source,
llvm::StringRef Name) {
>> return createSMDiagnosticError(Err);
>> }
>>
>> +inline llvm::Expected<llvm::orc::ThreadSafeModule>
>> +parseExampleModuleFromFile(llvm::StringRef FileName) {
>> + using namespace llvm;
>> + auto Ctx = std::make_unique<LLVMContext>();
>> + SMDiagnostic Err;
>> +
>> + if (auto M = parseIRFile(FileName, Err, *Ctx))
>> + return orc::ThreadSafeModule(std::move(M), std::move(Ctx));
>> +
>> + return createSMDiagnosticError(Err);
>> +}
>> +
>> #endif // LLVM_EXAMPLES_ORCV2EXAMPLES_EXAMPLEMODULES_H
>>
>> diff --git
a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>> new file mode 100644
>> index 000000000000..b93dd4924b05
>> --- /dev/null
>> +++ b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>> @@ -0,0 +1,18 @@
>> +set(LLVM_LINK_COMPONENTS
>> + Core
>> + ExecutionEngine
>> + IRReader
>> + JITLink
>> + OrcJIT
>> + OrcTargetProcess
>> + Support
>> + nativecodegen
>> + )
>> +
>> +add_llvm_example(LLJITWithRemoteDebugging
>> + LLJITWithRemoteDebugging.cpp
>> + RemoteJITUtils.cpp
>> +
>> + DEPENDS
>> + llvm-jitlink-executor
>> + )
>>
>> diff --git
a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>> new file mode 100644
>> index 000000000000..6da30f608206
>> --- /dev/null
>> +++
b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>> @@ -0,0 +1,258 @@
>> +//===--- LLJITWithRemoteDebugging.cpp - LLJIT targeting a child process
---===//
>> +//
>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
>> +// See https://llvm.org/LICENSE.txt for license information.
>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>> +//
>>
+//===----------------------------------------------------------------------===//
>> +//
>> +// This example shows how to use LLJIT and JITLink for out-of-process
execution
>> +// with debug support. A few notes beforehand:
>> +//
>> +// * Debuggers must implement the GDB JIT interface (gdb, udb, lldb
12+).
>> +// * Debug support is currently limited to ELF on x86-64 platforms
that run
>> +// Unix-like systems.
>> +// * There is a test for this example and it ships an IR file that is
prepared
>> +// for the instructions below.
>> +//
>> +//
>> +// The following command line session provides a complete walkthrough
of the
>> +// feature using LLDB 12:
>> +//
>> +// [Terminal 1] Prepare a debuggable out-of-process JIT session:
>> +//
>> +// > cd llvm-project/build
>> +// > ninja LLJITWithRemoteDebugging llvm-jitlink-executor
>> +// > cp ../llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll .
>> +// > bin/LLJITWithRemoteDebugging --wait-for-debugger
argc_sub1_elf.ll
>> +// Found out-of-process executor: bin/llvm-jitlink-executor
>> +// Launched executor in subprocess: 65535
>> +// Attach a debugger and press any key to continue.
>> +//
>> +//
>> +// [Terminal 2] Attach a debugger to the child process:
>> +//
>> +// (lldb) log enable lldb jit
>> +// (lldb) settings set plugin.jit-loader.gdb.enable on
>> +// (lldb) settings set target.source-map Inputs/ \
>> +//
/path/to/llvm-project/llvm/test/Examples/OrcV2Examples/Inputs/
>> +// (lldb) attach -p 65535
>> +// JITLoaderGDB::SetJITBreakpoint looking for JIT register hook
>> +// JITLoaderGDB::SetJITBreakpoint setting JIT breakpoint
>> +// Process 65535 stopped
>> +// (lldb) b sub1
>> +// Breakpoint 1: no locations (pending).
>> +// WARNING: Unable to resolve breakpoint to any actual locations.
>> +// (lldb) c
>> +// Process 65535 resuming
>> +//
>> +//
>> +// [Terminal 1] Press a key to start code generation and execution:
>> +//
>> +// Parsed input IR code from: argc_sub1_elf.ll
>> +// Initialized LLJIT for remote executor
>> +// Running: argc_sub1_elf.ll
>> +//
>> +//
>> +// [Terminal 2] Breakpoint hits; we change the argc value from 1 to 42:
>> +//
>> +// (lldb) JITLoaderGDB::JITDebugBreakpointHit hit JIT breakpoint
>> +// JITLoaderGDB::ReadJITDescriptorImpl registering JIT entry at
0x106b34000
>> +// 1 location added to breakpoint 1
>> +// Process 65535 stopped
>> +// * thread #1, queue = 'com.apple.main-thread', stop reason =
breakpoint 1.1
>> +// frame #0: JIT(0x106b34000)`sub1(x=1) at argc_sub1.c:1:28
>> +// -> 1 int sub1(int x) { return x - 1; }
>> +// 2 int main(int argc, char **argv) { return sub1(argc); }
>> +// (lldb) p x
>> +// (int) $0 = 1
>> +// (lldb) expr x = 42
>> +// (int) $1 = 42
>> +// (lldb) c
>> +//
>> +//
>> +// [Terminal 1] Example output reflects the modified value:
>> +//
>> +// Exit code: 41
>> +//
>>
+//===----------------------------------------------------------------------===//
>> +
>> +#include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
>> +#include "llvm/ExecutionEngine/Orc/LLJIT.h"
>> +#include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
>> +#include "llvm/Support/CommandLine.h"
>> +#include "llvm/Support/Error.h"
>> +#include "llvm/Support/FormatVariadic.h"
>> +#include "llvm/Support/InitLLVM.h"
>> +#include "llvm/Support/TargetSelect.h"
>> +#include "llvm/Support/raw_ostream.h"
>> +
>> +#include "../ExampleModules.h"
>> +#include "RemoteJITUtils.h"
>> +
>> +#include <memory>
>> +#include <string>
>> +
>> +using namespace llvm;
>> +using namespace llvm::orc;
>> +
>> +// The LLVM IR file to run.
>> +static cl::list<std::string> InputFiles(cl::Positional, cl::OneOrMore,
>> + cl::desc("<input files>"));
>> +
>> +// Command line arguments to pass to the JITed main function.
>> +static cl::list<std::string> InputArgv("args", cl::Positional,
>> + cl::desc("<program
arguments>..."),
>> + cl::ZeroOrMore,
cl::PositionalEatsArgs);
>> +
>> +// Given paths must exist on the remote target.
>> +static cl::list<std::string>
>> + Dylibs("dlopen", cl::desc("Dynamic libraries to load before
linking"),
>> + cl::value_desc("filename"), cl::ZeroOrMore);
>> +
>> +// File path of the executable to launch for execution in a child
process.
>> +// Inter-process communication will go through stdin/stdout pipes.
>> +static cl::opt<std::string>
>> + OOPExecutor("executor", cl::desc("Set the out-of-process executor"),
>> + cl::value_desc("filename"));
>> +
>> +// Network address of a running executor process that we can connected
through a
>> +// TCP socket. It may run locally or on a remote machine.
>> +static cl::opt<std::string> OOPExecutorConnect(
>> + "connect",
>> + cl::desc("Connect to an out-of-process executor through a TCP
socket"),
>> + cl::value_desc("<hostname>:<port>"));
>> +
>> +// Give the user a chance to connect a debugger. Once we connected the
executor
>> +// process, wait for the user to press a key (and print out its PID if
it's a
>> +// child process).
>> +static cl::opt<bool>
>> + WaitForDebugger("wait-for-debugger",
>> + cl::desc("Wait for user input before entering JITed
code"),
>> + cl::init(false));
>> +
>> +ExitOnError ExitOnErr;
>> +
>> +static std::unique_ptr<JITLinkExecutor> connectExecutor(const char
*Argv0,
>> +
ExecutionSession &ES) {
>> + // Connect to a running out-of-process executor through a TCP socket.
>> + if (!OOPExecutorConnect.empty()) {
>> + std::unique_ptr<TCPSocketJITLinkExecutor> Exec =
>> + ExitOnErr(JITLinkExecutor::ConnectTCPSocket(OOPExecutorConnect,
ES));
>> +
>> + outs() << "Connected to executor at " << OOPExecutorConnect << "\n";
>> + if (WaitForDebugger) {
>> + outs() << "Attach a debugger and press any key to continue.\n";
>> + fflush(stdin);
>> + getchar();
>> + }
>> +
>> + return std::move(Exec);
>> + }
>> +
>> + // Launch a out-of-process executor locally in a child process.
>> + std::unique_ptr<ChildProcessJITLinkExecutor> Exec = ExitOnErr(
>> + OOPExecutor.empty() ? JITLinkExecutor::FindLocal(Argv0)
>> + : JITLinkExecutor::CreateLocal(OOPExecutor));
>> +
>> + outs() << "Found out-of-process executor: " << Exec->getPath() <<
"\n";
>> +
>> + ExitOnErr(Exec->launch(ES));
>> + if (WaitForDebugger) {
>> + outs() << "Launched executor in subprocess: " << Exec->getPID() <<
"\n"
>> + << "Attach a debugger and press any key to continue.\n";
>> + fflush(stdin);
>> + getchar();
>> + }
>> +
>> + return std::move(Exec);
>> +}
>> +
>> +int main(int argc, char *argv[]) {
>> + InitLLVM X(argc, argv);
>> +
>> + InitializeNativeTarget();
>> + InitializeNativeTargetAsmPrinter();
>> +
>> + ExitOnErr.setBanner(std::string(argv[0]) + ": ");
>> + cl::ParseCommandLineOptions(argc, argv, "LLJITWithRemoteDebugging");
>> +
>> + auto ES = std::make_unique<ExecutionSession>();
>> + ES->setErrorReporter([&](Error Err) { ExitOnErr(std::move(Err)); });
>> +
>> + // Launch/connect the out-of-process executor.
>> + std::unique_ptr<JITLinkExecutor> Executor = connectExecutor(argv[0],
*ES);
>> +
>> + // Load the given IR files.
>> + std::vector<ThreadSafeModule> TSMs;
>> + for (const std::string &Path : InputFiles) {
>> + outs() << "Parsing input IR code from: " << Path << "\n";
>> + TSMs.push_back(ExitOnErr(parseExampleModuleFromFile(Path)));
>> + }
>> +
>> + StringRef TT;
>> + StringRef MainModuleName;
>> + TSMs.front().withModuleDo([&MainModuleName, &TT](Module &M) {
>> + MainModuleName = M.getName();
>> + TT = M.getTargetTriple();
>> + });
>> +
>> + for (const ThreadSafeModule &TSM : TSMs)
>> + ExitOnErr(TSM.withModuleDo([TT, MainModuleName](Module &M) -> Error
{
>> + if (M.getTargetTriple() != TT)
>> + return make_error<StringError>(
>> + formatv("Different target triples in input files:\n"
>> + " '{0}' in '{1}'\n '{2}' in '{3}'",
>> + TT, MainModuleName, M.getTargetTriple(),
M.getName()),
>> + inconvertibleErrorCode());
>> + return Error::success();
>> + }));
>> +
>> + // Create a target machine that matches the input triple.
>> + JITTargetMachineBuilder JTMB((Triple(TT)));
>> + JTMB.setCodeModel(CodeModel::Small);
>> + JTMB.setRelocationModel(Reloc::PIC_);
>> +
>> + // Create LLJIT and destroy it before disconnecting the target
process.
>> + {
>> + outs() << "Initializing LLJIT for remote executor\n";
>> + auto J = ExitOnErr(LLJITBuilder()
>> + .setExecutionSession(std::move(ES))
>> + .setJITTargetMachineBuilder(std::move(JTMB))
>> +
.setObjectLinkingLayerCreator(std::ref(*Executor))
>> + .create());
>> +
>> + // Add plugin for debug support.
>> + ExitOnErr(Executor->addDebugSupport(J->getObjLinkingLayer()));
>> +
>> + // Load required shared libraries on the remote target and add a
generator
>> + // for each of it, so the compiler can lookup their symbols.
>> + for (const std::string &Path : Dylibs)
>> +
J->getMainJITDylib().addGenerator(ExitOnErr(Executor->loadDylib(Path)));
>> +
>> + // Add the loaded IR module to the JIT. This will set up symbol
tables and
>> + // prepare for materialization.
>> + for (ThreadSafeModule &TSM : TSMs)
>> + ExitOnErr(J->addIRModule(std::move(TSM)));
>> +
>> + // The example uses a non-lazy JIT for simplicity. Thus, looking up
the main
>> + // function will materialize all reachable code. It also triggers
debug
>> + // registration in the remote target process.
>> + JITEvaluatedSymbol MainFn = ExitOnErr(J->lookup("main"));
>> +
>> + outs() << "Running: main(";
>> + int Pos = 0;
>> + for (const std::string &Arg : InputArgv)
>> + outs() << (Pos++ == 0 ? "" : ", ") << Arg;
>> + outs() << ")\n";
>> +
>> + // Execute the code in the remote target process and dump the
result. With
>> + // the debugger attached to the target, it should be possible to
inspect the
>> + // JITed code as if it was compiled statically.
>> + int Result = ExitOnErr(Executor->runAsMain(MainFn, InputArgv));
>> + outs() << "Exit code: " << Result << "\n";
>> + }
>> +
>> + ExitOnErr(Executor->disconnect());
>> + return 0;
>> +}
>>
>> diff --git
a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>> new file mode 100644
>> index 000000000000..abce14de0fe0
>> --- /dev/null
>> +++
b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>> @@ -0,0 +1,347 @@
>> +//===-- RemoteJITUtils.cpp - Utilities for remote-JITing --------*- C++
-*-===//
>> +//
>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
>> +// See https://llvm.org/LICENSE.txt for license information.
>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>> +//
>>
+//===----------------------------------------------------------------------===//
>> +
>> +#include "RemoteJITUtils.h"
>> +
>> +#include "llvm/ExecutionEngine/Orc/DebugObjectManagerPlugin.h"
>> +#include "llvm/ExecutionEngine/Orc/OrcRPCTargetProcessControl.h"
>> +#include "llvm/ExecutionEngine/Orc/Shared/RPCUtils.h"
>> +#include "llvm/ExecutionEngine/Orc/TPCDebugObjectRegistrar.h"
>> +#include "llvm/ExecutionEngine/Orc/TPCDynamicLibrarySearchGenerator.h"
>> +#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"
>> +#include "llvm/Support/FileSystem.h"
>> +#include "llvm/Support/Path.h"
>> +#include "llvm/Support/ToolOutputFile.h"
>> +
>> +#ifdef LLVM_ON_UNIX
>> +#include <netdb.h>
>> +#include <netinet/in.h>
>> +#include <sys/socket.h>
>> +#include <unistd.h>
>> +#endif // LLVM_ON_UNIX
>> +
>> +using namespace llvm;
>> +using namespace llvm::orc;
>> +
>> +namespace llvm {
>> +namespace orc {
>> +
>> +class RemoteTargetProcessControl
>> + : public OrcRPCTargetProcessControlBase<
>> +
shared::MultiThreadedRPCEndpoint<JITLinkExecutor::RPCChannel>> {
>> +public:
>> + using RPCChannel = JITLinkExecutor::RPCChannel;
>> + using RPCEndpoint = shared::MultiThreadedRPCEndpoint<RPCChannel>;
>> +
>> +private:
>> + using ThisT = RemoteTargetProcessControl;
>> + using BaseT = OrcRPCTargetProcessControlBase<RPCEndpoint>;
>> + using MemoryAccess = OrcRPCTPCMemoryAccess<ThisT>;
>> + using MemoryManager = OrcRPCTPCJITLinkMemoryManager<ThisT>;
>> +
>> +public:
>> + using BaseT::initializeORCRPCTPCBase;
>> +
>> + RemoteTargetProcessControl(ExecutionSession &ES,
>> + std::unique_ptr<RPCChannel> Channel,
>> + std::unique_ptr<RPCEndpoint> Endpoint);
>> +
>> + void initializeMemoryManagement();
>> + Error disconnect() override;
>> +
>> +private:
>> + std::unique_ptr<RPCChannel> Channel;
>> + std::unique_ptr<RPCEndpoint> Endpoint;
>> + std::unique_ptr<MemoryAccess> OwnedMemAccess;
>> + std::unique_ptr<MemoryManager> OwnedMemMgr;
>> + std::atomic<bool> Finished{false};
>> + std::thread ListenerThread;
>> +};
>> +
>> +RemoteTargetProcessControl::RemoteTargetProcessControl(
>> + ExecutionSession &ES, std::unique_ptr<RPCChannel> Channel,
>> + std::unique_ptr<RPCEndpoint> Endpoint)
>> + : BaseT(ES.getSymbolStringPool(), *Endpoint,
>> + [&ES](Error Err) { ES.reportError(std::move(Err)); }),
>> + Channel(std::move(Channel)), Endpoint(std::move(Endpoint)) {
>> +
>> + ListenerThread = std::thread([&]() {
>> + while (!Finished) {
>> + if (auto Err = this->Endpoint->handleOne()) {
>> + reportError(std::move(Err));
>> + return;
>> + }
>> + }
>> + });
>> +}
>> +
>> +void RemoteTargetProcessControl::initializeMemoryManagement() {
>> + OwnedMemAccess = std::make_unique<MemoryAccess>(*this);
>> + OwnedMemMgr = std::make_unique<MemoryManager>(*this);
>> +
>> + // Base class needs non-owning access.
>> + MemAccess = OwnedMemAccess.get();
>> + MemMgr = OwnedMemMgr.get();
>> +}
>> +
>> +Error RemoteTargetProcessControl::disconnect() {
>> + std::promise<MSVCPError> P;
>> + auto F = P.get_future();
>> + auto Err = closeConnection([&](Error Err) -> Error {
>> + P.set_value(std::move(Err));
>> + Finished = true;
>> + return Error::success();
>> + });
>> + ListenerThread.join();
>> + return joinErrors(std::move(Err), F.get());
>> +}
>> +
>> +} // namespace orc
>> +} // namespace llvm
>> +
>> +JITLinkExecutor::JITLinkExecutor() = default;
>> +JITLinkExecutor::~JITLinkExecutor() = default;
>> +
>> +Expected<std::unique_ptr<ObjectLayer>>
>> +JITLinkExecutor::operator()(ExecutionSession &ES, const Triple &TT) {
>> + return std::make_unique<ObjectLinkingLayer>(ES, TPC->getMemMgr());
>> +}
>> +
>> +Error JITLinkExecutor::addDebugSupport(ObjectLayer &ObjLayer) {
>> + auto Registrar = createJITLoaderGDBRegistrar(*TPC);
>> + if (!Registrar)
>> + return Registrar.takeError();
>> +
>> + cast<ObjectLinkingLayer>(&ObjLayer)->addPlugin(
>> +
std::make_unique<DebugObjectManagerPlugin>(ObjLayer.getExecutionSession(),
>> +
std::move(*Registrar)));
>> +
>> + return Error::success();
>> +}
>> +
>> +Expected<std::unique_ptr<DefinitionGenerator>>
>> +JITLinkExecutor::loadDylib(StringRef RemotePath) {
>> + if (auto Handle = TPC->loadDylib(RemotePath.data()))
>> + return std::make_unique<TPCDynamicLibrarySearchGenerator>(*TPC,
*Handle);
>> + else
>> + return Handle.takeError();
>> +}
>> +
>> +Expected<int> JITLinkExecutor::runAsMain(JITEvaluatedSymbol MainSym,
>> + ArrayRef<std::string> Args) {
>> + return TPC->runAsMain(MainSym.getAddress(), Args);
>> +}
>> +
>> +Error JITLinkExecutor::disconnect() { return TPC->disconnect(); }
>> +
>> +static std::string defaultPath(const char *HostArgv0, StringRef
ExecutorName) {
>> + // This just needs to be some symbol in the binary; C++ doesn't
>> + // allow taking the address of ::main however.
>> + void *P = (void *)(intptr_t)defaultPath;
>> + SmallString<256> FullName(sys::fs::getMainExecutable(HostArgv0, P));
>> + sys::path::remove_filename(FullName);
>> + sys::path::append(FullName, ExecutorName);
>> + return FullName.str().str();
>> +}
>> +
>> +Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
>> +JITLinkExecutor::FindLocal(const char *HostArgv) {
>> + std::string BestGuess = defaultPath(HostArgv,
"llvm-jitlink-executor");
>> + auto Executor = CreateLocal(BestGuess);
>> + if (!Executor) {
>> + consumeError(Executor.takeError());
>> + return make_error<StringError>(
>> + formatv("Unable to find usable executor: {0}", BestGuess),
>> + inconvertibleErrorCode());
>> + }
>> + return Executor;
>> +}
>> +
>> +Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
>> +JITLinkExecutor::CreateLocal(std::string ExecutablePath) {
>> + if (!sys::fs::can_execute(ExecutablePath))
>> + return make_error<StringError>(
>> + formatv("Specified executor invalid: {0}", ExecutablePath),
>> + inconvertibleErrorCode());
>> + return std::unique_ptr<ChildProcessJITLinkExecutor>(
>> + new ChildProcessJITLinkExecutor(std::move(ExecutablePath)));
>> +}
>> +
>> +TCPSocketJITLinkExecutor::TCPSocketJITLinkExecutor(
>> + std::unique_ptr<RemoteTargetProcessControl> TPC) {
>> + this->TPC = std::move(TPC);
>> +}
>> +
>> +#ifndef LLVM_ON_UNIX
>> +
>> +// FIXME: Add support for Windows.
>> +Error ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
>> + return make_error<StringError>(
>> + "Remote JITing not yet supported on non-unix platforms",
>> + inconvertibleErrorCode());
>> +}
>> +
>> +// FIXME: Add support for Windows.
>> +Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
>> +JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
>> + ExecutionSession &ES) {
>> + return make_error<StringError>(
>> + "Remote JITing not yet supported on non-unix platforms",
>> + inconvertibleErrorCode());
>> +}
>> +
>> +#else
>> +
>> +Error ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
>> + constexpr int ReadEnd = 0;
>> + constexpr int WriteEnd = 1;
>> +
>> + // Pipe FDs.
>> + int ToExecutor[2];
>> + int FromExecutor[2];
>> +
>> + // Create pipes to/from the executor..
>> + if (pipe(ToExecutor) != 0 || pipe(FromExecutor) != 0)
>> + return make_error<StringError>("Unable to create pipe for executor",
>> + inconvertibleErrorCode());
>> +
>> + ProcessID = fork();
>> + if (ProcessID == 0) {
>> + // In the child...
>> +
>> + // Close the parent ends of the pipes
>> + close(ToExecutor[WriteEnd]);
>> + close(FromExecutor[ReadEnd]);
>> +
>> + // Execute the child process.
>> + std::unique_ptr<char[]> ExecPath, FDSpecifier;
>> + {
>> + ExecPath = std::make_unique<char[]>(ExecutablePath.size() + 1);
>> + strcpy(ExecPath.get(), ExecutablePath.data());
>> +
>> + std::string FDSpecifierStr("filedescs=");
>> + FDSpecifierStr += utostr(ToExecutor[ReadEnd]);
>> + FDSpecifierStr += ',';
>> + FDSpecifierStr += utostr(FromExecutor[WriteEnd]);
>> + FDSpecifier = std::make_unique<char[]>(FDSpecifierStr.size() + 1);
>> + strcpy(FDSpecifier.get(), FDSpecifierStr.c_str());
>> + }
>> +
>> + char *const Args[] = {ExecPath.get(), FDSpecifier.get(), nullptr};
>> + int RC = execvp(ExecPath.get(), Args);
>> + if (RC != 0)
>> + return make_error<StringError>(
>> + "Unable to launch out-of-process executor '" + ExecutablePath
+ "'\n",
>> + inconvertibleErrorCode());
>> +
>> + llvm_unreachable("Fork won't return in success case");
>> + }
>> + // else we're the parent...
>> +
>> + // Close the child ends of the pipes
>> + close(ToExecutor[ReadEnd]);
>> + close(FromExecutor[WriteEnd]);
>> +
>> + auto Channel =
>> + std::make_unique<RPCChannel>(FromExecutor[ReadEnd],
ToExecutor[WriteEnd]);
>> + auto Endpoint =
>> +
std::make_unique<RemoteTargetProcessControl::RPCEndpoint>(*Channel, true);
>> +
>> + TPC = std::make_unique<RemoteTargetProcessControl>(ES,
std::move(Channel),
>> +
std::move(Endpoint));
>> +
>> + if (auto Err = TPC->initializeORCRPCTPCBase())
>> + return joinErrors(std::move(Err), TPC->disconnect());
>> +
>> + TPC->initializeMemoryManagement();
>> +
>> + shared::registerStringError<RPCChannel>();
>> + return Error::success();
>> +}
>> +
>> +static Expected<int> connectTCPSocketImpl(std::string Host,
>> + std::string PortStr) {
>> + addrinfo *AI;
>> + addrinfo Hints{};
>> + Hints.ai_family = AF_INET;
>> + Hints.ai_socktype = SOCK_STREAM;
>> + Hints.ai_flags = AI_NUMERICSERV;
>> +
>> + if (int EC = getaddrinfo(Host.c_str(), PortStr.c_str(), &Hints, &AI))
>> + return make_error<StringError>(
>> + formatv("address resolution failed ({0})", gai_strerror(EC)),
>> + inconvertibleErrorCode());
>> +
>> + // Cycle through the returned addrinfo structures and connect to the
first
>> + // reachable endpoint.
>> + int SockFD;
>> + addrinfo *Server;
>> + for (Server = AI; Server != nullptr; Server = Server->ai_next) {
>> + // If socket fails, maybe it's because the address family is not
supported.
>> + // Skip to the next addrinfo structure.
>> + if ((SockFD = socket(AI->ai_family, AI->ai_socktype,
AI->ai_protocol)) < 0)
>> + continue;
>> +
>> + // If connect works, we exit the loop with a working socket.
>> + if (connect(SockFD, Server->ai_addr, Server->ai_addrlen) == 0)
>> + break;
>> +
>> + close(SockFD);
>> + }
>> + freeaddrinfo(AI);
>> +
>> + // Did we reach the end of the loop without connecting to a valid
endpoint?
>> + if (Server == nullptr)
>> + return make_error<StringError>("invalid hostname",
>> + inconvertibleErrorCode());
>> +
>> + return SockFD;
>> +}
>> +
>> +Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
>> +JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
>> + ExecutionSession &ES) {
>> + auto CreateErr = [NetworkAddress](StringRef Details) {
>> + return make_error<StringError>(
>> + formatv("Failed to connect TCP socket '{0}': {1}",
NetworkAddress,
>> + Details),
>> + inconvertibleErrorCode());
>> + };
>> +
>> + StringRef Host, PortStr;
>> + std::tie(Host, PortStr) = NetworkAddress.split(':');
>> + if (Host.empty())
>> + return CreateErr("host name cannot be empty");
>> + if (PortStr.empty())
>> + return CreateErr("port cannot be empty");
>> + int Port = 0;
>> + if (PortStr.getAsInteger(10, Port))
>> + return CreateErr("port number is not a valid integer");
>> +
>> + Expected<int> SockFD = connectTCPSocketImpl(Host.str(),
PortStr.str());
>> + if (!SockFD)
>> + return CreateErr(toString(SockFD.takeError()));
>> +
>> + auto Channel = std::make_unique<RPCChannel>(*SockFD, *SockFD);
>> + auto Endpoint =
>> +
std::make_unique<RemoteTargetProcessControl::RPCEndpoint>(*Channel, true);
>> +
>> + auto TPC = std::make_unique<RemoteTargetProcessControl>(
>> + ES, std::move(Channel), std::move(Endpoint));
>> +
>> + if (auto Err = TPC->initializeORCRPCTPCBase())
>> + return joinErrors(std::move(Err), TPC->disconnect());
>> +
>> + TPC->initializeMemoryManagement();
>> + shared::registerStringError<RPCChannel>();
>> +
>> + return std::unique_ptr<TCPSocketJITLinkExecutor>(
>> + new TCPSocketJITLinkExecutor(std::move(TPC)));
>> +}
>> +
>> +#endif
>>
>> diff --git
a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>> new file mode 100644
>> index 000000000000..e629c0e036f5
>> --- /dev/null
>> +++
b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>> @@ -0,0 +1,111 @@
>> +//===-- RemoteJITUtils.h - Utilities for remote-JITing ----------*- C++
-*-===//
>> +//
>> +// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
>> +// See https://llvm.org/LICENSE.txt for license information.
>> +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
>> +//
>>
+//===----------------------------------------------------------------------===//
>> +//
>> +// Utilities for TargetProcessControl-based remote JITing with Orc and
JITLink.
>> +//
>>
+//===----------------------------------------------------------------------===//
>> +
>> +#ifndef
LLVM_EXAMPLES_ORCV2EXAMPLES_LLJITWITHREMOTEDEBUGGING_REMOTEJITUTILS_H
>> +#define
LLVM_EXAMPLES_ORCV2EXAMPLES_LLJITWITHREMOTEDEBUGGING_REMOTEJITUTILS_H
>> +
>> +#include "llvm/ADT/ArrayRef.h"
>> +#include "llvm/ADT/StringRef.h"
>> +#include "llvm/ADT/Triple.h"
>> +#include "llvm/ExecutionEngine/JITSymbol.h"
>> +#include "llvm/ExecutionEngine/Orc/Core.h"
>> +#include "llvm/ExecutionEngine/Orc/Layer.h"
>> +#include "llvm/ExecutionEngine/Orc/ObjectTransformLayer.h"
>> +#include "llvm/ExecutionEngine/Orc/Shared/FDRawByteChannel.h"
>> +#include "llvm/Support/Error.h"
>> +
>> +#include <memory>
>> +#include <string>
>> +
>> +#if !defined(_MSC_VER) && !defined(__MINGW32__)
>> +#include <unistd.h>
>> +#else
>> +#include <io.h>
>> +#endif
>> +
>> +namespace llvm {
>> +namespace orc {
>> +
>> +class ChildProcessJITLinkExecutor;
>> +class RemoteTargetProcessControl;
>> +class TCPSocketJITLinkExecutor;
>> +
>> +class JITLinkExecutor {
>> +public:
>> + using RPCChannel = shared::FDRawByteChannel;
>> +
>> + /// Create a JITLinkExecutor for the given exectuable on disk.
>> + static Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
>> + CreateLocal(std::string ExecutablePath);
>> +
>> + /// Find the default exectuable on disk and create a JITLinkExecutor
for it.
>> + static Expected<std::unique_ptr<ChildProcessJITLinkExecutor>>
>> + FindLocal(const char *JITArgv0);
>> +
>> + /// Create a JITLinkExecutor that connects to the given network
address
>> + /// through a TCP socket. A valid NetworkAddress provides hostname
and port,
>> + /// e.g. localhost:20000.
>> + static Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
>> + ConnectTCPSocket(StringRef NetworkAddress, ExecutionSession &ES);
>> +
>> + // Implement ObjectLinkingLayerCreator
>> + Expected<std::unique_ptr<ObjectLayer>> operator()(ExecutionSession &,
>> + const Triple &);
>> +
>> + Error addDebugSupport(ObjectLayer &ObjLayer);
>> +
>> + Expected<std::unique_ptr<DefinitionGenerator>>
>> + loadDylib(StringRef RemotePath);
>> +
>> + Expected<int> runAsMain(JITEvaluatedSymbol MainSym,
>> + ArrayRef<std::string> Args);
>> + Error disconnect();
>> +
>> + virtual ~JITLinkExecutor();
>> +
>> +protected:
>> + std::unique_ptr<RemoteTargetProcessControl> TPC;
>> +
>> + JITLinkExecutor();
>> +};
>> +
>> +/// JITLinkExecutor that runs in a child process on the local machine.
>> +class ChildProcessJITLinkExecutor : public JITLinkExecutor {
>> +public:
>> + Error launch(ExecutionSession &ES);
>> +
>> + pid_t getPID() const { return ProcessID; }
>> + StringRef getPath() const { return ExecutablePath; }
>> +
>> +private:
>> + std::string ExecutablePath;
>> + pid_t ProcessID;
>> +
>> + ChildProcessJITLinkExecutor(std::string ExecutablePath)
>> + : ExecutablePath(std::move(ExecutablePath)) {}
>> +
>> + static std::string defaultPath(const char *HostArgv0, StringRef
ExecutorName);
>> + friend class JITLinkExecutor;
>> +};
>> +
>> +/// JITLinkExecutor connected through a TCP socket.
>> +class TCPSocketJITLinkExecutor : public JITLinkExecutor {
>> +private:
>> + TCPSocketJITLinkExecutor(std::unique_ptr<RemoteTargetProcessControl>
TPC);
>> +
>> + friend class JITLinkExecutor;
>> +};
>> +
>> +} // namespace orc
>> +} // namespace llvm
>> +
>> +#endif
>>
>> diff --git a/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
>> new file mode 100644
>> index 000000000000..3a5c2bcefee1
>> --- /dev/null
>> +++ b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.c
>> @@ -0,0 +1,2 @@
>> +int sub1(int x) { return x - 1; }
>> +int main(int argc, char **argv) { return sub1(argc); }
>>
>> diff --git a/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
>> new file mode 100644
>> index 000000000000..659dbe109ec6
>> --- /dev/null
>> +++ b/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1_elf.ll
>> @@ -0,0 +1,52 @@
>> +; ModuleID = 'argc_sub1.c'
>> +target triple = "x86_64-unknown-unknown-elf"
>> +
>> +define i32 @sub1(i32) !dbg !8 {
>> + call void @llvm.dbg.value(metadata i32 %0, metadata !13, metadata
!DIExpression()), !dbg !14
>> + %2 = add nsw i32 %0, -1, !dbg !15
>> + ret i32 %2, !dbg !16
>> +}
>> +
>> +define i32 @main(i32, i8** nocapture readnone) !dbg !17 {
>> + call void @llvm.dbg.value(metadata i32 %0, metadata !24, metadata
!DIExpression()), !dbg !26
>> + call void @llvm.dbg.value(metadata i8** %1, metadata !25, metadata
!DIExpression()), !dbg !27
>> + %3 = tail call i32 @sub1(i32 %0), !dbg !28
>> + ret i32 %3, !dbg !29
>> +}
>> +
>> +declare void @llvm.dbg.value(metadata, metadata, metadata)
>> +
>> +!llvm.dbg.cu = !{!0}
>> +!llvm.module.flags = !{!3, !4, !5, !6}
>> +!llvm.ident = !{!7}
>> +
>> +!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer:
"clang version 7.0.1-8+deb10u2 (tags/RELEASE_701/final)", isOptimized:
true, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
>> +!1 = !DIFile(filename: "argc_sub1.c", directory: "Inputs/")
>> +!2 = !{}
>> +!3 = !{i32 2, !"Dwarf Version", i32 4}
>> +!4 = !{i32 2, !"Debug Info Version", i32 3}
>> +!5 = !{i32 1, !"wchar_size", i32 4}
>> +!6 = !{i32 7, !"PIC Level", i32 2}
>> +!7 = !{!"clang version 7.0.1-8+deb10u2 (tags/RELEASE_701/final)"}
>> +!8 = distinct !DISubprogram(name: "sub1", scope: !1, file: !1, line: 1,
type: !9, isLocal: false, isDefinition: true, scopeLine: 1, flags:
DIFlagPrototyped, isOptimized: true, unit: !0, retainedNodes: !12)
>> +!9 = !DISubroutineType(types: !10)
>> +!10 = !{!11, !11}
>> +!11 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
>> +!12 = !{!13}
>> +!13 = !DILocalVariable(name: "x", arg: 1, scope: !8, file: !1, line: 1,
type: !11)
>> +!14 = !DILocation(line: 1, column: 14, scope: !8)
>> +!15 = !DILocation(line: 1, column: 28, scope: !8)
>> +!16 = !DILocation(line: 1, column: 19, scope: !8)
>> +!17 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line:
2, type: !18, isLocal: false, isDefinition: true, scopeLine: 2, flags:
DIFlagPrototyped, isOptimized: true, unit: !0, retainedNodes: !23)
>> +!18 = !DISubroutineType(types: !19)
>> +!19 = !{!11, !11, !20}
>> +!20 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !21, size: 64)
>> +!21 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !22, size: 64)
>> +!22 = !DIBasicType(name: "char", size: 8, encoding: DW_ATE_signed_char)
>> +!23 = !{!24, !25}
>> +!24 = !DILocalVariable(name: "argc", arg: 1, scope: !17, file: !1,
line: 2, type: !11)
>> +!25 = !DILocalVariable(name: "argv", arg: 2, scope: !17, file: !1,
line: 2, type: !20)
>> +!26 = !DILocation(line: 2, column: 14, scope: !17)
>> +!27 = !DILocation(line: 2, column: 27, scope: !17)
>> +!28 = !DILocation(line: 2, column: 42, scope: !17)
>> +!29 = !DILocation(line: 2, column: 35, scope: !17)
>>
>> diff --git
a/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>> new file mode 100644
>> index 000000000000..a9c3946b55e2
>> --- /dev/null
>> +++ b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>> @@ -0,0 +1,10 @@
>> +# This test makes sure that the example builds and executes as expected.
>> +# Instructions for debugging can be found in
LLJITWithRemoteDebugging.cpp
>> +
>> +# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll
>> +# CHECK: Running: {{.*}}/Inputs/argc_sub1_elf.ll
>> +# CHECK: Exit code: 0
>> +
>> +# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll --args 2nd
3rd 4th
>> +# CHECK: Running: {{.*}}/Inputs/argc_sub1_elf.ll 2nd 3rd 4th
>> +# CHECK: Exit code: 3
>>
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
> --
> https://flowcrypt.com/pub/stefan.graenitz@gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210403/b9411951/attachment.html>
More information about the llvm-commits
mailing list