[llvm] 0589351 - [Orc][examples] Adopt ExecutorProcessControl API and re-enable LLJITWithRemoteDebugging
Azhar Mohammed via llvm-commits
llvm-commits at lists.llvm.org
Thu Aug 5 09:48:14 PDT 2021
Can you please disable those tests again until we have a fix?
> On Aug 5, 2021, at 3:00 AM, Stefan Gränitz <stefan.graenitz at gmail.com> wrote:
>
> Hi Azhar
>
> Thanks for the ping. You are right, my change for re-enabling the OrcV2Examples test did not only affect my own test but also others.
> @Valentin and Lang (cc) here: Did you add these tests while they were temporarily disabled via the lit.local.cfg?
>
> Not sure what exactly is the issue here. It might be due to sanitizers, but it might as well just be the only bot that runs these tests and the actual issue is something else. May I forward this to you?
>
> Thanks
> Stefan
>
> On 04/08/2021 18:36, Azhar Mohammed wrote:
>> Hi Stefan
>>
>> I see the OrcV2 examples tests are failing on Darwin, when built with the sanitizers enabled. Can you please take a look?
>>
>> Refer to https://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/7992/testReport/ <https://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/7992/testReport/>.
>>
>> Failed Tests (5):
>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-add-object-file.test
>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-basic-usage.test
>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-lazy.test
>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-reflect-process-symbols.test
>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-removable-code.test
>>
>> Thanks
>> Azhar
>>
>>> On Jul 29, 2021, at 5:41 AM, Stefan Gränitz via llvm-commits <llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>> wrote:
>>>
>>>
>>> Author: Stefan Gränitz
>>> Date: 2021-07-29T14:40:42+02:00
>>> New Revision: 058935145d6b0c600c35e8b83fa150896c725f8d
>>>
>>> URL: https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d <https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d>
>>> DIFF: https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d.diff <https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d.diff>
>>>
>>> LOG: [Orc][examples] Adopt ExecutorProcessControl API and re-enable LLJITWithRemoteDebugging
>>>
>>> The API change originated from D104694. The LLJITWithRemoteDebugging example and the test for it were disabled while it was in the works.
>>>
>>> Added:
>>>
>>>
>>> Modified:
>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>> llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>> llvm/test/Examples/lit.local.cfg
>>>
>>> Removed:
>>>
>>>
>>>
>>> ################################################################################
>>> diff --git a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>> index 06dfe88a898d5..2f8db1976107d 100644
>>> --- a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>> +++ b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>> @@ -10,7 +10,7 @@ set(LLVM_LINK_COMPONENTS
>>> nativecodegen
>>> )
>>>
>>> -if (LLVM_INCLUDE_UTILS AND NOT LLVM_INCLUDE_UTILS)
>>> +if (LLVM_INCLUDE_UTILS)
>>> add_llvm_example(LLJITWithRemoteDebugging
>>> LLJITWithRemoteDebugging.cpp
>>> RemoteJITUtils.cpp
>>> @@ -18,7 +18,4 @@ if (LLVM_INCLUDE_UTILS AND NOT LLVM_INCLUDE_UTILS)
>>> DEPENDS
>>> llvm-jitlink-executor
>>> )
>>> -else()
>>> - # Use a temporary no-op target until D104694 lands.
>>> - add_custom_target(LLJITWithRemoteDebugging)
>>> endif()
>>>
>>> diff --git a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>> index ca0953550e090..227ad09d7ae79 100644
>>> --- a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>> +++ b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>> @@ -133,12 +133,12 @@ static cl::opt<bool>
>>>
>>> ExitOnError ExitOnErr;
>>>
>>> -static std::unique_ptr<JITLinkExecutor> connectExecutor(const char *Argv0,
>>> - ExecutionSession &ES) {
>>> +static std::unique_ptr<JITLinkExecutor> connectExecutor(const char *Argv0) {
>>> // Connect to a running out-of-process executor through a TCP socket.
>>> if (!OOPExecutorConnect.empty()) {
>>> std::unique_ptr<TCPSocketJITLinkExecutor> Exec =
>>> - ExitOnErr(JITLinkExecutor::ConnectTCPSocket(OOPExecutorConnect, ES));
>>> + ExitOnErr(JITLinkExecutor::ConnectTCPSocket(OOPExecutorConnect,
>>> + std::ref(ExitOnErr)));
>>>
>>> outs() << "Connected to executor at " << OOPExecutorConnect << "\n";
>>> if (WaitForDebugger) {
>>> @@ -157,7 +157,7 @@ static std::unique_ptr<JITLinkExecutor> connectExecutor(const char *Argv0,
>>>
>>> outs() << "Found out-of-process executor: " << Exec->getPath() << "\n";
>>>
>>> - ExitOnErr(Exec->launch(ES));
>>> + ExitOnErr(Exec->launch(std::ref(ExitOnErr)));
>>> if (WaitForDebugger) {
>>> outs() << "Launched executor in subprocess: " << Exec->getPID() << "\n"
>>> << "Attach a debugger and press any key to continue.\n";
>>> @@ -177,11 +177,8 @@ int main(int argc, char *argv[]) {
>>> ExitOnErr.setBanner(std::string(argv[0]) + ": ");
>>> cl::ParseCommandLineOptions(argc, argv, "LLJITWithRemoteDebugging");
>>>
>>> - auto ES = std::make_unique<ExecutionSession>();
>>> - ES->setErrorReporter([&](Error Err) { ExitOnErr(std::move(Err)); });
>>> -
>>> // Launch/connect the out-of-process executor.
>>> - std::unique_ptr<JITLinkExecutor> Executor = connectExecutor(argv[0], *ES);
>>> + std::unique_ptr<JITLinkExecutor> Executor = connectExecutor(argv[0]);
>>>
>>> // Load the given IR files.
>>> std::vector<ThreadSafeModule> TSMs;
>>> @@ -215,6 +212,8 @@ int main(int argc, char *argv[]) {
>>>
>>> // Create LLJIT and destroy it before disconnecting the target process.
>>> {
>>> + std::unique_ptr<ExecutionSession> ES = Executor->startSession();
>>> +
>>> outs() << "Initializing LLJIT for remote executor\n";
>>> auto J = ExitOnErr(LLJITBuilder()
>>> .setExecutionSession(std::move(ES))
>>> @@ -253,6 +252,5 @@ int main(int argc, char *argv[]) {
>>> outs() << "Exit code: " << Result << "\n";
>>> }
>>>
>>> - ExitOnErr(Executor->disconnect());
>>> return 0;
>>> }
>>>
>>> diff --git a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>> index 8e4a283556617..2616dd225d021 100644
>>> --- a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>> +++ b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>> @@ -47,9 +47,9 @@ class RemoteExecutorProcessControl
>>> public:
>>> using BaseT::initializeORCRPCEPCBase;
>>>
>>> - RemoteExecutorProcessControl(ExecutionSession &ES,
>>> - std::unique_ptr<RPCChannel> Channel,
>>> - std::unique_ptr<RPCEndpoint> Endpoint);
>>> + RemoteExecutorProcessControl(std::unique_ptr<RPCChannel> Channel,
>>> + std::unique_ptr<RPCEndpoint> Endpoint,
>>> + BaseT::ErrorReporter ReportError);
>>>
>>> void initializeMemoryManagement();
>>> Error disconnect() override;
>>> @@ -64,10 +64,10 @@ class RemoteExecutorProcessControl
>>> };
>>>
>>> RemoteExecutorProcessControl::RemoteExecutorProcessControl(
>>> - ExecutionSession &ES, std::unique_ptr<RPCChannel> Channel,
>>> - std::unique_ptr<RPCEndpoint> Endpoint)
>>> - : BaseT(ES.getSymbolStringPool(), *Endpoint,
>>> - [&ES](Error Err) { ES.reportError(std::move(Err)); }),
>>> + std::unique_ptr<RPCChannel> Channel, std::unique_ptr<RPCEndpoint> Endpoint,
>>> + BaseT::ErrorReporter ReportError)
>>> + : BaseT(std::make_shared<SymbolStringPool>(), *Endpoint,
>>> + std::move(ReportError)),
>>> Channel(std::move(Channel)), Endpoint(std::move(Endpoint)) {
>>>
>>> ListenerThread = std::thread([&]() {
>>> @@ -109,11 +109,17 @@ JITLinkExecutor::~JITLinkExecutor() = default;
>>>
>>> Expected<std::unique_ptr<ObjectLayer>>
>>> JITLinkExecutor::operator()(ExecutionSession &ES, const Triple &TT) {
>>> + assert(EPC && "RemoteExecutorProcessControl must be initialized");
>>> return std::make_unique<ObjectLinkingLayer>(ES, EPC->getMemMgr());
>>> }
>>>
>>> +std::unique_ptr<ExecutionSession> JITLinkExecutor::startSession() {
>>> + assert(OwnedEPC && "RemoteExecutorProcessControl must be initialized");
>>> + return std::make_unique<ExecutionSession>(std::move(OwnedEPC));
>>> +}
>>> +
>>> Error JITLinkExecutor::addDebugSupport(ObjectLayer &ObjLayer) {
>>> - auto Registrar = createJITLoaderGDBRegistrar(*EPC);
>>> + auto Registrar = createJITLoaderGDBRegistrar(EPC->getExecutionSession());
>>> if (!Registrar)
>>> return Registrar.takeError();
>>>
>>> @@ -127,7 +133,8 @@ Error JITLinkExecutor::addDebugSupport(ObjectLayer &ObjLayer) {
>>> Expected<std::unique_ptr<DefinitionGenerator>>
>>> JITLinkExecutor::loadDylib(StringRef RemotePath) {
>>> if (auto Handle = EPC->loadDylib(RemotePath.data()))
>>> - return std::make_unique<EPCDynamicLibrarySearchGenerator>(*EPC, *Handle);
>>> + return std::make_unique<EPCDynamicLibrarySearchGenerator>(
>>> + EPC->getExecutionSession(), *Handle);
>>> else
>>> return Handle.takeError();
>>> }
>>> @@ -174,7 +181,8 @@ JITLinkExecutor::CreateLocal(std::string ExecutablePath) {
>>>
>>> TCPSocketJITLinkExecutor::TCPSocketJITLinkExecutor(
>>> std::unique_ptr<RemoteExecutorProcessControl> EPC) {
>>> - this->EPC = std::move(EPC);
>>> + this->OwnedEPC = std::move(EPC);
>>> + this->EPC = this->OwnedEPC.get();
>>> }
>>>
>>> #ifndef LLVM_ON_UNIX
>>> @@ -197,7 +205,8 @@ JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
>>>
>>> #else
>>>
>>> -Error ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
>>> +Error ChildProcessJITLinkExecutor::launch(
>>> + unique_function<void(Error)> ErrorReporter) {
>>> constexpr int ReadEnd = 0;
>>> constexpr int WriteEnd = 1;
>>>
>>> @@ -252,13 +261,14 @@ Error ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
>>> auto Endpoint = std::make_unique<RemoteExecutorProcessControl::RPCEndpoint>(
>>> *Channel, true);
>>>
>>> - EPC = std::make_unique<RemoteExecutorProcessControl>(ES, std::move(Channel),
>>> - std::move(Endpoint));
>>> + OwnedEPC = std::make_unique<RemoteExecutorProcessControl>(
>>> + std::move(Channel), std::move(Endpoint), std::move(ErrorReporter));
>>>
>>> - if (auto Err = EPC->initializeORCRPCEPCBase())
>>> - return joinErrors(std::move(Err), EPC->disconnect());
>>> + if (auto Err = OwnedEPC->initializeORCRPCEPCBase())
>>> + return joinErrors(std::move(Err), OwnedEPC->disconnect());
>>>
>>> - EPC->initializeMemoryManagement();
>>> + OwnedEPC->initializeMemoryManagement();
>>> + EPC = OwnedEPC.get();
>>>
>>> shared::registerStringError<RPCChannel>();
>>> return Error::success();
>>> @@ -305,7 +315,7 @@ static Expected<int> connectTCPSocketImpl(std::string Host,
>>>
>>> Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
>>> JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
>>> - ExecutionSession &ES) {
>>> + unique_function<void(Error)> ErrorReporter) {
>>> auto CreateErr = [NetworkAddress](StringRef Details) {
>>> return make_error<StringError>(
>>> formatv("Failed to connect TCP socket '{0}': {1}", NetworkAddress,
>>> @@ -332,7 +342,7 @@ JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
>>> *Channel, true);
>>>
>>> auto EPC = std::make_unique<RemoteExecutorProcessControl>(
>>> - ES, std::move(Channel), std::move(Endpoint));
>>> + std::move(Channel), std::move(Endpoint), std::move(ErrorReporter));
>>>
>>> if (auto Err = EPC->initializeORCRPCEPCBase())
>>> return joinErrors(std::move(Err), EPC->disconnect());
>>>
>>> diff --git a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>> index baa376003a67b..5b15b1e9964f7 100644
>>> --- a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>> +++ b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>> @@ -56,12 +56,16 @@ class JITLinkExecutor {
>>> /// through a TCP socket. A valid NetworkAddress provides hostname and port,
>>> /// e.g. localhost:20000.
>>> static Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
>>> - ConnectTCPSocket(StringRef NetworkAddress, ExecutionSession &ES);
>>> + ConnectTCPSocket(StringRef NetworkAddress,
>>> + unique_function<void(Error)> ErrorReporter);
>>>
>>> // Implement ObjectLinkingLayerCreator
>>> Expected<std::unique_ptr<ObjectLayer>> operator()(ExecutionSession &,
>>> const Triple &);
>>>
>>> + std::unique_ptr<ExecutionSession> startSession();
>>> + Error disconnect();
>>> +
>>> Error addDebugSupport(ObjectLayer &ObjLayer);
>>>
>>> Expected<std::unique_ptr<DefinitionGenerator>>
>>> @@ -69,12 +73,12 @@ class JITLinkExecutor {
>>>
>>> Expected<int> runAsMain(JITEvaluatedSymbol MainSym,
>>> ArrayRef<std::string> Args);
>>> - Error disconnect();
>>>
>>> virtual ~JITLinkExecutor();
>>>
>>> protected:
>>> - std::unique_ptr<RemoteExecutorProcessControl> EPC;
>>> + std::unique_ptr<RemoteExecutorProcessControl> OwnedEPC{nullptr};
>>> + RemoteExecutorProcessControl *EPC{nullptr};
>>>
>>> JITLinkExecutor();
>>> };
>>> @@ -82,7 +86,7 @@ class JITLinkExecutor {
>>> /// JITLinkExecutor that runs in a child process on the local machine.
>>> class ChildProcessJITLinkExecutor : public JITLinkExecutor {
>>> public:
>>> - Error launch(ExecutionSession &ES);
>>> + Error launch(unique_function<void(Error)> ErrorReporter);
>>>
>>> pid_t getPID() const { return ProcessID; }
>>> StringRef getPath() const { return ExecutablePath; }
>>>
>>> diff --git a/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>> index a1ea7f9db1fd0..a09d72a76bef9 100644
>>> --- a/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>> +++ b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>> @@ -1,10 +1,10 @@
>>> # This test makes sure that the example builds and executes as expected.
>>> # Instructions for debugging can be found in LLJITWithRemoteDebugging.cpp
>>>
>>> -# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll | FileCheck --check-prefix=CHECK1 %s
>>> -# CHECK1: Parsing input IR code from: {{.*}}/Inputs/argc_sub1_elf.ll
>>> -# CHECK1: Running: main()
>>> -# CHECK1: Exit code: 0
>>> +# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll | FileCheck --check-prefix=CHECK0 %s
>>> +# CHECK0: Parsing input IR code from: {{.*}}/Inputs/argc_sub1_elf.ll
>>> +# CHECK0: Running: main()
>>> +# CHECK0: Exit code: 0
>>>
>>> # RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll --args 2nd 3rd 4th | FileCheck --check-prefix=CHECK3 %s
>>> # CHECK3: Parsing input IR code from: {{.*}}/Inputs/argc_sub1_elf.ll
>>>
>>> diff --git a/llvm/test/Examples/lit.local.cfg b/llvm/test/Examples/lit.local.cfg
>>> index f23a918956ba7..a9f3860333603 100644
>>> --- a/llvm/test/Examples/lit.local.cfg
>>> +++ b/llvm/test/Examples/lit.local.cfg
>>> @@ -1,7 +1,5 @@
>>> -#if not config.build_examples or sys.platform in ['win32']:
>>> -
>>> -# Mark both lljit-with-* tests unsupported until D104694 lands.
>>> -config.unsupported = True
>>> +if not config.build_examples or sys.platform in ['win32']:
>>> + config.unsupported = True
>>>
>>> # Test discovery should ignore subdirectories that contain test inputs.
>>> config.excludes = ['Inputs']
>>>
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits>
>>
> --
> https://weliveindetail.github.io/blog/about/ <https://weliveindetail.github.io/blog/about/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210805/ff3da0d8/attachment.html>
More information about the llvm-commits
mailing list