[llvm] 0589351 - [Orc][examples] Adopt ExecutorProcessControl API and re-enable LLJITWithRemoteDebugging
Stefan Gränitz via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 6 02:34:49 PDT 2021
Ok, I disabled the C API tests with: https://reviews.llvm.org/rG9c63e5b415d1
On 06/08/2021 09:09, Valentin Churavy wrote:
> Yes these were added while the orc tests were temporarily disabled.
>
> I am on vacation this week, so no objection from me to disable them
> again for now.
>
> Best,
> Valentin
>
> On Thu, Aug 5, 2021, 18:48 Azhar Mohammed <azhar at apple.com
> <mailto:azhar at apple.com>> wrote:
>
> Can you please disable those tests again until we have a fix?
>
>> On Aug 5, 2021, at 3:00 AM, Stefan Gränitz
>> <stefan.graenitz at gmail.com <mailto:stefan.graenitz at gmail.com>> wrote:
>>
>> Hi Azhar
>>
>> Thanks for the ping. You are right, my change for re-enabling the
>> OrcV2Examples test did not only affect my own test but also others.
>> @Valentin and Lang (cc) here: Did you add these tests while they
>> were temporarily disabled via the lit.local.cfg?
>>
>> Not sure what exactly is the issue here. It might be due to
>> sanitizers, but it might as well just be the only bot that runs
>> these tests and the actual issue is something else. May I forward
>> this to you?
>>
>> Thanks
>> Stefan
>>
>> On 04/08/2021 18:36, Azhar Mohammed wrote:
>>> Hi Stefan
>>>
>>> I see the OrcV2 examples tests are failing on Darwin, when built
>>> with the sanitizers enabled. Can you please take a look?
>>>
>>> Refer
>>> to https://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/7992/testReport/
>>> <https://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/7992/testReport/>.
>>>
>>> Failed Tests (5):
>>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-add-object-file.test
>>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-basic-usage.test
>>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-lazy.test
>>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-reflect-process-symbols.test
>>> LLVM :: Examples/OrcV2Examples/orcv2-cbindings-removable-code.test
>>>
>>> Thanks
>>> Azhar
>>>
>>>> On Jul 29, 2021, at 5:41 AM, Stefan Gränitz via llvm-commits
>>>> <llvm-commits at lists.llvm.org
>>>> <mailto:llvm-commits at lists.llvm.org>> wrote:
>>>>
>>>>
>>>> Author: Stefan Gränitz
>>>> Date: 2021-07-29T14:40:42+02:00
>>>> New Revision: 058935145d6b0c600c35e8b83fa150896c725f8d
>>>>
>>>> URL:
>>>> https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d
>>>> <https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d>
>>>> DIFF:
>>>> https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d.diff
>>>> <https://github.com/llvm/llvm-project/commit/058935145d6b0c600c35e8b83fa150896c725f8d.diff>
>>>>
>>>> LOG: [Orc][examples] Adopt ExecutorProcessControl API and
>>>> re-enable LLJITWithRemoteDebugging
>>>>
>>>> The API change originated from D104694. The
>>>> LLJITWithRemoteDebugging example and the test for it were
>>>> disabled while it was in the works.
>>>>
>>>> Added:
>>>>
>>>>
>>>> Modified:
>>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>>> llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>>> llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>>> llvm/test/Examples/lit.local.cfg
>>>>
>>>> Removed:
>>>>
>>>>
>>>>
>>>> ################################################################################
>>>> diff --git
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>>> index 06dfe88a898d5..2f8db1976107d 100644
>>>> ---
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>>> +++
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/CMakeLists.txt
>>>> @@ -10,7 +10,7 @@ set(LLVM_LINK_COMPONENTS
>>>> nativecodegen
>>>> )
>>>>
>>>> -if (LLVM_INCLUDE_UTILS AND NOT LLVM_INCLUDE_UTILS)
>>>> +if (LLVM_INCLUDE_UTILS)
>>>> add_llvm_example(LLJITWithRemoteDebugging
>>>> LLJITWithRemoteDebugging.cpp
>>>> RemoteJITUtils.cpp
>>>> @@ -18,7 +18,4 @@ if (LLVM_INCLUDE_UTILS AND NOT
>>>> LLVM_INCLUDE_UTILS)
>>>> DEPENDS
>>>> llvm-jitlink-executor
>>>> )
>>>> -else()
>>>> - # Use a temporary no-op target until D104694 lands.
>>>> - add_custom_target(LLJITWithRemoteDebugging)
>>>> endif()
>>>>
>>>> diff --git
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>>> index ca0953550e090..227ad09d7ae79 100644
>>>> ---
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>>> +++
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/LLJITWithRemoteDebugging.cpp
>>>> @@ -133,12 +133,12 @@ static cl::opt<bool>
>>>>
>>>> ExitOnError ExitOnErr;
>>>>
>>>> -static std::unique_ptr<JITLinkExecutor> connectExecutor(const
>>>> char *Argv0,
>>>> -
>>>> ExecutionSession
>>>> &ES) {
>>>> +static std::unique_ptr<JITLinkExecutor> connectExecutor(const
>>>> char *Argv0) {
>>>> // Connect to a running out-of-process executor through a TCP
>>>> socket.
>>>> if (!OOPExecutorConnect.empty()) {
>>>> std::unique_ptr<TCPSocketJITLinkExecutor> Exec =
>>>> -
>>>> ExitOnErr(JITLinkExecutor::ConnectTCPSocket(OOPExecutorConnect,
>>>> ES));
>>>> +
>>>> ExitOnErr(JITLinkExecutor::ConnectTCPSocket(OOPExecutorConnect,
>>>> +
>>>> std::ref(ExitOnErr)));
>>>>
>>>> outs() << "Connected to executor at " << OOPExecutorConnect
>>>> << "\n";
>>>> if (WaitForDebugger) {
>>>> @@ -157,7 +157,7 @@ static std::unique_ptr<JITLinkExecutor>
>>>> connectExecutor(const char *Argv0,
>>>>
>>>> outs() << "Found out-of-process executor: " <<
>>>> Exec->getPath() << "\n";
>>>>
>>>> - ExitOnErr(Exec->launch(ES));
>>>> + ExitOnErr(Exec->launch(std::ref(ExitOnErr)));
>>>> if (WaitForDebugger) {
>>>> outs() << "Launched executor in subprocess: " <<
>>>> Exec->getPID() << "\n"
>>>> << "Attach a debugger and press any key to continue.\n";
>>>> @@ -177,11 +177,8 @@ int main(int argc, char *argv[]) {
>>>> ExitOnErr.setBanner(std::string(argv[0]) + ": ");
>>>> cl::ParseCommandLineOptions(argc, argv,
>>>> "LLJITWithRemoteDebugging");
>>>>
>>>> - auto ES = std::make_unique<ExecutionSession>();
>>>> - ES->setErrorReporter([&](Error Err) {
>>>> ExitOnErr(std::move(Err)); });
>>>> -
>>>> // Launch/connect the out-of-process executor.
>>>> - std::unique_ptr<JITLinkExecutor> Executor =
>>>> connectExecutor(argv[0], *ES);
>>>> + std::unique_ptr<JITLinkExecutor> Executor =
>>>> connectExecutor(argv[0]);
>>>>
>>>> // Load the given IR files.
>>>> std::vector<ThreadSafeModule> TSMs;
>>>> @@ -215,6 +212,8 @@ int main(int argc, char *argv[]) {
>>>>
>>>> // Create LLJIT and destroy it before disconnecting the
>>>> target process.
>>>> {
>>>> + std::unique_ptr<ExecutionSession> ES =
>>>> Executor->startSession();
>>>> +
>>>> outs() << "Initializing LLJIT for remote executor\n";
>>>> auto J = ExitOnErr(LLJITBuilder()
>>>> .setExecutionSession(std::move(ES))
>>>> @@ -253,6 +252,5 @@ int main(int argc, char *argv[]) {
>>>> outs() << "Exit code: " << Result << "\n";
>>>> }
>>>>
>>>> - ExitOnErr(Executor->disconnect());
>>>> return 0;
>>>> }
>>>>
>>>> diff --git
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>>> index 8e4a283556617..2616dd225d021 100644
>>>> ---
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>>> +++
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.cpp
>>>> @@ -47,9 +47,9 @@ class RemoteExecutorProcessControl
>>>> public:
>>>> using BaseT::initializeORCRPCEPCBase;
>>>>
>>>> - RemoteExecutorProcessControl(ExecutionSession &ES,
>>>> - std::unique_ptr<RPCChannel>
>>>> Channel,
>>>> - std::unique_ptr<RPCEndpoint>
>>>> Endpoint);
>>>> + RemoteExecutorProcessControl(std::unique_ptr<RPCChannel>
>>>> Channel,
>>>> + std::unique_ptr<RPCEndpoint>
>>>> Endpoint,
>>>> + BaseT::ErrorReporter ReportError);
>>>>
>>>> void initializeMemoryManagement();
>>>> Error disconnect() override;
>>>> @@ -64,10 +64,10 @@ class RemoteExecutorProcessControl
>>>> };
>>>>
>>>> RemoteExecutorProcessControl::RemoteExecutorProcessControl(
>>>> - ExecutionSession &ES, std::unique_ptr<RPCChannel> Channel,
>>>> - std::unique_ptr<RPCEndpoint> Endpoint)
>>>> - : BaseT(ES.getSymbolStringPool(), *Endpoint,
>>>> - [&ES](Error Err) { ES.reportError(std::move(Err)); }),
>>>> + std::unique_ptr<RPCChannel> Channel,
>>>> std::unique_ptr<RPCEndpoint> Endpoint,
>>>> + BaseT::ErrorReporter ReportError)
>>>> + : BaseT(std::make_shared<SymbolStringPool>(), *Endpoint,
>>>> + std::move(ReportError)),
>>>> Channel(std::move(Channel)), Endpoint(std::move(Endpoint)) {
>>>>
>>>> ListenerThread = std::thread([&]() {
>>>> @@ -109,11 +109,17 @@ JITLinkExecutor::~JITLinkExecutor() =
>>>> default;
>>>>
>>>> Expected<std::unique_ptr<ObjectLayer>>
>>>> JITLinkExecutor::operator()(ExecutionSession &ES, const Triple
>>>> &TT) {
>>>> + assert(EPC && "RemoteExecutorProcessControl must be
>>>> initialized");
>>>> return std::make_unique<ObjectLinkingLayer>(ES,
>>>> EPC->getMemMgr());
>>>> }
>>>>
>>>> +std::unique_ptr<ExecutionSession>
>>>> JITLinkExecutor::startSession() {
>>>> + assert(OwnedEPC && "RemoteExecutorProcessControl must be
>>>> initialized");
>>>> + return std::make_unique<ExecutionSession>(std::move(OwnedEPC));
>>>> +}
>>>> +
>>>> Error JITLinkExecutor::addDebugSupport(ObjectLayer &ObjLayer) {
>>>> - auto Registrar = createJITLoaderGDBRegistrar(*EPC);
>>>> + auto Registrar =
>>>> createJITLoaderGDBRegistrar(EPC->getExecutionSession());
>>>> if (!Registrar)
>>>> return Registrar.takeError();
>>>>
>>>> @@ -127,7 +133,8 @@ Error
>>>> JITLinkExecutor::addDebugSupport(ObjectLayer &ObjLayer) {
>>>> Expected<std::unique_ptr<DefinitionGenerator>>
>>>> JITLinkExecutor::loadDylib(StringRef RemotePath) {
>>>> if (auto Handle = EPC->loadDylib(RemotePath.data()))
>>>> - return
>>>> std::make_unique<EPCDynamicLibrarySearchGenerator>(*EPC, *Handle);
>>>> + return std::make_unique<EPCDynamicLibrarySearchGenerator>(
>>>> + EPC->getExecutionSession(), *Handle);
>>>> else
>>>> return Handle.takeError();
>>>> }
>>>> @@ -174,7 +181,8 @@ JITLinkExecutor::CreateLocal(std::string
>>>> ExecutablePath) {
>>>>
>>>> TCPSocketJITLinkExecutor::TCPSocketJITLinkExecutor(
>>>> std::unique_ptr<RemoteExecutorProcessControl> EPC) {
>>>> - this->EPC = std::move(EPC);
>>>> + this->OwnedEPC = std::move(EPC);
>>>> + this->EPC = this->OwnedEPC.get();
>>>> }
>>>>
>>>> #ifndef LLVM_ON_UNIX
>>>> @@ -197,7 +205,8 @@ JITLinkExecutor::ConnectTCPSocket(StringRef
>>>> NetworkAddress,
>>>>
>>>> #else
>>>>
>>>> -Error ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
>>>> +Error ChildProcessJITLinkExecutor::launch(
>>>> + unique_function<void(Error)> ErrorReporter) {
>>>> constexpr int ReadEnd = 0;
>>>> constexpr int WriteEnd = 1;
>>>>
>>>> @@ -252,13 +261,14 @@ Error
>>>> ChildProcessJITLinkExecutor::launch(ExecutionSession &ES) {
>>>> auto Endpoint =
>>>> std::make_unique<RemoteExecutorProcessControl::RPCEndpoint>(
>>>> *Channel, true);
>>>>
>>>> - EPC = std::make_unique<RemoteExecutorProcessControl>(ES,
>>>> std::move(Channel),
>>>> -
>>>> std::move(Endpoint));
>>>> + OwnedEPC = std::make_unique<RemoteExecutorProcessControl>(
>>>> + std::move(Channel), std::move(Endpoint),
>>>> std::move(ErrorReporter));
>>>>
>>>> - if (auto Err = EPC->initializeORCRPCEPCBase())
>>>> - return joinErrors(std::move(Err), EPC->disconnect());
>>>> + if (auto Err = OwnedEPC->initializeORCRPCEPCBase())
>>>> + return joinErrors(std::move(Err), OwnedEPC->disconnect());
>>>>
>>>> - EPC->initializeMemoryManagement();
>>>> + OwnedEPC->initializeMemoryManagement();
>>>> + EPC = OwnedEPC.get();
>>>>
>>>> shared::registerStringError<RPCChannel>();
>>>> return Error::success();
>>>> @@ -305,7 +315,7 @@ static Expected<int>
>>>> connectTCPSocketImpl(std::string Host,
>>>>
>>>> Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
>>>> JITLinkExecutor::ConnectTCPSocket(StringRef NetworkAddress,
>>>> - ExecutionSession &ES) {
>>>> + unique_function<void(Error)>
>>>> ErrorReporter) {
>>>> auto CreateErr = [NetworkAddress](StringRef Details) {
>>>> return make_error<StringError>(
>>>> formatv("Failed to connect TCP socket '{0}': {1}",
>>>> NetworkAddress,
>>>> @@ -332,7 +342,7 @@ JITLinkExecutor::ConnectTCPSocket(StringRef
>>>> NetworkAddress,
>>>> *Channel, true);
>>>>
>>>> auto EPC = std::make_unique<RemoteExecutorProcessControl>(
>>>> - ES, std::move(Channel), std::move(Endpoint));
>>>> + std::move(Channel), std::move(Endpoint),
>>>> std::move(ErrorReporter));
>>>>
>>>> if (auto Err = EPC->initializeORCRPCEPCBase())
>>>> return joinErrors(std::move(Err), EPC->disconnect());
>>>>
>>>> diff --git
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>>> index baa376003a67b..5b15b1e9964f7 100644
>>>> ---
>>>> a/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>>> +++
>>>> b/llvm/examples/OrcV2Examples/LLJITWithRemoteDebugging/RemoteJITUtils.h
>>>> @@ -56,12 +56,16 @@ class JITLinkExecutor {
>>>> /// through a TCP socket. A valid NetworkAddress provides
>>>> hostname and port,
>>>> /// e.g. localhost:20000.
>>>> static Expected<std::unique_ptr<TCPSocketJITLinkExecutor>>
>>>> - ConnectTCPSocket(StringRef NetworkAddress, ExecutionSession
>>>> &ES);
>>>> + ConnectTCPSocket(StringRef NetworkAddress,
>>>> + unique_function<void(Error)> ErrorReporter);
>>>>
>>>> // Implement ObjectLinkingLayerCreator
>>>> Expected<std::unique_ptr<ObjectLayer>>
>>>> operator()(ExecutionSession &,
>>>> const
>>>> Triple &);
>>>>
>>>> + std::unique_ptr<ExecutionSession> startSession();
>>>> + Error disconnect();
>>>> +
>>>> Error addDebugSupport(ObjectLayer &ObjLayer);
>>>>
>>>> Expected<std::unique_ptr<DefinitionGenerator>>
>>>> @@ -69,12 +73,12 @@ class JITLinkExecutor {
>>>>
>>>> Expected<int> runAsMain(JITEvaluatedSymbol MainSym,
>>>> ArrayRef<std::string> Args);
>>>> - Error disconnect();
>>>>
>>>> virtual ~JITLinkExecutor();
>>>>
>>>> protected:
>>>> - std::unique_ptr<RemoteExecutorProcessControl> EPC;
>>>> + std::unique_ptr<RemoteExecutorProcessControl> OwnedEPC{nullptr};
>>>> + RemoteExecutorProcessControl *EPC{nullptr};
>>>>
>>>> JITLinkExecutor();
>>>> };
>>>> @@ -82,7 +86,7 @@ class JITLinkExecutor {
>>>> /// JITLinkExecutor that runs in a child process on the local
>>>> machine.
>>>> class ChildProcessJITLinkExecutor : public JITLinkExecutor {
>>>> public:
>>>> - Error launch(ExecutionSession &ES);
>>>> + Error launch(unique_function<void(Error)> ErrorReporter);
>>>>
>>>> pid_t getPID() const { return ProcessID; }
>>>> StringRef getPath() const { return ExecutablePath; }
>>>>
>>>> diff --git
>>>> a/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>>> b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>>> index a1ea7f9db1fd0..a09d72a76bef9 100644
>>>> ---
>>>> a/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>>> +++
>>>> b/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
>>>> @@ -1,10 +1,10 @@
>>>> # This test makes sure that the example builds and executes as
>>>> expected.
>>>> # Instructions for debugging can be found in
>>>> LLJITWithRemoteDebugging.cpp
>>>>
>>>> -# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll |
>>>> FileCheck --check-prefix=CHECK1 %s
>>>> -# CHECK1: Parsing input IR code from:
>>>> {{.*}}/Inputs/argc_sub1_elf.ll
>>>> -# CHECK1: Running: main()
>>>> -# CHECK1: Exit code: 0
>>>> +# RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll |
>>>> FileCheck --check-prefix=CHECK0 %s
>>>> +# CHECK0: Parsing input IR code from:
>>>> {{.*}}/Inputs/argc_sub1_elf.ll
>>>> +# CHECK0: Running: main()
>>>> +# CHECK0: Exit code: 0
>>>>
>>>> # RUN: LLJITWithRemoteDebugging %p/Inputs/argc_sub1_elf.ll
>>>> --args 2nd 3rd 4th | FileCheck --check-prefix=CHECK3 %s
>>>> # CHECK3: Parsing input IR code from:
>>>> {{.*}}/Inputs/argc_sub1_elf.ll
>>>>
>>>> diff --git a/llvm/test/Examples/lit.local.cfg
>>>> b/llvm/test/Examples/lit.local.cfg
>>>> index f23a918956ba7..a9f3860333603 100644
>>>> --- a/llvm/test/Examples/lit.local.cfg
>>>> +++ b/llvm/test/Examples/lit.local.cfg
>>>> @@ -1,7 +1,5 @@
>>>> -#if not config.build_examples or sys.platform in ['win32']:
>>>> -
>>>> -# Mark both lljit-with-* tests unsupported until D104694 lands.
>>>> -config.unsupported = True
>>>> +if not config.build_examples or sys.platform in ['win32']:
>>>> + config.unsupported = True
>>>>
>>>> # Test discovery should ignore subdirectories that contain test
>>>> inputs.
>>>> config.excludes = ['Inputs']
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> llvm-commits mailing list
>>>> llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>> <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits>
>>>
>> --
>> https://weliveindetail.github.io/blog/about/ <https://weliveindetail.github.io/blog/about/>
>
--
https://weliveindetail.github.io/blog/about/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210806/46a61090/attachment.html>
More information about the llvm-commits
mailing list