[llvm] r257305 - [Orc] Add support for remote JITing to the ORC API.
Aaron Ballman via llvm-commits
llvm-commits at lists.llvm.org
Fri Jan 15 11:26:50 PST 2016
On Fri, Jan 15, 2016 at 2:23 PM, David Blaikie <dblaikie at gmail.com> wrote:
>
>
> On Fri, Jan 15, 2016 at 9:21 AM, Aaron Ballman <aaron at aaronballman.com>
> wrote:
>>
>> On Fri, Jan 15, 2016 at 12:12 PM, David Blaikie <dblaikie at gmail.com>
>> wrote:
>> >
>> >
>> > On Fri, Jan 15, 2016 at 8:33 AM, Aaron Ballman via llvm-commits
>> > <llvm-commits at lists.llvm.org> wrote:
>> >>
>> >> On Sun, Jan 10, 2016 at 8:40 PM, Lang Hames via llvm-commits
>> >> <llvm-commits at lists.llvm.org> wrote:
>> >> > Author: lhames
>> >> > Date: Sun Jan 10 19:40:11 2016
>> >> > New Revision: 257305
>> >> >
>> >> > URL: http://llvm.org/viewvc/llvm-project?rev=257305&view=rev
>> >> > Log:
>> >> > [Orc] Add support for remote JITing to the ORC API.
>> >> >
>> >> > This patch adds utilities to ORC for managing a remote JIT target. It
>> >> > consists
>> >> > of:
>> >> >
>> >> > 1. A very primitive RPC system for making calls over a byte-stream.
>> >> > See
>> >> > RPCChannel.h, RPCUtils.h.
>> >> >
>> >> > 2. An RPC API defined in the above system for managing memory,
>> >> > looking
>> >> > up
>> >> > symbols, creating stubs, etc. on a remote target. See
>> >> > OrcRemoteTargetRPCAPI.h.
>> >> >
>> >> > 3. An interface for creating high-level JIT components (memory
>> >> > managers,
>> >> > callback managers, stub managers, etc.) that operate over the RPC
>> >> > API.
>> >> > See
>> >> > OrcRemoteTargetClient.h.
>> >> >
>> >> > 4. A helper class for building servers that can handle the RPC calls.
>> >> > See
>> >> > OrcRemoteTargetServer.h.
>> >> >
>> >> > The system is designed to work neatly with the existing ORC
>> >> > components
>> >> > and
>> >> > functionality. In particular, the ORC callback API (and consequently
>> >> > the
>> >> > CompileOnDemandLayer) is supported, enabling lazy compilation of
>> >> > remote
>> >> > code.
>> >> >
>> >> > Assuming this doesn't trigger any builder failures, a follow-up patch
>> >> > will be
>> >> > committed which tests these utilities by using them to replace LLI's
>> >> > existing
>> >> > remote-JITing demo code.
>> >> >
>> >> >
>> >> > Added:
>> >> >
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
>> >> >
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
>> >> >
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/RPCChannel.h
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/RPCUtils.h
>> >> > llvm/trunk/lib/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.cpp
>> >> > llvm/trunk/unittests/ExecutionEngine/Orc/RPCUtilsTest.cpp
>> >> > Modified:
>> >> > llvm/trunk/lib/ExecutionEngine/Orc/CMakeLists.txt
>> >> > llvm/trunk/unittests/ExecutionEngine/Orc/CMakeLists.txt
>> >> >
>> >> > Added:
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
>> >> > URL:
>> >> >
>> >> > http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h?rev=257305&view=auto
>> >> >
>> >> >
>> >> > ==============================================================================
>> >> > ---
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
>> >> > (added)
>> >> > +++
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
>> >> > Sun Jan 10 19:40:11 2016
>> >> > @@ -0,0 +1,743 @@
>> >> > +//===---- OrcRemoteTargetClient.h - Orc Remote-target Client ----*-
>> >> > C++
>> >> > -*-===//
>> >> > +//
>> >> > +// The LLVM Compiler Infrastructure
>> >> > +//
>> >> > +// This file is distributed under the University of Illinois Open
>> >> > Source
>> >> > +// License. See LICENSE.TXT for details.
>> >> > +//
>> >> >
>> >> >
>> >> > +//===----------------------------------------------------------------------===//
>> >> > +//
>> >> > +// This file defines the OrcRemoteTargetClient class and helpers.
>> >> > This
>> >> > class
>> >> > +// can be used to communicate over an RPCChannel with an
>> >> > OrcRemoteTargetServer
>> >> > +// instance to support remote-JITing.
>> >> > +//
>> >> >
>> >> >
>> >> > +//===----------------------------------------------------------------------===//
>> >> > +
>> >> > +#ifndef LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETCLIENT_H
>> >> > +#define LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETCLIENT_H
>> >> > +
>> >> > +#include "OrcRemoteTargetRPCAPI.h"
>> >> > +
>> >> > +#define DEBUG_TYPE "orc-remote"
>> >> > +
>> >> > +namespace llvm {
>> >> > +namespace orc {
>> >> > +namespace remote {
>> >> > +
>> >> > +/// This class provides utilities (including memory manager,
>> >> > indirect
>> >> > stubs
>> >> > +/// manager, and compile callback manager types) that support remote
>> >> > JITing
>> >> > +/// in ORC.
>> >> > +///
>> >> > +/// Each of the utility classes talks to a JIT server (an instance
>> >> > of
>> >> > the
>> >> > +/// OrcRemoteTargetServer class) via an RPC system (see RPCUtils.h)
>> >> > to
>> >> > carry out
>> >> > +/// its actions.
>> >> > +template <typename ChannelT>
>> >> > +class OrcRemoteTargetClient : public OrcRemoteTargetRPCAPI {
>> >> > +public:
>> >> > + /// Remote memory manager.
>> >> > + class RCMemoryManager : public RuntimeDyld::MemoryManager {
>> >> > + public:
>> >> > + RCMemoryManager(OrcRemoteTargetClient &Client,
>> >> > ResourceIdMgr::ResourceId Id)
>> >> > + : Client(Client), Id(Id) {
>> >> > + DEBUG(dbgs() << "Created remote allocator " << Id << "\n");
>> >> > + }
>> >> > +
>> >> > + ~RCMemoryManager() {
>> >> > + Client.destroyRemoteAllocator(Id);
>> >> > + DEBUG(dbgs() << "Destroyed remote allocator " << Id << "\n");
>> >> > + }
>> >> > +
>> >> > + uint8_t *allocateCodeSection(uintptr_t Size, unsigned Alignment,
>> >> > + unsigned SectionID,
>> >> > + StringRef SectionName) override {
>> >> > + Unmapped.back().CodeAllocs.emplace_back(Size, Alignment);
>> >> > + uint8_t *Alloc = reinterpret_cast<uint8_t *>(
>> >> > + Unmapped.back().CodeAllocs.back().getLocalAddress());
>> >> > + DEBUG(dbgs() << "Allocator " << Id << " allocated code for "
>> >> > + << SectionName << ": " << Alloc << " (" << Size
>> >> > + << " bytes, alignment " << Alignment << ")\n");
>> >> > + return Alloc;
>> >> > + }
>> >> > +
>> >> > + uint8_t *allocateDataSection(uintptr_t Size, unsigned Alignment,
>> >> > + unsigned SectionID, StringRef
>> >> > SectionName,
>> >> > + bool IsReadOnly) override {
>> >> > + if (IsReadOnly) {
>> >> > + Unmapped.back().RODataAllocs.emplace_back(Size, Alignment);
>> >> > + uint8_t *Alloc = reinterpret_cast<uint8_t *>(
>> >> > + Unmapped.back().RODataAllocs.back().getLocalAddress());
>> >> > + DEBUG(dbgs() << "Allocator " << Id << " allocated ro-data
>> >> > for "
>> >> > + << SectionName << ": " << Alloc << " (" << Size
>> >> > + << " bytes, alignment " << Alignment << ")\n");
>> >> > + return Alloc;
>> >> > + } // else...
>> >> > +
>> >> > + Unmapped.back().RWDataAllocs.emplace_back(Size, Alignment);
>> >> > + uint8_t *Alloc = reinterpret_cast<uint8_t *>(
>> >> > + Unmapped.back().RWDataAllocs.back().getLocalAddress());
>> >> > + DEBUG(dbgs() << "Allocator " << Id << " allocated rw-data for
>> >> > "
>> >> > + << SectionName << ": " << Alloc << " (" << Size
>> >> > + << " bytes, alignment " << Alignment << "\n");
>> >> > + return Alloc;
>> >> > + }
>> >> > +
>> >> > + void reserveAllocationSpace(uintptr_t CodeSize, uint32_t
>> >> > CodeAlign,
>> >> > + uintptr_t RODataSize, uint32_t
>> >> > RODataAlign,
>> >> > + uintptr_t RWDataSize,
>> >> > + uint32_t RWDataAlign) override {
>> >> > + Unmapped.push_back(ObjectAllocs());
>> >> > +
>> >> > + DEBUG(dbgs() << "Allocator " << Id << " reserved:\n");
>> >> > +
>> >> > + if (CodeSize != 0) {
>> >> > + if (auto EC =
>> >> > Client.reserveMem(Unmapped.back().RemoteCodeAddr,
>> >> > Id,
>> >> > + CodeSize, CodeAlign)) {
>> >> > + // FIXME; Add error to poll.
>> >> > + llvm_unreachable("Failed reserving remote memory.");
>> >> > + }
>> >> > + DEBUG(dbgs() << " code: "
>> >> > + << format("0x%016x",
>> >> > Unmapped.back().RemoteCodeAddr)
>> >> > + << " (" << CodeSize << " bytes, alignment " <<
>> >> > CodeAlign
>> >> > + << ")\n");
>> >> > + }
>> >> > +
>> >> > + if (RODataSize != 0) {
>> >> > + if (auto EC =
>> >> > Client.reserveMem(Unmapped.back().RemoteRODataAddr, Id,
>> >> > + RODataSize, RODataAlign)) {
>> >> > + // FIXME; Add error to poll.
>> >> > + llvm_unreachable("Failed reserving remote memory.");
>> >> > + }
>> >> > + DEBUG(dbgs() << " ro-data: "
>> >> > + << format("0x%016x",
>> >> > Unmapped.back().RemoteRODataAddr)
>> >> > + << " (" << RODataSize << " bytes, alignment "
>> >> > + << RODataAlign << ")\n");
>> >> > + }
>> >> > +
>> >> > + if (RWDataSize != 0) {
>> >> > + if (auto EC =
>> >> > Client.reserveMem(Unmapped.back().RemoteRWDataAddr, Id,
>> >> > + RWDataSize, RWDataAlign)) {
>> >> > + // FIXME; Add error to poll.
>> >> > + llvm_unreachable("Failed reserving remote memory.");
>> >> > + }
>> >> > + DEBUG(dbgs() << " rw-data: "
>> >> > + << format("0x%016x",
>> >> > Unmapped.back().RemoteRWDataAddr)
>> >> > + << " (" << RWDataSize << " bytes, alignment "
>> >> > + << RWDataAlign << ")\n");
>> >> > + }
>> >> > + }
>> >> > +
>> >> > + bool needsToReserveAllocationSpace() override { return true; }
>> >> > +
>> >> > + void registerEHFrames(uint8_t *Addr, uint64_t LoadAddr,
>> >> > + size_t Size) override {}
>> >> > +
>> >> > + void deregisterEHFrames(uint8_t *addr, uint64_t LoadAddr,
>> >> > + size_t Size) override {}
>> >> > +
>> >> > + void notifyObjectLoaded(RuntimeDyld &Dyld,
>> >> > + const object::ObjectFile &Obj) override
>> >> > {
>> >> > + DEBUG(dbgs() << "Allocator " << Id << " applied mappings:\n");
>> >> > + for (auto &ObjAllocs : Unmapped) {
>> >> > + {
>> >> > + TargetAddress NextCodeAddr = ObjAllocs.RemoteCodeAddr;
>> >> > + for (auto &Alloc : ObjAllocs.CodeAllocs) {
>> >> > + NextCodeAddr = RoundUpToAlignment(NextCodeAddr,
>> >> > Alloc.getAlign());
>> >> > + Dyld.mapSectionAddress(Alloc.getLocalAddress(),
>> >> > NextCodeAddr);
>> >> > + DEBUG(dbgs() << " code: "
>> >> > + << static_cast<void
>> >> > *>(Alloc.getLocalAddress())
>> >> > + << " -> " << format("0x%016x",
>> >> > NextCodeAddr)
>> >> > << "\n");
>> >> > + Alloc.setRemoteAddress(NextCodeAddr);
>> >> > + NextCodeAddr += Alloc.getSize();
>> >> > + }
>> >> > + }
>> >> > + {
>> >> > + TargetAddress NextRODataAddr = ObjAllocs.RemoteRODataAddr;
>> >> > + for (auto &Alloc : ObjAllocs.RODataAllocs) {
>> >> > + NextRODataAddr =
>> >> > + RoundUpToAlignment(NextRODataAddr,
>> >> > Alloc.getAlign());
>> >> > + Dyld.mapSectionAddress(Alloc.getLocalAddress(),
>> >> > NextRODataAddr);
>> >> > + DEBUG(dbgs() << " ro-data: "
>> >> > + << static_cast<void
>> >> > *>(Alloc.getLocalAddress())
>> >> > + << " -> " << format("0x%016x",
>> >> > NextRODataAddr)
>> >> > + << "\n");
>> >> > + Alloc.setRemoteAddress(NextRODataAddr);
>> >> > + NextRODataAddr += Alloc.getSize();
>> >> > + }
>> >> > + }
>> >> > + {
>> >> > + TargetAddress NextRWDataAddr = ObjAllocs.RemoteRWDataAddr;
>> >> > + for (auto &Alloc : ObjAllocs.RWDataAllocs) {
>> >> > + NextRWDataAddr =
>> >> > + RoundUpToAlignment(NextRWDataAddr,
>> >> > Alloc.getAlign());
>> >> > + Dyld.mapSectionAddress(Alloc.getLocalAddress(),
>> >> > NextRWDataAddr);
>> >> > + DEBUG(dbgs() << " rw-data: "
>> >> > + << static_cast<void
>> >> > *>(Alloc.getLocalAddress())
>> >> > + << " -> " << format("0x%016x",
>> >> > NextRWDataAddr)
>> >> > + << "\n");
>> >> > + Alloc.setRemoteAddress(NextRWDataAddr);
>> >> > + NextRWDataAddr += Alloc.getSize();
>> >> > + }
>> >> > + }
>> >> > + Unfinalized.push_back(std::move(ObjAllocs));
>> >> > + }
>> >> > + Unmapped.clear();
>> >> > + }
>> >> > +
>> >> > + bool finalizeMemory(std::string *ErrMsg = nullptr) override {
>> >> > + DEBUG(dbgs() << "Allocator " << Id << " finalizing:\n");
>> >> > +
>> >> > + for (auto &ObjAllocs : Unfinalized) {
>> >> > +
>> >> > + for (auto &Alloc : ObjAllocs.CodeAllocs) {
>> >> > + DEBUG(dbgs() << " copying code: "
>> >> > + << static_cast<void
>> >> > *>(Alloc.getLocalAddress())
>> >> > << " -> "
>> >> > + << format("0x%016x",
>> >> > Alloc.getRemoteAddress())
>> >> > << " ("
>> >> > + << Alloc.getSize() << " bytes)\n");
>> >> > + Client.writeMem(Alloc.getRemoteAddress(),
>> >> > Alloc.getLocalAddress(),
>> >> > + Alloc.getSize());
>> >> > + }
>> >> > +
>> >> > + if (ObjAllocs.RemoteCodeAddr) {
>> >> > + DEBUG(dbgs() << " setting R-X permissions on code block:
>> >> > "
>> >> > + << format("0x%016x",
>> >> > ObjAllocs.RemoteCodeAddr)
>> >> > << "\n");
>> >> > + Client.setProtections(Id, ObjAllocs.RemoteCodeAddr,
>> >> > + sys::Memory::MF_READ |
>> >> > sys::Memory::MF_EXEC);
>> >> > + }
>> >> > +
>> >> > + for (auto &Alloc : ObjAllocs.RODataAllocs) {
>> >> > + DEBUG(dbgs() << " copying ro-data: "
>> >> > + << static_cast<void
>> >> > *>(Alloc.getLocalAddress())
>> >> > << " -> "
>> >> > + << format("0x%016x",
>> >> > Alloc.getRemoteAddress())
>> >> > << " ("
>> >> > + << Alloc.getSize() << " bytes)\n");
>> >> > + Client.writeMem(Alloc.getRemoteAddress(),
>> >> > Alloc.getLocalAddress(),
>> >> > + Alloc.getSize());
>> >> > + }
>> >> > +
>> >> > + if (ObjAllocs.RemoteRODataAddr) {
>> >> > + DEBUG(dbgs() << " setting R-- permissions on ro-data
>> >> > block:
>> >> > "
>> >> > + << format("0x%016x",
>> >> > ObjAllocs.RemoteRODataAddr)
>> >> > + << "\n");
>> >> > + Client.setProtections(Id, ObjAllocs.RemoteRODataAddr,
>> >> > + sys::Memory::MF_READ);
>> >> > + }
>> >> > +
>> >> > + for (auto &Alloc : ObjAllocs.RWDataAllocs) {
>> >> > + DEBUG(dbgs() << " copying rw-data: "
>> >> > + << static_cast<void
>> >> > *>(Alloc.getLocalAddress())
>> >> > << " -> "
>> >> > + << format("0x%016x",
>> >> > Alloc.getRemoteAddress())
>> >> > << " ("
>> >> > + << Alloc.getSize() << " bytes)\n");
>> >> > + Client.writeMem(Alloc.getRemoteAddress(),
>> >> > Alloc.getLocalAddress(),
>> >> > + Alloc.getSize());
>> >> > + }
>> >> > +
>> >> > + if (ObjAllocs.RemoteRWDataAddr) {
>> >> > + DEBUG(dbgs() << " setting RW- permissions on rw-data
>> >> > block:
>> >> > "
>> >> > + << format("0x%016x",
>> >> > ObjAllocs.RemoteRWDataAddr)
>> >> > + << "\n");
>> >> > + Client.setProtections(Id, ObjAllocs.RemoteRWDataAddr,
>> >> > + sys::Memory::MF_READ |
>> >> > sys::Memory::MF_WRITE);
>> >> > + }
>> >> > + }
>> >> > + Unfinalized.clear();
>> >> > +
>> >> > + return false;
>> >> > + }
>> >> > +
>> >> > + private:
>> >> > + class Alloc {
>> >> > + public:
>> >> > + Alloc(uint64_t Size, unsigned Align)
>> >> > + : Size(Size), Align(Align), Contents(new char[Size + Align
>> >> > -
>> >> > 1]),
>> >> > + RemoteAddr(0) {}
>> >> > +
>> >> > + uint64_t getSize() const { return Size; }
>> >> > +
>> >> > + unsigned getAlign() const { return Align; }
>> >> > +
>> >> > + char *getLocalAddress() const {
>> >> > + uintptr_t LocalAddr =
>> >> > reinterpret_cast<uintptr_t>(Contents.get());
>> >> > + LocalAddr = RoundUpToAlignment(LocalAddr, Align);
>> >> > + return reinterpret_cast<char *>(LocalAddr);
>> >> > + }
>> >> > +
>> >> > + void setRemoteAddress(TargetAddress RemoteAddr) {
>> >> > + this->RemoteAddr = RemoteAddr;
>> >> > + }
>> >> > +
>> >> > + TargetAddress getRemoteAddress() const { return RemoteAddr; }
>> >> > +
>> >> > + private:
>> >> > + uint64_t Size;
>> >> > + unsigned Align;
>> >> > + std::unique_ptr<char[]> Contents;
>> >> > + TargetAddress RemoteAddr;
>> >> > + };
>> >> > +
>> >> > + struct ObjectAllocs {
>> >> > + ObjectAllocs()
>> >> > + : RemoteCodeAddr(0), RemoteRODataAddr(0),
>> >> > RemoteRWDataAddr(0)
>> >> > {}
>> >> > + TargetAddress RemoteCodeAddr;
>> >> > + TargetAddress RemoteRODataAddr;
>> >> > + TargetAddress RemoteRWDataAddr;
>> >> > + std::vector<Alloc> CodeAllocs, RODataAllocs, RWDataAllocs;
>> >> > + };
>> >> > +
>> >> > + OrcRemoteTargetClient &Client;
>> >> > + ResourceIdMgr::ResourceId Id;
>> >> > + std::vector<ObjectAllocs> Unmapped;
>> >> > + std::vector<ObjectAllocs> Unfinalized;
>> >> > + };
>> >> > +
>> >> > + /// Remote indirect stubs manager.
>> >> > + class RCIndirectStubsManager : public IndirectStubsManager {
>> >> > + public:
>> >> > + RCIndirectStubsManager(OrcRemoteTargetClient &Remote,
>> >> > + ResourceIdMgr::ResourceId Id)
>> >> > + : Remote(Remote), Id(Id) {}
>> >> > +
>> >> > + ~RCIndirectStubsManager() {
>> >> > Remote.destroyIndirectStubsManager(Id);
>> >> > }
>> >> > +
>> >> > + std::error_code createStub(StringRef StubName, TargetAddress
>> >> > StubAddr,
>> >> > + JITSymbolFlags StubFlags) override {
>> >> > + if (auto EC = reserveStubs(1))
>> >> > + return EC;
>> >> > +
>> >> > + return createStubInternal(StubName, StubAddr, StubFlags);
>> >> > + }
>> >> > +
>> >> > + std::error_code createStubs(const StubInitsMap &StubInits)
>> >> > override
>> >> > {
>> >> > + if (auto EC = reserveStubs(StubInits.size()))
>> >> > + return EC;
>> >> > +
>> >> > + for (auto &Entry : StubInits)
>> >> > + if (auto EC = createStubInternal(Entry.first(),
>> >> > Entry.second.first,
>> >> > + Entry.second.second))
>> >> > + return EC;
>> >> > +
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + JITSymbol findStub(StringRef Name, bool ExportedStubsOnly)
>> >> > override
>> >> > {
>> >> > + auto I = StubIndexes.find(Name);
>> >> > + if (I == StubIndexes.end())
>> >> > + return nullptr;
>> >> > + auto Key = I->second.first;
>> >> > + auto Flags = I->second.second;
>> >> > + auto StubSymbol = JITSymbol(getStubAddr(Key), Flags);
>> >> > + if (ExportedStubsOnly && !StubSymbol.isExported())
>> >> > + return nullptr;
>> >> > + return StubSymbol;
>> >> > + }
>> >> > +
>> >> > + JITSymbol findPointer(StringRef Name) override {
>> >> > + auto I = StubIndexes.find(Name);
>> >> > + if (I == StubIndexes.end())
>> >> > + return nullptr;
>> >> > + auto Key = I->second.first;
>> >> > + auto Flags = I->second.second;
>> >> > + return JITSymbol(getPtrAddr(Key), Flags);
>> >> > + }
>> >> > +
>> >> > + std::error_code updatePointer(StringRef Name,
>> >> > + TargetAddress NewAddr) override {
>> >> > + auto I = StubIndexes.find(Name);
>> >> > + assert(I != StubIndexes.end() && "No stub pointer for
>> >> > symbol");
>> >> > + auto Key = I->second.first;
>> >> > + return Remote.writePointer(getPtrAddr(Key), NewAddr);
>> >> > + }
>> >> > +
>> >> > + private:
>> >> > + struct RemoteIndirectStubsInfo {
>> >> > + RemoteIndirectStubsInfo(TargetAddress StubBase, TargetAddress
>> >> > PtrBase,
>> >> > + unsigned NumStubs)
>> >> > + : StubBase(StubBase), PtrBase(PtrBase), NumStubs(NumStubs)
>> >> > {}
>> >> > + TargetAddress StubBase;
>> >> > + TargetAddress PtrBase;
>> >> > + unsigned NumStubs;
>> >> > + };
>> >> > +
>> >> > + OrcRemoteTargetClient &Remote;
>> >> > + ResourceIdMgr::ResourceId Id;
>> >> > + std::vector<RemoteIndirectStubsInfo> RemoteIndirectStubsInfos;
>> >> > + typedef std::pair<uint16_t, uint16_t> StubKey;
>> >> > + std::vector<StubKey> FreeStubs;
>> >> > + StringMap<std::pair<StubKey, JITSymbolFlags>> StubIndexes;
>> >> > +
>> >> > + std::error_code reserveStubs(unsigned NumStubs) {
>> >> > + if (NumStubs <= FreeStubs.size())
>> >> > + return std::error_code();
>> >> > +
>> >> > + unsigned NewStubsRequired = NumStubs - FreeStubs.size();
>> >> > + TargetAddress StubBase;
>> >> > + TargetAddress PtrBase;
>> >> > + unsigned NumStubsEmitted;
>> >> > +
>> >> > + Remote.emitIndirectStubs(StubBase, PtrBase, NumStubsEmitted,
>> >> > Id,
>> >> > + NewStubsRequired);
>> >> > +
>> >> > + unsigned NewBlockId = RemoteIndirectStubsInfos.size();
>> >> > + RemoteIndirectStubsInfos.push_back(
>> >> > + RemoteIndirectStubsInfo(StubBase, PtrBase,
>> >> > NumStubsEmitted));
>> >> > +
>> >> > + for (unsigned I = 0; I < NumStubsEmitted; ++I)
>> >> > + FreeStubs.push_back(std::make_pair(NewBlockId, I));
>> >> > +
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + std::error_code createStubInternal(StringRef StubName,
>> >> > + TargetAddress InitAddr,
>> >> > + JITSymbolFlags StubFlags) {
>> >> > + auto Key = FreeStubs.back();
>> >> > + FreeStubs.pop_back();
>> >> > + StubIndexes[StubName] = std::make_pair(Key, StubFlags);
>> >> > + return Remote.writePointer(getPtrAddr(Key), InitAddr);
>> >> > + }
>> >> > +
>> >> > + TargetAddress getStubAddr(StubKey K) {
>> >> > + assert(RemoteIndirectStubsInfos[K.first].StubBase != 0 &&
>> >> > + "Missing stub address");
>> >> > + return RemoteIndirectStubsInfos[K.first].StubBase +
>> >> > + K.second * Remote.getIndirectStubSize();
>> >> > + }
>> >> > +
>> >> > + TargetAddress getPtrAddr(StubKey K) {
>> >> > + assert(RemoteIndirectStubsInfos[K.first].PtrBase != 0 &&
>> >> > + "Missing pointer address");
>> >> > + return RemoteIndirectStubsInfos[K.first].PtrBase +
>> >> > + K.second * Remote.getPointerSize();
>> >> > + }
>> >> > + };
>> >> > +
>> >> > + /// Remote compile callback manager.
>> >> > + class RCCompileCallbackManager : public JITCompileCallbackManager
>> >> > {
>> >> > + public:
>> >> > + RCCompileCallbackManager(TargetAddress ErrorHandlerAddress,
>> >> > + OrcRemoteTargetClient &Remote)
>> >> > + : JITCompileCallbackManager(ErrorHandlerAddress),
>> >> > Remote(Remote) {
>> >> > + assert(!Remote.CompileCallback && "Compile callback already
>> >> > set");
>> >> > + Remote.CompileCallback = [this](TargetAddress TrampolineAddr)
>> >> > {
>> >> > + return executeCompileCallback(TrampolineAddr);
>> >> > + };
>> >> > + Remote.emitResolverBlock();
>> >> > + }
>> >> > +
>> >> > + private:
>> >> > + void grow() {
>> >> > + TargetAddress BlockAddr = 0;
>> >> > + uint32_t NumTrampolines = 0;
>> >> > + auto EC = Remote.emitTrampolineBlock(BlockAddr,
>> >> > NumTrampolines);
>> >> > + assert(!EC && "Failed to create trampolines");
>> >> > +
>> >> > + uint32_t TrampolineSize = Remote.getTrampolineSize();
>> >> > + for (unsigned I = 0; I < NumTrampolines; ++I)
>> >> > + this->AvailableTrampolines.push_back(BlockAddr + (I *
>> >> > TrampolineSize));
>> >> > + }
>> >> > +
>> >> > + OrcRemoteTargetClient &Remote;
>> >> > + };
>> >> > +
>> >> > + /// Create an OrcRemoteTargetClient.
>> >> > + /// Channel is the ChannelT instance to communicate on. It is
>> >> > assumed
>> >> > that
>> >> > + /// the channel is ready to be read from and written to.
>> >> > + static ErrorOr<OrcRemoteTargetClient> Create(ChannelT &Channel) {
>> >> > + std::error_code EC;
>> >> > + OrcRemoteTargetClient H(Channel, EC);
>> >> > + if (EC)
>> >> > + return EC;
>> >> > + return H;
>> >> > + }
>> >> > +
>> >> > + /// Call the int(void) function at the given address in the target
>> >> > and return
>> >> > + /// its result.
>> >> > + std::error_code callIntVoid(int &Result, TargetAddress Addr) {
>> >> > + DEBUG(dbgs() << "Calling int(*)(void) " << format("0x%016x",
>> >> > Addr)
>> >> > << "\n");
>> >> > +
>> >> > + if (auto EC = call<CallIntVoid>(Channel, Addr))
>> >> > + return EC;
>> >> > +
>> >> > + unsigned NextProcId;
>> >> > + if (auto EC = listenForCompileRequests(NextProcId))
>> >> > + return EC;
>> >> > +
>> >> > + if (NextProcId != CallIntVoidResponseId)
>> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
>> >> > +
>> >> > + return handle<CallIntVoidResponse>(Channel, [&](int R) {
>> >> > + Result = R;
>> >> > + DEBUG(dbgs() << "Result: " << R << "\n");
>> >> > + return std::error_code();
>> >> > + });
>> >> > + }
>> >> > +
>> >> > + /// Call the int(int, char*[]) function at the given address in
>> >> > the
>> >> > target and
>> >> > + /// return its result.
>> >> > + std::error_code callMain(int &Result, TargetAddress Addr,
>> >> > + const std::vector<std::string> &Args) {
>> >> > + DEBUG(dbgs() << "Calling int(*)(int, char*[]) " <<
>> >> > format("0x%016x", Addr)
>> >> > + << "\n");
>> >> > +
>> >> > + if (auto EC = call<CallMain>(Channel, Addr, Args))
>> >> > + return EC;
>> >> > +
>> >> > + unsigned NextProcId;
>> >> > + if (auto EC = listenForCompileRequests(NextProcId))
>> >> > + return EC;
>> >> > +
>> >> > + if (NextProcId != CallMainResponseId)
>> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
>> >> > +
>> >> > + return handle<CallMainResponse>(Channel, [&](int R) {
>> >> > + Result = R;
>> >> > + DEBUG(dbgs() << "Result: " << R << "\n");
>> >> > + return std::error_code();
>> >> > + });
>> >> > + }
>> >> > +
>> >> > + /// Call the void() function at the given address in the target
>> >> > and
>> >> > wait for
>> >> > + /// it to finish.
>> >> > + std::error_code callVoidVoid(TargetAddress Addr) {
>> >> > + DEBUG(dbgs() << "Calling void(*)(void) " << format("0x%016x",
>> >> > Addr)
>> >> > + << "\n");
>> >> > +
>> >> > + if (auto EC = call<CallVoidVoid>(Channel, Addr))
>> >> > + return EC;
>> >> > +
>> >> > + unsigned NextProcId;
>> >> > + if (auto EC = listenForCompileRequests(NextProcId))
>> >> > + return EC;
>> >> > +
>> >> > + if (NextProcId != CallVoidVoidResponseId)
>> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
>> >> > +
>> >> > + return handle<CallVoidVoidResponse>(Channel, doNothing);
>> >> > + }
>> >> > +
>> >> > + /// Create an RCMemoryManager which will allocate its memory on
>> >> > the
>> >> > remote
>> >> > + /// target.
>> >> > + std::error_code
>> >> > + createRemoteMemoryManager(std::unique_ptr<RCMemoryManager> &MM) {
>> >> > + assert(!MM && "MemoryManager should be null before creation.");
>> >> > +
>> >> > + auto Id = AllocatorIds.getNext();
>> >> > + if (auto EC = call<CreateRemoteAllocator>(Channel, Id))
>> >> > + return EC;
>> >> > + MM = llvm::make_unique<RCMemoryManager>(*this, Id);
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + /// Create an RCIndirectStubsManager that will allocate stubs on
>> >> > the
>> >> > remote
>> >> > + /// target.
>> >> > + std::error_code
>> >> > + createIndirectStubsManager(std::unique_ptr<RCIndirectStubsManager>
>> >> > &I) {
>> >> > + assert(!I && "Indirect stubs manager should be null before
>> >> > creation.");
>> >> > + auto Id = IndirectStubOwnerIds.getNext();
>> >> > + if (auto EC = call<CreateIndirectStubsOwner>(Channel, Id))
>> >> > + return EC;
>> >> > + I = llvm::make_unique<RCIndirectStubsManager>(*this, Id);
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + /// Search for symbols in the remote process. Note: This should be
>> >> > used by
>> >> > + /// symbol resolvers *after* they've searched the local symbol
>> >> > table
>> >> > in the
>> >> > + /// JIT stack.
>> >> > + std::error_code getSymbolAddress(TargetAddress &Addr, StringRef
>> >> > Name)
>> >> > {
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + // Request remote symbol address.
>> >> > + if (auto EC = call<GetSymbolAddress>(Channel, Name))
>> >> > + return EC;
>> >> > +
>> >> > + return expect<GetSymbolAddressResponse>(Channel,
>> >> > [&](TargetAddress
>> >> > &A) {
>> >> > + Addr = A;
>> >> > + DEBUG(dbgs() << "Remote address lookup " << Name << " = "
>> >> > + << format("0x%016x", Addr) << "\n");
>> >> > + return std::error_code();
>> >> > + });
>> >> > + }
>> >> > +
>> >> > + /// Get the triple for the remote target.
>> >> > + const std::string &getTargetTriple() const { return
>> >> > RemoteTargetTriple; }
>> >> > +
>> >> > + std::error_code terminateSession() { return
>> >> > call<TerminateSession>(Channel); }
>> >> > +
>> >> > +private:
>> >> > + OrcRemoteTargetClient(ChannelT &Channel, std::error_code &EC)
>> >> > + : Channel(Channel), RemotePointerSize(0), RemotePageSize(0),
>> >> > + RemoteTrampolineSize(0), RemoteIndirectStubSize(0) {
>> >> > + if ((EC = call<GetRemoteInfo>(Channel)))
>> >> > + return;
>> >> > +
>> >> > + EC = expect<GetRemoteInfoResponse>(
>> >> > + Channel, readArgs(RemoteTargetTriple, RemotePointerSize,
>> >> > RemotePageSize,
>> >> > + RemoteTrampolineSize,
>> >> > RemoteIndirectStubSize));
>> >> > + }
>> >> > +
>> >> > + void destroyRemoteAllocator(ResourceIdMgr::ResourceId Id) {
>> >> > + if (auto EC = call<DestroyRemoteAllocator>(Channel, Id)) {
>> >> > + // FIXME: This will be triggered by a removeModuleSet call:
>> >> > Propagate
>> >> > + // error return up through that.
>> >> > + llvm_unreachable("Failed to destroy remote allocator.");
>> >> > + AllocatorIds.release(Id);
>> >> > + }
>> >> > + }
>> >> > +
>> >> > + std::error_code
>> >> > destroyIndirectStubsManager(ResourceIdMgr::ResourceId
>> >> > Id) {
>> >> > + IndirectStubOwnerIds.release(Id);
>> >> > + return call<DestroyIndirectStubsOwner>(Channel, Id);
>> >> > + }
>> >> > +
>> >> > + std::error_code emitIndirectStubs(TargetAddress &StubBase,
>> >> > + TargetAddress &PtrBase,
>> >> > + uint32_t &NumStubsEmitted,
>> >> > + ResourceIdMgr::ResourceId Id,
>> >> > + uint32_t NumStubsRequired) {
>> >> > + if (auto EC = call<EmitIndirectStubs>(Channel, Id,
>> >> > NumStubsRequired))
>> >> > + return EC;
>> >> > +
>> >> > + return expect<EmitIndirectStubsResponse>(
>> >> > + Channel, readArgs(StubBase, PtrBase, NumStubsEmitted));
>> >> > + }
>> >> > +
>> >> > + std::error_code emitResolverBlock() {
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + return call<EmitResolverBlock>(Channel);
>> >> > + }
>> >> > +
>> >> > + std::error_code emitTrampolineBlock(TargetAddress &BlockAddr,
>> >> > + uint32_t &NumTrampolines) {
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + if (auto EC = call<EmitTrampolineBlock>(Channel))
>> >> > + return EC;
>> >> > +
>> >> > + return expect<EmitTrampolineBlockResponse>(
>> >> > + Channel, [&](TargetAddress BAddr, uint32_t NTrampolines) {
>> >> > + BlockAddr = BAddr;
>> >> > + NumTrampolines = NTrampolines;
>> >> > + return std::error_code();
>> >> > + });
>> >> > + }
>> >> > +
>> >> > + uint32_t getIndirectStubSize() const { return
>> >> > RemoteIndirectStubSize;
>> >> > }
>> >> > + uint32_t getPageSize() const { return RemotePageSize; }
>> >> > + uint32_t getPointerSize() const { return RemotePointerSize; }
>> >> > +
>> >> > + uint32_t getTrampolineSize() const { return RemoteTrampolineSize;
>> >> > }
>> >> > +
>> >> > + std::error_code listenForCompileRequests(uint32_t &NextId) {
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + if (auto EC = getNextProcId(Channel, NextId))
>> >> > + return EC;
>> >> > +
>> >> > + while (NextId == RequestCompileId) {
>> >> > + TargetAddress TrampolineAddr = 0;
>> >> > + if (auto EC = handle<RequestCompile>(Channel,
>> >> > readArgs(TrampolineAddr)))
>> >> > + return EC;
>> >> > +
>> >> > + TargetAddress ImplAddr = CompileCallback(TrampolineAddr);
>> >> > + if (auto EC = call<RequestCompileResponse>(Channel, ImplAddr))
>> >> > + return EC;
>> >> > +
>> >> > + if (auto EC = getNextProcId(Channel, NextId))
>> >> > + return EC;
>> >> > + }
>> >> > +
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + std::error_code readMem(char *Dst, TargetAddress Src, uint64_t
>> >> > Size)
>> >> > {
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + if (auto EC = call<ReadMem>(Channel, Src, Size))
>> >> > + return EC;
>> >> > +
>> >> > + if (auto EC = expect<ReadMemResponse>(
>> >> > + Channel, [&]() { return Channel.readBytes(Dst, Size);
>> >> > }))
>> >> > + return EC;
>> >> > +
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + std::error_code reserveMem(TargetAddress &RemoteAddr,
>> >> > + ResourceIdMgr::ResourceId Id, uint64_t
>> >> > Size,
>> >> > + uint32_t Align) {
>> >> > +
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + if (auto EC = call<ReserveMem>(Channel, Id, Size, Align))
>> >> > + return EC;
>> >> > +
>> >> > + if (auto EC = expect<ReserveMemResponse>(Channel,
>> >> > [&](TargetAddress
>> >> > Addr) {
>> >> > + RemoteAddr = Addr;
>> >> > + return std::error_code();
>> >> > + }))
>> >> > + return EC;
>> >> > +
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + std::error_code setProtections(ResourceIdMgr::ResourceId Id,
>> >> > + TargetAddress RemoteSegAddr,
>> >> > + unsigned ProtFlags) {
>> >> > + return call<SetProtections>(Channel, Id, RemoteSegAddr,
>> >> > ProtFlags);
>> >> > + }
>> >> > +
>> >> > + std::error_code writeMem(TargetAddress Addr, const char *Src,
>> >> > uint64_t Size) {
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + // Make the send call.
>> >> > + if (auto EC = call<WriteMem>(Channel, Addr, Size))
>> >> > + return EC;
>> >> > +
>> >> > + // Follow this up with the section contents.
>> >> > + if (auto EC = Channel.appendBytes(Src, Size))
>> >> > + return EC;
>> >> > +
>> >> > + return Channel.send();
>> >> > + }
>> >> > +
>> >> > + std::error_code writePointer(TargetAddress Addr, TargetAddress
>> >> > PtrVal) {
>> >> > + // Check for an 'out-of-band' error, e.g. from an MM destructor.
>> >> > + if (ExistingError)
>> >> > + return ExistingError;
>> >> > +
>> >> > + return call<WritePtr>(Channel, Addr, PtrVal);
>> >> > + }
>> >> > +
>> >> > + static std::error_code doNothing() { return std::error_code(); }
>> >> > +
>> >> > + ChannelT &Channel;
>> >> > + std::error_code ExistingError;
>> >> > + std::string RemoteTargetTriple;
>> >> > + uint32_t RemotePointerSize;
>> >> > + uint32_t RemotePageSize;
>> >> > + uint32_t RemoteTrampolineSize;
>> >> > + uint32_t RemoteIndirectStubSize;
>> >> > + ResourceIdMgr AllocatorIds, IndirectStubOwnerIds;
>> >> > + std::function<TargetAddress(TargetAddress)> CompileCallback;
>> >> > +};
>> >> > +
>> >> > +} // end namespace remote
>> >> > +} // end namespace orc
>> >> > +} // end namespace llvm
>> >> > +
>> >> > +#undef DEBUG_TYPE
>> >> > +
>> >> > +#endif
>> >> >
>> >> > Added:
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
>> >> > URL:
>> >> >
>> >> > http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h?rev=257305&view=auto
>> >> >
>> >> >
>> >> > ==============================================================================
>> >> > ---
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
>> >> > (added)
>> >> > +++
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
>> >> > Sun Jan 10 19:40:11 2016
>> >> > @@ -0,0 +1,185 @@
>> >> > +//===--- OrcRemoteTargetRPCAPI.h - Orc Remote-target RPC API ----*-
>> >> > C++
>> >> > -*-===//
>> >> > +//
>> >> > +// The LLVM Compiler Infrastructure
>> >> > +//
>> >> > +// This file is distributed under the University of Illinois Open
>> >> > Source
>> >> > +// License. See LICENSE.TXT for details.
>> >> > +//
>> >> >
>> >> >
>> >> > +//===----------------------------------------------------------------------===//
>> >> > +//
>> >> > +// This file defines the Orc remote-target RPC API. It should not be
>> >> > used
>> >> > +// directly, but is used by the RemoteTargetClient and
>> >> > RemoteTargetServer
>> >> > +// classes.
>> >> > +//
>> >> >
>> >> >
>> >> > +//===----------------------------------------------------------------------===//
>> >> > +
>> >> > +#ifndef LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETRPCAPI_H
>> >> > +#define LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETRPCAPI_H
>> >> > +
>> >> > +#include "JITSymbol.h"
>> >> > +#include "RPCChannel.h"
>> >> > +#include "RPCUtils.h"
>> >> > +
>> >> > +namespace llvm {
>> >> > +namespace orc {
>> >> > +namespace remote {
>> >> > +
>> >> > +class OrcRemoteTargetRPCAPI : public RPC<RPCChannel> {
>> >> > +protected:
>> >> > + class ResourceIdMgr {
>> >> > + public:
>> >> > + typedef uint64_t ResourceId;
>> >> > + ResourceIdMgr() : NextId(0) {}
>> >> > + ResourceId getNext() {
>> >> > + if (!FreeIds.empty()) {
>> >> > + ResourceId I = FreeIds.back();
>> >> > + FreeIds.pop_back();
>> >> > + return I;
>> >> > + }
>> >> > + return NextId++;
>> >> > + }
>> >> > + void release(ResourceId I) { FreeIds.push_back(I); }
>> >> > +
>> >> > + private:
>> >> > + ResourceId NextId;
>> >> > + std::vector<ResourceId> FreeIds;
>> >> > + };
>> >> > +
>> >> > +public:
>> >> > + enum JITProcId : uint32_t {
>> >> > + InvalidId = 0,
>> >> > + CallIntVoidId,
>> >> > + CallIntVoidResponseId,
>> >> > + CallMainId,
>> >> > + CallMainResponseId,
>> >> > + CallVoidVoidId,
>> >> > + CallVoidVoidResponseId,
>> >> > + CreateRemoteAllocatorId,
>> >> > + CreateIndirectStubsOwnerId,
>> >> > + DestroyRemoteAllocatorId,
>> >> > + DestroyIndirectStubsOwnerId,
>> >> > + EmitIndirectStubsId,
>> >> > + EmitIndirectStubsResponseId,
>> >> > + EmitResolverBlockId,
>> >> > + EmitTrampolineBlockId,
>> >> > + EmitTrampolineBlockResponseId,
>> >> > + GetSymbolAddressId,
>> >> > + GetSymbolAddressResponseId,
>> >> > + GetRemoteInfoId,
>> >> > + GetRemoteInfoResponseId,
>> >> > + ReadMemId,
>> >> > + ReadMemResponseId,
>> >> > + ReserveMemId,
>> >> > + ReserveMemResponseId,
>> >> > + RequestCompileId,
>> >> > + RequestCompileResponseId,
>> >> > + SetProtectionsId,
>> >> > + TerminateSessionId,
>> >> > + WriteMemId,
>> >> > + WritePtrId
>> >> > + };
>> >> > +
>> >> > + static const char *getJITProcIdName(JITProcId Id);
>> >> > +
>> >> > + typedef Procedure<CallIntVoidId, TargetAddress /* FnAddr */>
>> >> > CallIntVoid;
>> >> > +
>> >> > + typedef Procedure<CallIntVoidResponseId, int /* Result */>
>> >> > + CallIntVoidResponse;
>> >> > +
>> >> > + typedef Procedure<CallMainId, TargetAddress /* FnAddr */,
>> >> > + std::vector<std::string> /* Args */>
>> >> > + CallMain;
>> >> > +
>> >> > + typedef Procedure<CallMainResponseId, int /* Result */>
>> >> > CallMainResponse;
>> >> > +
>> >> > + typedef Procedure<CallVoidVoidId, TargetAddress /* FnAddr */>
>> >> > CallVoidVoid;
>> >> > +
>> >> > + typedef Procedure<CallVoidVoidResponseId> CallVoidVoidResponse;
>> >> > +
>> >> > + typedef Procedure<CreateRemoteAllocatorId,
>> >> > + ResourceIdMgr::ResourceId /* Allocator ID */>
>> >> > + CreateRemoteAllocator;
>> >> > +
>> >> > + typedef Procedure<CreateIndirectStubsOwnerId,
>> >> > + ResourceIdMgr::ResourceId /* StubsOwner ID */>
>> >> > + CreateIndirectStubsOwner;
>> >> > +
>> >> > + typedef Procedure<DestroyRemoteAllocatorId,
>> >> > + ResourceIdMgr::ResourceId /* Allocator ID */>
>> >> > + DestroyRemoteAllocator;
>> >> > +
>> >> > + typedef Procedure<DestroyIndirectStubsOwnerId,
>> >> > + ResourceIdMgr::ResourceId /* StubsOwner ID */>
>> >> > + DestroyIndirectStubsOwner;
>> >> > +
>> >> > + typedef Procedure<EmitIndirectStubsId,
>> >> > + ResourceIdMgr::ResourceId /* StubsOwner ID */,
>> >> > + uint32_t /* NumStubsRequired */>
>> >> > + EmitIndirectStubs;
>> >> > +
>> >> > + typedef Procedure<
>> >> > + EmitIndirectStubsResponseId, TargetAddress /* StubsBaseAddr
>> >> > */,
>> >> > + TargetAddress /* PtrsBaseAddr */, uint32_t /* NumStubsEmitted
>> >> > */>
>> >> > + EmitIndirectStubsResponse;
>> >> > +
>> >> > + typedef Procedure<EmitResolverBlockId> EmitResolverBlock;
>> >> > +
>> >> > + typedef Procedure<EmitTrampolineBlockId> EmitTrampolineBlock;
>> >> > +
>> >> > + typedef Procedure<EmitTrampolineBlockResponseId,
>> >> > + TargetAddress /* BlockAddr */,
>> >> > + uint32_t /* NumTrampolines */>
>> >> > + EmitTrampolineBlockResponse;
>> >> > +
>> >> > + typedef Procedure<GetSymbolAddressId, std::string /*SymbolName*/>
>> >> > + GetSymbolAddress;
>> >> > +
>> >> > + typedef Procedure<GetSymbolAddressResponseId, uint64_t /*
>> >> > SymbolAddr
>> >> > */>
>> >> > + GetSymbolAddressResponse;
>> >> > +
>> >> > + typedef Procedure<GetRemoteInfoId> GetRemoteInfo;
>> >> > +
>> >> > + typedef Procedure<GetRemoteInfoResponseId, std::string /* Triple
>> >> > */,
>> >> > + uint32_t /* PointerSize */, uint32_t /* PageSize
>> >> > */,
>> >> > + uint32_t /* TrampolineSize */,
>> >> > + uint32_t /* IndirectStubSize */>
>> >> > + GetRemoteInfoResponse;
>> >> > +
>> >> > + typedef Procedure<ReadMemId, TargetAddress /* Src */, uint64_t /*
>> >> > Size */>
>> >> > + ReadMem;
>> >> > +
>> >> > + typedef Procedure<ReadMemResponseId> ReadMemResponse;
>> >> > +
>> >> > + typedef Procedure<ReserveMemId, ResourceIdMgr::ResourceId /* Id
>> >> > */,
>> >> > + uint64_t /* Size */, uint32_t /* Align */>
>> >> > + ReserveMem;
>> >> > +
>> >> > + typedef Procedure<ReserveMemResponseId, TargetAddress /* Addr */>
>> >> > + ReserveMemResponse;
>> >> > +
>> >> > + typedef Procedure<RequestCompileId, TargetAddress /*
>> >> > TrampolineAddr
>> >> > */>
>> >> > + RequestCompile;
>> >> > +
>> >> > + typedef Procedure<RequestCompileResponseId, TargetAddress /*
>> >> > ImplAddr
>> >> > */>
>> >> > + RequestCompileResponse;
>> >> > +
>> >> > + typedef Procedure<SetProtectionsId, ResourceIdMgr::ResourceId /*
>> >> > Id
>> >> > */,
>> >> > + TargetAddress /* Dst */, uint32_t /* ProtFlags
>> >> > */>
>> >> > + SetProtections;
>> >> > +
>> >> > + typedef Procedure<TerminateSessionId> TerminateSession;
>> >> > +
>> >> > + typedef Procedure<WriteMemId, TargetAddress /* Dst */, uint64_t /*
>> >> > Size */
>> >> > + /* Data should follow */>
>> >> > + WriteMem;
>> >> > +
>> >> > + typedef Procedure<WritePtrId, TargetAddress /* Dst */,
>> >> > + TargetAddress /* Val */>
>> >> > + WritePtr;
>> >> > +};
>> >> > +
>> >> > +} // end namespace remote
>> >> > +} // end namespace orc
>> >> > +} // end namespace llvm
>> >> > +
>> >> > +#endif
>> >> >
>> >> > Added:
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
>> >> > URL:
>> >> >
>> >> > http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h?rev=257305&view=auto
>> >> >
>> >> >
>> >> > ==============================================================================
>> >> > ---
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
>> >> > (added)
>> >> > +++
>> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
>> >> > Sun Jan 10 19:40:11 2016
>> >> > @@ -0,0 +1,479 @@
>> >> > +//===---- OrcRemoteTargetServer.h - Orc Remote-target Server ----*-
>> >> > C++
>> >> > -*-===//
>> >> > +//
>> >> > +// The LLVM Compiler Infrastructure
>> >> > +//
>> >> > +// This file is distributed under the University of Illinois Open
>> >> > Source
>> >> > +// License. See LICENSE.TXT for details.
>> >> > +//
>> >> >
>> >> >
>> >> > +//===----------------------------------------------------------------------===//
>> >> > +//
>> >> > +// This file defines the OrcRemoteTargetServer class. It can be used
>> >> > to
>> >> > build a
>> >> > +// JIT server that can execute code sent from an
>> >> > OrcRemoteTargetClient.
>> >> > +//
>> >> >
>> >> >
>> >> > +//===----------------------------------------------------------------------===//
>> >> > +
>> >> > +#ifndef LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETSERVER_H
>> >> > +#define LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETSERVER_H
>> >> > +
>> >> > +#include "OrcRemoteTargetRPCAPI.h"
>> >> > +#include "llvm/ExecutionEngine/RTDyldMemoryManager.h"
>> >> > +#include "llvm/Support/Debug.h"
>> >> > +#include "llvm/Support/Format.h"
>> >> > +#include "llvm/Support/Process.h"
>> >> > +#include "llvm/Support/raw_ostream.h"
>> >> > +#include <map>
>> >> > +
>> >> > +#define DEBUG_TYPE "orc-remote"
>> >> > +
>> >> > +namespace llvm {
>> >> > +namespace orc {
>> >> > +namespace remote {
>> >> > +
>> >> > +template <typename ChannelT, typename TargetT>
>> >> > +class OrcRemoteTargetServer : public OrcRemoteTargetRPCAPI {
>> >> > +public:
>> >> > + typedef std::function<TargetAddress(const std::string &Name)>
>> >> > + SymbolLookupFtor;
>> >> > +
>> >> > + OrcRemoteTargetServer(ChannelT &Channel, SymbolLookupFtor
>> >> > SymbolLookup)
>> >> > + : Channel(Channel), SymbolLookup(std::move(SymbolLookup)) {}
>> >> > +
>> >> > + std::error_code getNextProcId(JITProcId &Id) {
>> >> > + return deserialize(Channel, Id);
>> >> > + }
>> >> > +
>> >> > + std::error_code handleKnownProcedure(JITProcId Id) {
>> >> > + DEBUG(dbgs() << "Handling known proc: " << getJITProcIdName(Id)
>> >> > <<
>> >> > "\n");
>> >> > +
>> >> > + switch (Id) {
>> >> > + case CallIntVoidId:
>> >> > + return handleCallIntVoid();
>> >> > + case CallMainId:
>> >> > + return handleCallMain();
>> >> > + case CallVoidVoidId:
>> >> > + return handleCallVoidVoid();
>> >> > + case CreateRemoteAllocatorId:
>> >> > + return handleCreateRemoteAllocator();
>> >> > + case CreateIndirectStubsOwnerId:
>> >> > + return handleCreateIndirectStubsOwner();
>> >> > + case DestroyRemoteAllocatorId:
>> >> > + return handleDestroyRemoteAllocator();
>> >> > + case EmitIndirectStubsId:
>> >> > + return handleEmitIndirectStubs();
>> >> > + case EmitResolverBlockId:
>> >> > + return handleEmitResolverBlock();
>> >> > + case EmitTrampolineBlockId:
>> >> > + return handleEmitTrampolineBlock();
>> >> > + case GetSymbolAddressId:
>> >> > + return handleGetSymbolAddress();
>> >> > + case GetRemoteInfoId:
>> >> > + return handleGetRemoteInfo();
>> >> > + case ReadMemId:
>> >> > + return handleReadMem();
>> >> > + case ReserveMemId:
>> >> > + return handleReserveMem();
>> >> > + case SetProtectionsId:
>> >> > + return handleSetProtections();
>> >> > + case WriteMemId:
>> >> > + return handleWriteMem();
>> >> > + case WritePtrId:
>> >> > + return handleWritePtr();
>> >> > + default:
>> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
>> >> > + }
>> >> > +
>> >> > + llvm_unreachable("Unhandled JIT RPC procedure Id.");
>> >> > + }
>> >> > +
>> >> > + std::error_code requestCompile(TargetAddress &CompiledFnAddr,
>> >> > + TargetAddress TrampolineAddr) {
>> >> > + if (auto EC = call<RequestCompile>(Channel, TrampolineAddr))
>> >> > + return EC;
>> >> > +
>> >> > + while (1) {
>> >> > + JITProcId Id = InvalidId;
>> >> > + if (auto EC = getNextProcId(Id))
>> >> > + return EC;
>> >> > +
>> >> > + switch (Id) {
>> >> > + case RequestCompileResponseId:
>> >> > + return handle<RequestCompileResponse>(Channel,
>> >> > +
>> >> > readArgs(CompiledFnAddr));
>> >> > + default:
>> >> > + if (auto EC = handleKnownProcedure(Id))
>> >> > + return EC;
>> >> > + }
>> >> > + }
>> >> > +
>> >> > + llvm_unreachable("Fell through request-compile command loop.");
>> >> > + }
>> >> > +
>> >> > +private:
>> >> > + struct Allocator {
>> >> > + Allocator() = default;
>> >> > + Allocator(Allocator &&) = default;
>> >> > + Allocator &operator=(Allocator &&) = default;
>> >> > +
>> >> > + ~Allocator() {
>> >> > + for (auto &Alloc : Allocs)
>> >> > + sys::Memory::releaseMappedMemory(Alloc.second);
>> >> > + }
>> >> > +
>> >> > + std::error_code allocate(void *&Addr, size_t Size, uint32_t
>> >> > Align)
>> >> > {
>> >> > + std::error_code EC;
>> >> > + sys::MemoryBlock MB = sys::Memory::allocateMappedMemory(
>> >> > + Size, nullptr, sys::Memory::MF_READ |
>> >> > sys::Memory::MF_WRITE,
>> >> > EC);
>> >> > + if (EC)
>> >> > + return EC;
>> >> > +
>> >> > + Addr = MB.base();
>> >> > + assert(Allocs.find(MB.base()) == Allocs.end() && "Duplicate
>> >> > alloc");
>> >> > + Allocs[MB.base()] = std::move(MB);
>> >> > + return std::error_code();
>> >> > + }
>> >> > +
>> >> > + std::error_code setProtections(void *block, unsigned Flags) {
>> >> > + auto I = Allocs.find(block);
>> >> > + if (I == Allocs.end())
>> >> > + return
>> >> > orcError(OrcErrorCode::RemoteMProtectAddrUnrecognized);
>> >> > + return sys::Memory::protectMappedMemory(I->second, Flags);
>> >> > + }
>> >> > +
>> >> > + private:
>> >> > + std::map<void *, sys::MemoryBlock> Allocs;
>> >> > + };
>> >> > +
>> >> > + static std::error_code doNothing() { return std::error_code(); }
>> >> > +
>> >> > + static TargetAddress reenter(void *JITTargetAddr, void
>> >> > *TrampolineAddr) {
>> >> > + TargetAddress CompiledFnAddr = 0;
>> >> > +
>> >> > + auto T = static_cast<OrcRemoteTargetServer *>(JITTargetAddr);
>> >> > + auto EC = T->requestCompile(
>> >> > + CompiledFnAddr, static_cast<TargetAddress>(
>> >> > +
>> >> > reinterpret_cast<uintptr_t>(TrampolineAddr)));
>> >> > + assert(!EC && "Compile request failed");
>> >> > + return CompiledFnAddr;
>> >> > + }
>> >> > +
>> >> > + std::error_code handleCallIntVoid() {
>> >> > + typedef int (*IntVoidFnTy)();
>> >> > +
>> >> > + IntVoidFnTy Fn = nullptr;
>> >> > + if (auto EC = handle<CallIntVoid>(Channel, [&](TargetAddress
>> >> > Addr)
>> >> > {
>> >> > + Fn =
>> >> > reinterpret_cast<IntVoidFnTy>(static_cast<uintptr_t>(Addr));
>> >> > + return std::error_code();
>> >> > + }))
>> >> > + return EC;
>> >> > +
>> >> > + DEBUG(dbgs() << " Calling " << reinterpret_cast<void *>(Fn) <<
>> >> > "\n");
>> >> > + int Result = Fn();
>> >> > + DEBUG(dbgs() << " Result = " << Result << "\n");
>> >> > +
>> >> > + return call<CallIntVoidResponse>(Channel, Result);
>> >> > + }
>> >> > +
>> >> > + std::error_code handleCallMain() {
>> >> > + typedef int (*MainFnTy)(int, const char *[]);
>> >> > +
>> >> > + MainFnTy Fn = nullptr;
>> >> > + std::vector<std::string> Args;
>> >> > + if (auto EC = handle<CallMain>(
>> >> > + Channel, [&](TargetAddress Addr,
>> >> > std::vector<std::string>
>> >> > &A) {
>> >> > + Fn =
>> >> > reinterpret_cast<MainFnTy>(static_cast<uintptr_t>(Addr));
>> >> > + Args = std::move(A);
>> >> > + return std::error_code();
>> >> > + }))
>> >> > + return EC;
>> >> > +
>> >> > + int ArgC = Args.size() + 1;
>> >> > + int Idx = 1;
>> >> > + std::unique_ptr<const char *[]> ArgV(new const char *[ArgC +
>> >> > 1]);
>> >> > + ArgV[0] = "<jit process>";
>> >> > + for (auto &Arg : Args)
>> >> > + ArgV[Idx++] = Arg.c_str();
>> >> > +
>> >> > + DEBUG(dbgs() << " Calling " << reinterpret_cast<void *>(Fn) <<
>> >> > "\n");
>> >>
>> >> This is causing a warning:
>> >>
>> >>
>> >>
>> >> /opt/llvm/build-llvm/src/llvm/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h:215:5:
>> >> warning: ISO C++ forbids casting between pointer-to-function and
>> >> pointer-to-object [enabled by default]
>> >
>> >
>> > I'm not seeing this on a clang self-host. Which compiler is warning
>> > about
>> > this? (should we enable this warning when building with clang, or
>> > disable it
>> > when building with whatever is firing it?)
>>
>> I'm not certain of the compiler version information. This is on our
>> attribute documentation bot, which I *think* is GCC. Tanya would know
>> more if we need exact details.
>>
>> I think it's a reasonable diagnostic to keep for portability reasons.
>> We usually do pretty good about keeping this one clean, though it does
>> crop up from time to time.
>
>
> Hmm, interesting - Clang does have this warning and we do build with
> -pedantic by default, but Clang does /not/ fire this warning when building
> in C++11 mode...
>
> The implementation in Clang (SemaCast.cpp:2022) mentions that this behavior
> is "conditionally supported" - which is, implementations may allow it and if
> they allow the conversion in both directions then it must yield the original
> pointer value (it must roundtrip).
>
> The comment also mentions that compilers generally support it even in C++03
> mode, because you need it to use dlsym and GetProcAddress. So it seems we
> probably require this of any implementation we use anyway.
That is true -- we do require that behavior to be supported anyway,
and if it's not supported we're likely to find out very quickly. So
perhaps we can simply disable that diagnostic for the various
compilers.
~Aaron
>
>>
>>
>> > If it's worth keeping, the easy/reasonable fix is to cast the function
>> > pointer to intptr_t first, then to void* to print. (or just to intptr_t
>> > then
>> > hex format with llvm::format)
>>
>> Yes, I just wasn't certain whether Lang wanted to keep the DEBUG()
>> statements or not, since another easy fix is to remove the debug
>> logging if it's not required.
>>
>> ~Aaron
>
>
More information about the llvm-commits
mailing list