[llvm] r257305 - [Orc] Add support for remote JITing to the ORC API.
David Blaikie via llvm-commits
llvm-commits at lists.llvm.org
Fri Jan 15 11:45:45 PST 2016
On Fri, Jan 15, 2016 at 11:26 AM, Aaron Ballman <aaron at aaronballman.com>
wrote:
> On Fri, Jan 15, 2016 at 2:23 PM, David Blaikie <dblaikie at gmail.com> wrote:
> >
> >
> > On Fri, Jan 15, 2016 at 9:21 AM, Aaron Ballman <aaron at aaronballman.com>
> > wrote:
> >>
> >> On Fri, Jan 15, 2016 at 12:12 PM, David Blaikie <dblaikie at gmail.com>
> >> wrote:
> >> >
> >> >
> >> > On Fri, Jan 15, 2016 at 8:33 AM, Aaron Ballman via llvm-commits
> >> > <llvm-commits at lists.llvm.org> wrote:
> >> >>
> >> >> On Sun, Jan 10, 2016 at 8:40 PM, Lang Hames via llvm-commits
> >> >> <llvm-commits at lists.llvm.org> wrote:
> >> >> > Author: lhames
> >> >> > Date: Sun Jan 10 19:40:11 2016
> >> >> > New Revision: 257305
> >> >> >
> >> >> > URL: http://llvm.org/viewvc/llvm-project?rev=257305&view=rev
> >> >> > Log:
> >> >> > [Orc] Add support for remote JITing to the ORC API.
> >> >> >
> >> >> > This patch adds utilities to ORC for managing a remote JIT target.
> It
> >> >> > consists
> >> >> > of:
> >> >> >
> >> >> > 1. A very primitive RPC system for making calls over a byte-stream.
> >> >> > See
> >> >> > RPCChannel.h, RPCUtils.h.
> >> >> >
> >> >> > 2. An RPC API defined in the above system for managing memory,
> >> >> > looking
> >> >> > up
> >> >> > symbols, creating stubs, etc. on a remote target. See
> >> >> > OrcRemoteTargetRPCAPI.h.
> >> >> >
> >> >> > 3. An interface for creating high-level JIT components (memory
> >> >> > managers,
> >> >> > callback managers, stub managers, etc.) that operate over the RPC
> >> >> > API.
> >> >> > See
> >> >> > OrcRemoteTargetClient.h.
> >> >> >
> >> >> > 4. A helper class for building servers that can handle the RPC
> calls.
> >> >> > See
> >> >> > OrcRemoteTargetServer.h.
> >> >> >
> >> >> > The system is designed to work neatly with the existing ORC
> >> >> > components
> >> >> > and
> >> >> > functionality. In particular, the ORC callback API (and
> consequently
> >> >> > the
> >> >> > CompileOnDemandLayer) is supported, enabling lazy compilation of
> >> >> > remote
> >> >> > code.
> >> >> >
> >> >> > Assuming this doesn't trigger any builder failures, a follow-up
> patch
> >> >> > will be
> >> >> > committed which tests these utilities by using them to replace
> LLI's
> >> >> > existing
> >> >> > remote-JITing demo code.
> >> >> >
> >> >> >
> >> >> > Added:
> >> >> >
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
> >> >> >
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
> >> >> >
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/RPCChannel.h
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/RPCUtils.h
> >> >> > llvm/trunk/lib/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.cpp
> >> >> > llvm/trunk/unittests/ExecutionEngine/Orc/RPCUtilsTest.cpp
> >> >> > Modified:
> >> >> > llvm/trunk/lib/ExecutionEngine/Orc/CMakeLists.txt
> >> >> > llvm/trunk/unittests/ExecutionEngine/Orc/CMakeLists.txt
> >> >> >
> >> >> > Added:
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
> >> >> > URL:
> >> >> >
> >> >> >
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h?rev=257305&view=auto
> >> >> >
> >> >> >
> >> >> >
> ==============================================================================
> >> >> > ---
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
> >> >> > (added)
> >> >> > +++
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
> >> >> > Sun Jan 10 19:40:11 2016
> >> >> > @@ -0,0 +1,743 @@
> >> >> > +//===---- OrcRemoteTargetClient.h - Orc Remote-target Client
> ----*-
> >> >> > C++
> >> >> > -*-===//
> >> >> > +//
> >> >> > +// The LLVM Compiler Infrastructure
> >> >> > +//
> >> >> > +// This file is distributed under the University of Illinois Open
> >> >> > Source
> >> >> > +// License. See LICENSE.TXT for details.
> >> >> > +//
> >> >> >
> >> >> >
> >> >> >
> +//===----------------------------------------------------------------------===//
> >> >> > +//
> >> >> > +// This file defines the OrcRemoteTargetClient class and helpers.
> >> >> > This
> >> >> > class
> >> >> > +// can be used to communicate over an RPCChannel with an
> >> >> > OrcRemoteTargetServer
> >> >> > +// instance to support remote-JITing.
> >> >> > +//
> >> >> >
> >> >> >
> >> >> >
> +//===----------------------------------------------------------------------===//
> >> >> > +
> >> >> > +#ifndef LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETCLIENT_H
> >> >> > +#define LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETCLIENT_H
> >> >> > +
> >> >> > +#include "OrcRemoteTargetRPCAPI.h"
> >> >> > +
> >> >> > +#define DEBUG_TYPE "orc-remote"
> >> >> > +
> >> >> > +namespace llvm {
> >> >> > +namespace orc {
> >> >> > +namespace remote {
> >> >> > +
> >> >> > +/// This class provides utilities (including memory manager,
> >> >> > indirect
> >> >> > stubs
> >> >> > +/// manager, and compile callback manager types) that support
> remote
> >> >> > JITing
> >> >> > +/// in ORC.
> >> >> > +///
> >> >> > +/// Each of the utility classes talks to a JIT server (an instance
> >> >> > of
> >> >> > the
> >> >> > +/// OrcRemoteTargetServer class) via an RPC system (see
> RPCUtils.h)
> >> >> > to
> >> >> > carry out
> >> >> > +/// its actions.
> >> >> > +template <typename ChannelT>
> >> >> > +class OrcRemoteTargetClient : public OrcRemoteTargetRPCAPI {
> >> >> > +public:
> >> >> > + /// Remote memory manager.
> >> >> > + class RCMemoryManager : public RuntimeDyld::MemoryManager {
> >> >> > + public:
> >> >> > + RCMemoryManager(OrcRemoteTargetClient &Client,
> >> >> > ResourceIdMgr::ResourceId Id)
> >> >> > + : Client(Client), Id(Id) {
> >> >> > + DEBUG(dbgs() << "Created remote allocator " << Id << "\n");
> >> >> > + }
> >> >> > +
> >> >> > + ~RCMemoryManager() {
> >> >> > + Client.destroyRemoteAllocator(Id);
> >> >> > + DEBUG(dbgs() << "Destroyed remote allocator " << Id <<
> "\n");
> >> >> > + }
> >> >> > +
> >> >> > + uint8_t *allocateCodeSection(uintptr_t Size, unsigned
> Alignment,
> >> >> > + unsigned SectionID,
> >> >> > + StringRef SectionName) override {
> >> >> > + Unmapped.back().CodeAllocs.emplace_back(Size, Alignment);
> >> >> > + uint8_t *Alloc = reinterpret_cast<uint8_t *>(
> >> >> > + Unmapped.back().CodeAllocs.back().getLocalAddress());
> >> >> > + DEBUG(dbgs() << "Allocator " << Id << " allocated code for "
> >> >> > + << SectionName << ": " << Alloc << " (" << Size
> >> >> > + << " bytes, alignment " << Alignment << ")\n");
> >> >> > + return Alloc;
> >> >> > + }
> >> >> > +
> >> >> > + uint8_t *allocateDataSection(uintptr_t Size, unsigned
> Alignment,
> >> >> > + unsigned SectionID, StringRef
> >> >> > SectionName,
> >> >> > + bool IsReadOnly) override {
> >> >> > + if (IsReadOnly) {
> >> >> > + Unmapped.back().RODataAllocs.emplace_back(Size,
> Alignment);
> >> >> > + uint8_t *Alloc = reinterpret_cast<uint8_t *>(
> >> >> > +
> Unmapped.back().RODataAllocs.back().getLocalAddress());
> >> >> > + DEBUG(dbgs() << "Allocator " << Id << " allocated ro-data
> >> >> > for "
> >> >> > + << SectionName << ": " << Alloc << " (" <<
> Size
> >> >> > + << " bytes, alignment " << Alignment <<
> ")\n");
> >> >> > + return Alloc;
> >> >> > + } // else...
> >> >> > +
> >> >> > + Unmapped.back().RWDataAllocs.emplace_back(Size, Alignment);
> >> >> > + uint8_t *Alloc = reinterpret_cast<uint8_t *>(
> >> >> > + Unmapped.back().RWDataAllocs.back().getLocalAddress());
> >> >> > + DEBUG(dbgs() << "Allocator " << Id << " allocated rw-data
> for
> >> >> > "
> >> >> > + << SectionName << ": " << Alloc << " (" << Size
> >> >> > + << " bytes, alignment " << Alignment << "\n");
> >> >> > + return Alloc;
> >> >> > + }
> >> >> > +
> >> >> > + void reserveAllocationSpace(uintptr_t CodeSize, uint32_t
> >> >> > CodeAlign,
> >> >> > + uintptr_t RODataSize, uint32_t
> >> >> > RODataAlign,
> >> >> > + uintptr_t RWDataSize,
> >> >> > + uint32_t RWDataAlign) override {
> >> >> > + Unmapped.push_back(ObjectAllocs());
> >> >> > +
> >> >> > + DEBUG(dbgs() << "Allocator " << Id << " reserved:\n");
> >> >> > +
> >> >> > + if (CodeSize != 0) {
> >> >> > + if (auto EC =
> >> >> > Client.reserveMem(Unmapped.back().RemoteCodeAddr,
> >> >> > Id,
> >> >> > + CodeSize, CodeAlign)) {
> >> >> > + // FIXME; Add error to poll.
> >> >> > + llvm_unreachable("Failed reserving remote memory.");
> >> >> > + }
> >> >> > + DEBUG(dbgs() << " code: "
> >> >> > + << format("0x%016x",
> >> >> > Unmapped.back().RemoteCodeAddr)
> >> >> > + << " (" << CodeSize << " bytes, alignment "
> <<
> >> >> > CodeAlign
> >> >> > + << ")\n");
> >> >> > + }
> >> >> > +
> >> >> > + if (RODataSize != 0) {
> >> >> > + if (auto EC =
> >> >> > Client.reserveMem(Unmapped.back().RemoteRODataAddr, Id,
> >> >> > + RODataSize, RODataAlign))
> {
> >> >> > + // FIXME; Add error to poll.
> >> >> > + llvm_unreachable("Failed reserving remote memory.");
> >> >> > + }
> >> >> > + DEBUG(dbgs() << " ro-data: "
> >> >> > + << format("0x%016x",
> >> >> > Unmapped.back().RemoteRODataAddr)
> >> >> > + << " (" << RODataSize << " bytes, alignment "
> >> >> > + << RODataAlign << ")\n");
> >> >> > + }
> >> >> > +
> >> >> > + if (RWDataSize != 0) {
> >> >> > + if (auto EC =
> >> >> > Client.reserveMem(Unmapped.back().RemoteRWDataAddr, Id,
> >> >> > + RWDataSize, RWDataAlign))
> {
> >> >> > + // FIXME; Add error to poll.
> >> >> > + llvm_unreachable("Failed reserving remote memory.");
> >> >> > + }
> >> >> > + DEBUG(dbgs() << " rw-data: "
> >> >> > + << format("0x%016x",
> >> >> > Unmapped.back().RemoteRWDataAddr)
> >> >> > + << " (" << RWDataSize << " bytes, alignment "
> >> >> > + << RWDataAlign << ")\n");
> >> >> > + }
> >> >> > + }
> >> >> > +
> >> >> > + bool needsToReserveAllocationSpace() override { return true; }
> >> >> > +
> >> >> > + void registerEHFrames(uint8_t *Addr, uint64_t LoadAddr,
> >> >> > + size_t Size) override {}
> >> >> > +
> >> >> > + void deregisterEHFrames(uint8_t *addr, uint64_t LoadAddr,
> >> >> > + size_t Size) override {}
> >> >> > +
> >> >> > + void notifyObjectLoaded(RuntimeDyld &Dyld,
> >> >> > + const object::ObjectFile &Obj)
> override
> >> >> > {
> >> >> > + DEBUG(dbgs() << "Allocator " << Id << " applied
> mappings:\n");
> >> >> > + for (auto &ObjAllocs : Unmapped) {
> >> >> > + {
> >> >> > + TargetAddress NextCodeAddr = ObjAllocs.RemoteCodeAddr;
> >> >> > + for (auto &Alloc : ObjAllocs.CodeAllocs) {
> >> >> > + NextCodeAddr = RoundUpToAlignment(NextCodeAddr,
> >> >> > Alloc.getAlign());
> >> >> > + Dyld.mapSectionAddress(Alloc.getLocalAddress(),
> >> >> > NextCodeAddr);
> >> >> > + DEBUG(dbgs() << " code: "
> >> >> > + << static_cast<void
> >> >> > *>(Alloc.getLocalAddress())
> >> >> > + << " -> " << format("0x%016x",
> >> >> > NextCodeAddr)
> >> >> > << "\n");
> >> >> > + Alloc.setRemoteAddress(NextCodeAddr);
> >> >> > + NextCodeAddr += Alloc.getSize();
> >> >> > + }
> >> >> > + }
> >> >> > + {
> >> >> > + TargetAddress NextRODataAddr =
> ObjAllocs.RemoteRODataAddr;
> >> >> > + for (auto &Alloc : ObjAllocs.RODataAllocs) {
> >> >> > + NextRODataAddr =
> >> >> > + RoundUpToAlignment(NextRODataAddr,
> >> >> > Alloc.getAlign());
> >> >> > + Dyld.mapSectionAddress(Alloc.getLocalAddress(),
> >> >> > NextRODataAddr);
> >> >> > + DEBUG(dbgs() << " ro-data: "
> >> >> > + << static_cast<void
> >> >> > *>(Alloc.getLocalAddress())
> >> >> > + << " -> " << format("0x%016x",
> >> >> > NextRODataAddr)
> >> >> > + << "\n");
> >> >> > + Alloc.setRemoteAddress(NextRODataAddr);
> >> >> > + NextRODataAddr += Alloc.getSize();
> >> >> > + }
> >> >> > + }
> >> >> > + {
> >> >> > + TargetAddress NextRWDataAddr =
> ObjAllocs.RemoteRWDataAddr;
> >> >> > + for (auto &Alloc : ObjAllocs.RWDataAllocs) {
> >> >> > + NextRWDataAddr =
> >> >> > + RoundUpToAlignment(NextRWDataAddr,
> >> >> > Alloc.getAlign());
> >> >> > + Dyld.mapSectionAddress(Alloc.getLocalAddress(),
> >> >> > NextRWDataAddr);
> >> >> > + DEBUG(dbgs() << " rw-data: "
> >> >> > + << static_cast<void
> >> >> > *>(Alloc.getLocalAddress())
> >> >> > + << " -> " << format("0x%016x",
> >> >> > NextRWDataAddr)
> >> >> > + << "\n");
> >> >> > + Alloc.setRemoteAddress(NextRWDataAddr);
> >> >> > + NextRWDataAddr += Alloc.getSize();
> >> >> > + }
> >> >> > + }
> >> >> > + Unfinalized.push_back(std::move(ObjAllocs));
> >> >> > + }
> >> >> > + Unmapped.clear();
> >> >> > + }
> >> >> > +
> >> >> > + bool finalizeMemory(std::string *ErrMsg = nullptr) override {
> >> >> > + DEBUG(dbgs() << "Allocator " << Id << " finalizing:\n");
> >> >> > +
> >> >> > + for (auto &ObjAllocs : Unfinalized) {
> >> >> > +
> >> >> > + for (auto &Alloc : ObjAllocs.CodeAllocs) {
> >> >> > + DEBUG(dbgs() << " copying code: "
> >> >> > + << static_cast<void
> >> >> > *>(Alloc.getLocalAddress())
> >> >> > << " -> "
> >> >> > + << format("0x%016x",
> >> >> > Alloc.getRemoteAddress())
> >> >> > << " ("
> >> >> > + << Alloc.getSize() << " bytes)\n");
> >> >> > + Client.writeMem(Alloc.getRemoteAddress(),
> >> >> > Alloc.getLocalAddress(),
> >> >> > + Alloc.getSize());
> >> >> > + }
> >> >> > +
> >> >> > + if (ObjAllocs.RemoteCodeAddr) {
> >> >> > + DEBUG(dbgs() << " setting R-X permissions on code
> block:
> >> >> > "
> >> >> > + << format("0x%016x",
> >> >> > ObjAllocs.RemoteCodeAddr)
> >> >> > << "\n");
> >> >> > + Client.setProtections(Id, ObjAllocs.RemoteCodeAddr,
> >> >> > + sys::Memory::MF_READ |
> >> >> > sys::Memory::MF_EXEC);
> >> >> > + }
> >> >> > +
> >> >> > + for (auto &Alloc : ObjAllocs.RODataAllocs) {
> >> >> > + DEBUG(dbgs() << " copying ro-data: "
> >> >> > + << static_cast<void
> >> >> > *>(Alloc.getLocalAddress())
> >> >> > << " -> "
> >> >> > + << format("0x%016x",
> >> >> > Alloc.getRemoteAddress())
> >> >> > << " ("
> >> >> > + << Alloc.getSize() << " bytes)\n");
> >> >> > + Client.writeMem(Alloc.getRemoteAddress(),
> >> >> > Alloc.getLocalAddress(),
> >> >> > + Alloc.getSize());
> >> >> > + }
> >> >> > +
> >> >> > + if (ObjAllocs.RemoteRODataAddr) {
> >> >> > + DEBUG(dbgs() << " setting R-- permissions on ro-data
> >> >> > block:
> >> >> > "
> >> >> > + << format("0x%016x",
> >> >> > ObjAllocs.RemoteRODataAddr)
> >> >> > + << "\n");
> >> >> > + Client.setProtections(Id, ObjAllocs.RemoteRODataAddr,
> >> >> > + sys::Memory::MF_READ);
> >> >> > + }
> >> >> > +
> >> >> > + for (auto &Alloc : ObjAllocs.RWDataAllocs) {
> >> >> > + DEBUG(dbgs() << " copying rw-data: "
> >> >> > + << static_cast<void
> >> >> > *>(Alloc.getLocalAddress())
> >> >> > << " -> "
> >> >> > + << format("0x%016x",
> >> >> > Alloc.getRemoteAddress())
> >> >> > << " ("
> >> >> > + << Alloc.getSize() << " bytes)\n");
> >> >> > + Client.writeMem(Alloc.getRemoteAddress(),
> >> >> > Alloc.getLocalAddress(),
> >> >> > + Alloc.getSize());
> >> >> > + }
> >> >> > +
> >> >> > + if (ObjAllocs.RemoteRWDataAddr) {
> >> >> > + DEBUG(dbgs() << " setting RW- permissions on rw-data
> >> >> > block:
> >> >> > "
> >> >> > + << format("0x%016x",
> >> >> > ObjAllocs.RemoteRWDataAddr)
> >> >> > + << "\n");
> >> >> > + Client.setProtections(Id, ObjAllocs.RemoteRWDataAddr,
> >> >> > + sys::Memory::MF_READ |
> >> >> > sys::Memory::MF_WRITE);
> >> >> > + }
> >> >> > + }
> >> >> > + Unfinalized.clear();
> >> >> > +
> >> >> > + return false;
> >> >> > + }
> >> >> > +
> >> >> > + private:
> >> >> > + class Alloc {
> >> >> > + public:
> >> >> > + Alloc(uint64_t Size, unsigned Align)
> >> >> > + : Size(Size), Align(Align), Contents(new char[Size +
> Align
> >> >> > -
> >> >> > 1]),
> >> >> > + RemoteAddr(0) {}
> >> >> > +
> >> >> > + uint64_t getSize() const { return Size; }
> >> >> > +
> >> >> > + unsigned getAlign() const { return Align; }
> >> >> > +
> >> >> > + char *getLocalAddress() const {
> >> >> > + uintptr_t LocalAddr =
> >> >> > reinterpret_cast<uintptr_t>(Contents.get());
> >> >> > + LocalAddr = RoundUpToAlignment(LocalAddr, Align);
> >> >> > + return reinterpret_cast<char *>(LocalAddr);
> >> >> > + }
> >> >> > +
> >> >> > + void setRemoteAddress(TargetAddress RemoteAddr) {
> >> >> > + this->RemoteAddr = RemoteAddr;
> >> >> > + }
> >> >> > +
> >> >> > + TargetAddress getRemoteAddress() const { return RemoteAddr;
> }
> >> >> > +
> >> >> > + private:
> >> >> > + uint64_t Size;
> >> >> > + unsigned Align;
> >> >> > + std::unique_ptr<char[]> Contents;
> >> >> > + TargetAddress RemoteAddr;
> >> >> > + };
> >> >> > +
> >> >> > + struct ObjectAllocs {
> >> >> > + ObjectAllocs()
> >> >> > + : RemoteCodeAddr(0), RemoteRODataAddr(0),
> >> >> > RemoteRWDataAddr(0)
> >> >> > {}
> >> >> > + TargetAddress RemoteCodeAddr;
> >> >> > + TargetAddress RemoteRODataAddr;
> >> >> > + TargetAddress RemoteRWDataAddr;
> >> >> > + std::vector<Alloc> CodeAllocs, RODataAllocs, RWDataAllocs;
> >> >> > + };
> >> >> > +
> >> >> > + OrcRemoteTargetClient &Client;
> >> >> > + ResourceIdMgr::ResourceId Id;
> >> >> > + std::vector<ObjectAllocs> Unmapped;
> >> >> > + std::vector<ObjectAllocs> Unfinalized;
> >> >> > + };
> >> >> > +
> >> >> > + /// Remote indirect stubs manager.
> >> >> > + class RCIndirectStubsManager : public IndirectStubsManager {
> >> >> > + public:
> >> >> > + RCIndirectStubsManager(OrcRemoteTargetClient &Remote,
> >> >> > + ResourceIdMgr::ResourceId Id)
> >> >> > + : Remote(Remote), Id(Id) {}
> >> >> > +
> >> >> > + ~RCIndirectStubsManager() {
> >> >> > Remote.destroyIndirectStubsManager(Id);
> >> >> > }
> >> >> > +
> >> >> > + std::error_code createStub(StringRef StubName, TargetAddress
> >> >> > StubAddr,
> >> >> > + JITSymbolFlags StubFlags) override
> {
> >> >> > + if (auto EC = reserveStubs(1))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return createStubInternal(StubName, StubAddr, StubFlags);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code createStubs(const StubInitsMap &StubInits)
> >> >> > override
> >> >> > {
> >> >> > + if (auto EC = reserveStubs(StubInits.size()))
> >> >> > + return EC;
> >> >> > +
> >> >> > + for (auto &Entry : StubInits)
> >> >> > + if (auto EC = createStubInternal(Entry.first(),
> >> >> > Entry.second.first,
> >> >> > + Entry.second.second))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + JITSymbol findStub(StringRef Name, bool ExportedStubsOnly)
> >> >> > override
> >> >> > {
> >> >> > + auto I = StubIndexes.find(Name);
> >> >> > + if (I == StubIndexes.end())
> >> >> > + return nullptr;
> >> >> > + auto Key = I->second.first;
> >> >> > + auto Flags = I->second.second;
> >> >> > + auto StubSymbol = JITSymbol(getStubAddr(Key), Flags);
> >> >> > + if (ExportedStubsOnly && !StubSymbol.isExported())
> >> >> > + return nullptr;
> >> >> > + return StubSymbol;
> >> >> > + }
> >> >> > +
> >> >> > + JITSymbol findPointer(StringRef Name) override {
> >> >> > + auto I = StubIndexes.find(Name);
> >> >> > + if (I == StubIndexes.end())
> >> >> > + return nullptr;
> >> >> > + auto Key = I->second.first;
> >> >> > + auto Flags = I->second.second;
> >> >> > + return JITSymbol(getPtrAddr(Key), Flags);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code updatePointer(StringRef Name,
> >> >> > + TargetAddress NewAddr) override
> {
> >> >> > + auto I = StubIndexes.find(Name);
> >> >> > + assert(I != StubIndexes.end() && "No stub pointer for
> >> >> > symbol");
> >> >> > + auto Key = I->second.first;
> >> >> > + return Remote.writePointer(getPtrAddr(Key), NewAddr);
> >> >> > + }
> >> >> > +
> >> >> > + private:
> >> >> > + struct RemoteIndirectStubsInfo {
> >> >> > + RemoteIndirectStubsInfo(TargetAddress StubBase,
> TargetAddress
> >> >> > PtrBase,
> >> >> > + unsigned NumStubs)
> >> >> > + : StubBase(StubBase), PtrBase(PtrBase),
> NumStubs(NumStubs)
> >> >> > {}
> >> >> > + TargetAddress StubBase;
> >> >> > + TargetAddress PtrBase;
> >> >> > + unsigned NumStubs;
> >> >> > + };
> >> >> > +
> >> >> > + OrcRemoteTargetClient &Remote;
> >> >> > + ResourceIdMgr::ResourceId Id;
> >> >> > + std::vector<RemoteIndirectStubsInfo> RemoteIndirectStubsInfos;
> >> >> > + typedef std::pair<uint16_t, uint16_t> StubKey;
> >> >> > + std::vector<StubKey> FreeStubs;
> >> >> > + StringMap<std::pair<StubKey, JITSymbolFlags>> StubIndexes;
> >> >> > +
> >> >> > + std::error_code reserveStubs(unsigned NumStubs) {
> >> >> > + if (NumStubs <= FreeStubs.size())
> >> >> > + return std::error_code();
> >> >> > +
> >> >> > + unsigned NewStubsRequired = NumStubs - FreeStubs.size();
> >> >> > + TargetAddress StubBase;
> >> >> > + TargetAddress PtrBase;
> >> >> > + unsigned NumStubsEmitted;
> >> >> > +
> >> >> > + Remote.emitIndirectStubs(StubBase, PtrBase, NumStubsEmitted,
> >> >> > Id,
> >> >> > + NewStubsRequired);
> >> >> > +
> >> >> > + unsigned NewBlockId = RemoteIndirectStubsInfos.size();
> >> >> > + RemoteIndirectStubsInfos.push_back(
> >> >> > + RemoteIndirectStubsInfo(StubBase, PtrBase,
> >> >> > NumStubsEmitted));
> >> >> > +
> >> >> > + for (unsigned I = 0; I < NumStubsEmitted; ++I)
> >> >> > + FreeStubs.push_back(std::make_pair(NewBlockId, I));
> >> >> > +
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code createStubInternal(StringRef StubName,
> >> >> > + TargetAddress InitAddr,
> >> >> > + JITSymbolFlags StubFlags) {
> >> >> > + auto Key = FreeStubs.back();
> >> >> > + FreeStubs.pop_back();
> >> >> > + StubIndexes[StubName] = std::make_pair(Key, StubFlags);
> >> >> > + return Remote.writePointer(getPtrAddr(Key), InitAddr);
> >> >> > + }
> >> >> > +
> >> >> > + TargetAddress getStubAddr(StubKey K) {
> >> >> > + assert(RemoteIndirectStubsInfos[K.first].StubBase != 0 &&
> >> >> > + "Missing stub address");
> >> >> > + return RemoteIndirectStubsInfos[K.first].StubBase +
> >> >> > + K.second * Remote.getIndirectStubSize();
> >> >> > + }
> >> >> > +
> >> >> > + TargetAddress getPtrAddr(StubKey K) {
> >> >> > + assert(RemoteIndirectStubsInfos[K.first].PtrBase != 0 &&
> >> >> > + "Missing pointer address");
> >> >> > + return RemoteIndirectStubsInfos[K.first].PtrBase +
> >> >> > + K.second * Remote.getPointerSize();
> >> >> > + }
> >> >> > + };
> >> >> > +
> >> >> > + /// Remote compile callback manager.
> >> >> > + class RCCompileCallbackManager : public
> JITCompileCallbackManager
> >> >> > {
> >> >> > + public:
> >> >> > + RCCompileCallbackManager(TargetAddress ErrorHandlerAddress,
> >> >> > + OrcRemoteTargetClient &Remote)
> >> >> > + : JITCompileCallbackManager(ErrorHandlerAddress),
> >> >> > Remote(Remote) {
> >> >> > + assert(!Remote.CompileCallback && "Compile callback already
> >> >> > set");
> >> >> > + Remote.CompileCallback = [this](TargetAddress
> TrampolineAddr)
> >> >> > {
> >> >> > + return executeCompileCallback(TrampolineAddr);
> >> >> > + };
> >> >> > + Remote.emitResolverBlock();
> >> >> > + }
> >> >> > +
> >> >> > + private:
> >> >> > + void grow() {
> >> >> > + TargetAddress BlockAddr = 0;
> >> >> > + uint32_t NumTrampolines = 0;
> >> >> > + auto EC = Remote.emitTrampolineBlock(BlockAddr,
> >> >> > NumTrampolines);
> >> >> > + assert(!EC && "Failed to create trampolines");
> >> >> > +
> >> >> > + uint32_t TrampolineSize = Remote.getTrampolineSize();
> >> >> > + for (unsigned I = 0; I < NumTrampolines; ++I)
> >> >> > + this->AvailableTrampolines.push_back(BlockAddr + (I *
> >> >> > TrampolineSize));
> >> >> > + }
> >> >> > +
> >> >> > + OrcRemoteTargetClient &Remote;
> >> >> > + };
> >> >> > +
> >> >> > + /// Create an OrcRemoteTargetClient.
> >> >> > + /// Channel is the ChannelT instance to communicate on. It is
> >> >> > assumed
> >> >> > that
> >> >> > + /// the channel is ready to be read from and written to.
> >> >> > + static ErrorOr<OrcRemoteTargetClient> Create(ChannelT &Channel)
> {
> >> >> > + std::error_code EC;
> >> >> > + OrcRemoteTargetClient H(Channel, EC);
> >> >> > + if (EC)
> >> >> > + return EC;
> >> >> > + return H;
> >> >> > + }
> >> >> > +
> >> >> > + /// Call the int(void) function at the given address in the
> target
> >> >> > and return
> >> >> > + /// its result.
> >> >> > + std::error_code callIntVoid(int &Result, TargetAddress Addr) {
> >> >> > + DEBUG(dbgs() << "Calling int(*)(void) " << format("0x%016x",
> >> >> > Addr)
> >> >> > << "\n");
> >> >> > +
> >> >> > + if (auto EC = call<CallIntVoid>(Channel, Addr))
> >> >> > + return EC;
> >> >> > +
> >> >> > + unsigned NextProcId;
> >> >> > + if (auto EC = listenForCompileRequests(NextProcId))
> >> >> > + return EC;
> >> >> > +
> >> >> > + if (NextProcId != CallIntVoidResponseId)
> >> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
> >> >> > +
> >> >> > + return handle<CallIntVoidResponse>(Channel, [&](int R) {
> >> >> > + Result = R;
> >> >> > + DEBUG(dbgs() << "Result: " << R << "\n");
> >> >> > + return std::error_code();
> >> >> > + });
> >> >> > + }
> >> >> > +
> >> >> > + /// Call the int(int, char*[]) function at the given address in
> >> >> > the
> >> >> > target and
> >> >> > + /// return its result.
> >> >> > + std::error_code callMain(int &Result, TargetAddress Addr,
> >> >> > + const std::vector<std::string> &Args) {
> >> >> > + DEBUG(dbgs() << "Calling int(*)(int, char*[]) " <<
> >> >> > format("0x%016x", Addr)
> >> >> > + << "\n");
> >> >> > +
> >> >> > + if (auto EC = call<CallMain>(Channel, Addr, Args))
> >> >> > + return EC;
> >> >> > +
> >> >> > + unsigned NextProcId;
> >> >> > + if (auto EC = listenForCompileRequests(NextProcId))
> >> >> > + return EC;
> >> >> > +
> >> >> > + if (NextProcId != CallMainResponseId)
> >> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
> >> >> > +
> >> >> > + return handle<CallMainResponse>(Channel, [&](int R) {
> >> >> > + Result = R;
> >> >> > + DEBUG(dbgs() << "Result: " << R << "\n");
> >> >> > + return std::error_code();
> >> >> > + });
> >> >> > + }
> >> >> > +
> >> >> > + /// Call the void() function at the given address in the target
> >> >> > and
> >> >> > wait for
> >> >> > + /// it to finish.
> >> >> > + std::error_code callVoidVoid(TargetAddress Addr) {
> >> >> > + DEBUG(dbgs() << "Calling void(*)(void) " << format("0x%016x",
> >> >> > Addr)
> >> >> > + << "\n");
> >> >> > +
> >> >> > + if (auto EC = call<CallVoidVoid>(Channel, Addr))
> >> >> > + return EC;
> >> >> > +
> >> >> > + unsigned NextProcId;
> >> >> > + if (auto EC = listenForCompileRequests(NextProcId))
> >> >> > + return EC;
> >> >> > +
> >> >> > + if (NextProcId != CallVoidVoidResponseId)
> >> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
> >> >> > +
> >> >> > + return handle<CallVoidVoidResponse>(Channel, doNothing);
> >> >> > + }
> >> >> > +
> >> >> > + /// Create an RCMemoryManager which will allocate its memory on
> >> >> > the
> >> >> > remote
> >> >> > + /// target.
> >> >> > + std::error_code
> >> >> > + createRemoteMemoryManager(std::unique_ptr<RCMemoryManager> &MM)
> {
> >> >> > + assert(!MM && "MemoryManager should be null before
> creation.");
> >> >> > +
> >> >> > + auto Id = AllocatorIds.getNext();
> >> >> > + if (auto EC = call<CreateRemoteAllocator>(Channel, Id))
> >> >> > + return EC;
> >> >> > + MM = llvm::make_unique<RCMemoryManager>(*this, Id);
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + /// Create an RCIndirectStubsManager that will allocate stubs on
> >> >> > the
> >> >> > remote
> >> >> > + /// target.
> >> >> > + std::error_code
> >> >> > +
> createIndirectStubsManager(std::unique_ptr<RCIndirectStubsManager>
> >> >> > &I) {
> >> >> > + assert(!I && "Indirect stubs manager should be null before
> >> >> > creation.");
> >> >> > + auto Id = IndirectStubOwnerIds.getNext();
> >> >> > + if (auto EC = call<CreateIndirectStubsOwner>(Channel, Id))
> >> >> > + return EC;
> >> >> > + I = llvm::make_unique<RCIndirectStubsManager>(*this, Id);
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + /// Search for symbols in the remote process. Note: This should
> be
> >> >> > used by
> >> >> > + /// symbol resolvers *after* they've searched the local symbol
> >> >> > table
> >> >> > in the
> >> >> > + /// JIT stack.
> >> >> > + std::error_code getSymbolAddress(TargetAddress &Addr, StringRef
> >> >> > Name)
> >> >> > {
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + // Request remote symbol address.
> >> >> > + if (auto EC = call<GetSymbolAddress>(Channel, Name))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return expect<GetSymbolAddressResponse>(Channel,
> >> >> > [&](TargetAddress
> >> >> > &A) {
> >> >> > + Addr = A;
> >> >> > + DEBUG(dbgs() << "Remote address lookup " << Name << " = "
> >> >> > + << format("0x%016x", Addr) << "\n");
> >> >> > + return std::error_code();
> >> >> > + });
> >> >> > + }
> >> >> > +
> >> >> > + /// Get the triple for the remote target.
> >> >> > + const std::string &getTargetTriple() const { return
> >> >> > RemoteTargetTriple; }
> >> >> > +
> >> >> > + std::error_code terminateSession() { return
> >> >> > call<TerminateSession>(Channel); }
> >> >> > +
> >> >> > +private:
> >> >> > + OrcRemoteTargetClient(ChannelT &Channel, std::error_code &EC)
> >> >> > + : Channel(Channel), RemotePointerSize(0), RemotePageSize(0),
> >> >> > + RemoteTrampolineSize(0), RemoteIndirectStubSize(0) {
> >> >> > + if ((EC = call<GetRemoteInfo>(Channel)))
> >> >> > + return;
> >> >> > +
> >> >> > + EC = expect<GetRemoteInfoResponse>(
> >> >> > + Channel, readArgs(RemoteTargetTriple, RemotePointerSize,
> >> >> > RemotePageSize,
> >> >> > + RemoteTrampolineSize,
> >> >> > RemoteIndirectStubSize));
> >> >> > + }
> >> >> > +
> >> >> > + void destroyRemoteAllocator(ResourceIdMgr::ResourceId Id) {
> >> >> > + if (auto EC = call<DestroyRemoteAllocator>(Channel, Id)) {
> >> >> > + // FIXME: This will be triggered by a removeModuleSet call:
> >> >> > Propagate
> >> >> > + // error return up through that.
> >> >> > + llvm_unreachable("Failed to destroy remote allocator.");
> >> >> > + AllocatorIds.release(Id);
> >> >> > + }
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code
> >> >> > destroyIndirectStubsManager(ResourceIdMgr::ResourceId
> >> >> > Id) {
> >> >> > + IndirectStubOwnerIds.release(Id);
> >> >> > + return call<DestroyIndirectStubsOwner>(Channel, Id);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code emitIndirectStubs(TargetAddress &StubBase,
> >> >> > + TargetAddress &PtrBase,
> >> >> > + uint32_t &NumStubsEmitted,
> >> >> > + ResourceIdMgr::ResourceId Id,
> >> >> > + uint32_t NumStubsRequired) {
> >> >> > + if (auto EC = call<EmitIndirectStubs>(Channel, Id,
> >> >> > NumStubsRequired))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return expect<EmitIndirectStubsResponse>(
> >> >> > + Channel, readArgs(StubBase, PtrBase, NumStubsEmitted));
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code emitResolverBlock() {
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + return call<EmitResolverBlock>(Channel);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code emitTrampolineBlock(TargetAddress &BlockAddr,
> >> >> > + uint32_t &NumTrampolines) {
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + if (auto EC = call<EmitTrampolineBlock>(Channel))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return expect<EmitTrampolineBlockResponse>(
> >> >> > + Channel, [&](TargetAddress BAddr, uint32_t NTrampolines) {
> >> >> > + BlockAddr = BAddr;
> >> >> > + NumTrampolines = NTrampolines;
> >> >> > + return std::error_code();
> >> >> > + });
> >> >> > + }
> >> >> > +
> >> >> > + uint32_t getIndirectStubSize() const { return
> >> >> > RemoteIndirectStubSize;
> >> >> > }
> >> >> > + uint32_t getPageSize() const { return RemotePageSize; }
> >> >> > + uint32_t getPointerSize() const { return RemotePointerSize; }
> >> >> > +
> >> >> > + uint32_t getTrampolineSize() const { return
> RemoteTrampolineSize;
> >> >> > }
> >> >> > +
> >> >> > + std::error_code listenForCompileRequests(uint32_t &NextId) {
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + if (auto EC = getNextProcId(Channel, NextId))
> >> >> > + return EC;
> >> >> > +
> >> >> > + while (NextId == RequestCompileId) {
> >> >> > + TargetAddress TrampolineAddr = 0;
> >> >> > + if (auto EC = handle<RequestCompile>(Channel,
> >> >> > readArgs(TrampolineAddr)))
> >> >> > + return EC;
> >> >> > +
> >> >> > + TargetAddress ImplAddr = CompileCallback(TrampolineAddr);
> >> >> > + if (auto EC = call<RequestCompileResponse>(Channel,
> ImplAddr))
> >> >> > + return EC;
> >> >> > +
> >> >> > + if (auto EC = getNextProcId(Channel, NextId))
> >> >> > + return EC;
> >> >> > + }
> >> >> > +
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code readMem(char *Dst, TargetAddress Src, uint64_t
> >> >> > Size)
> >> >> > {
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + if (auto EC = call<ReadMem>(Channel, Src, Size))
> >> >> > + return EC;
> >> >> > +
> >> >> > + if (auto EC = expect<ReadMemResponse>(
> >> >> > + Channel, [&]() { return Channel.readBytes(Dst, Size);
> >> >> > }))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code reserveMem(TargetAddress &RemoteAddr,
> >> >> > + ResourceIdMgr::ResourceId Id,
> uint64_t
> >> >> > Size,
> >> >> > + uint32_t Align) {
> >> >> > +
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + if (auto EC = call<ReserveMem>(Channel, Id, Size, Align))
> >> >> > + return EC;
> >> >> > +
> >> >> > + if (auto EC = expect<ReserveMemResponse>(Channel,
> >> >> > [&](TargetAddress
> >> >> > Addr) {
> >> >> > + RemoteAddr = Addr;
> >> >> > + return std::error_code();
> >> >> > + }))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code setProtections(ResourceIdMgr::ResourceId Id,
> >> >> > + TargetAddress RemoteSegAddr,
> >> >> > + unsigned ProtFlags) {
> >> >> > + return call<SetProtections>(Channel, Id, RemoteSegAddr,
> >> >> > ProtFlags);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code writeMem(TargetAddress Addr, const char *Src,
> >> >> > uint64_t Size) {
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + // Make the send call.
> >> >> > + if (auto EC = call<WriteMem>(Channel, Addr, Size))
> >> >> > + return EC;
> >> >> > +
> >> >> > + // Follow this up with the section contents.
> >> >> > + if (auto EC = Channel.appendBytes(Src, Size))
> >> >> > + return EC;
> >> >> > +
> >> >> > + return Channel.send();
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code writePointer(TargetAddress Addr, TargetAddress
> >> >> > PtrVal) {
> >> >> > + // Check for an 'out-of-band' error, e.g. from an MM
> destructor.
> >> >> > + if (ExistingError)
> >> >> > + return ExistingError;
> >> >> > +
> >> >> > + return call<WritePtr>(Channel, Addr, PtrVal);
> >> >> > + }
> >> >> > +
> >> >> > + static std::error_code doNothing() { return std::error_code(); }
> >> >> > +
> >> >> > + ChannelT &Channel;
> >> >> > + std::error_code ExistingError;
> >> >> > + std::string RemoteTargetTriple;
> >> >> > + uint32_t RemotePointerSize;
> >> >> > + uint32_t RemotePageSize;
> >> >> > + uint32_t RemoteTrampolineSize;
> >> >> > + uint32_t RemoteIndirectStubSize;
> >> >> > + ResourceIdMgr AllocatorIds, IndirectStubOwnerIds;
> >> >> > + std::function<TargetAddress(TargetAddress)> CompileCallback;
> >> >> > +};
> >> >> > +
> >> >> > +} // end namespace remote
> >> >> > +} // end namespace orc
> >> >> > +} // end namespace llvm
> >> >> > +
> >> >> > +#undef DEBUG_TYPE
> >> >> > +
> >> >> > +#endif
> >> >> >
> >> >> > Added:
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
> >> >> > URL:
> >> >> >
> >> >> >
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h?rev=257305&view=auto
> >> >> >
> >> >> >
> >> >> >
> ==============================================================================
> >> >> > ---
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
> >> >> > (added)
> >> >> > +++
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetRPCAPI.h
> >> >> > Sun Jan 10 19:40:11 2016
> >> >> > @@ -0,0 +1,185 @@
> >> >> > +//===--- OrcRemoteTargetRPCAPI.h - Orc Remote-target RPC API
> ----*-
> >> >> > C++
> >> >> > -*-===//
> >> >> > +//
> >> >> > +// The LLVM Compiler Infrastructure
> >> >> > +//
> >> >> > +// This file is distributed under the University of Illinois Open
> >> >> > Source
> >> >> > +// License. See LICENSE.TXT for details.
> >> >> > +//
> >> >> >
> >> >> >
> >> >> >
> +//===----------------------------------------------------------------------===//
> >> >> > +//
> >> >> > +// This file defines the Orc remote-target RPC API. It should not
> be
> >> >> > used
> >> >> > +// directly, but is used by the RemoteTargetClient and
> >> >> > RemoteTargetServer
> >> >> > +// classes.
> >> >> > +//
> >> >> >
> >> >> >
> >> >> >
> +//===----------------------------------------------------------------------===//
> >> >> > +
> >> >> > +#ifndef LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETRPCAPI_H
> >> >> > +#define LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETRPCAPI_H
> >> >> > +
> >> >> > +#include "JITSymbol.h"
> >> >> > +#include "RPCChannel.h"
> >> >> > +#include "RPCUtils.h"
> >> >> > +
> >> >> > +namespace llvm {
> >> >> > +namespace orc {
> >> >> > +namespace remote {
> >> >> > +
> >> >> > +class OrcRemoteTargetRPCAPI : public RPC<RPCChannel> {
> >> >> > +protected:
> >> >> > + class ResourceIdMgr {
> >> >> > + public:
> >> >> > + typedef uint64_t ResourceId;
> >> >> > + ResourceIdMgr() : NextId(0) {}
> >> >> > + ResourceId getNext() {
> >> >> > + if (!FreeIds.empty()) {
> >> >> > + ResourceId I = FreeIds.back();
> >> >> > + FreeIds.pop_back();
> >> >> > + return I;
> >> >> > + }
> >> >> > + return NextId++;
> >> >> > + }
> >> >> > + void release(ResourceId I) { FreeIds.push_back(I); }
> >> >> > +
> >> >> > + private:
> >> >> > + ResourceId NextId;
> >> >> > + std::vector<ResourceId> FreeIds;
> >> >> > + };
> >> >> > +
> >> >> > +public:
> >> >> > + enum JITProcId : uint32_t {
> >> >> > + InvalidId = 0,
> >> >> > + CallIntVoidId,
> >> >> > + CallIntVoidResponseId,
> >> >> > + CallMainId,
> >> >> > + CallMainResponseId,
> >> >> > + CallVoidVoidId,
> >> >> > + CallVoidVoidResponseId,
> >> >> > + CreateRemoteAllocatorId,
> >> >> > + CreateIndirectStubsOwnerId,
> >> >> > + DestroyRemoteAllocatorId,
> >> >> > + DestroyIndirectStubsOwnerId,
> >> >> > + EmitIndirectStubsId,
> >> >> > + EmitIndirectStubsResponseId,
> >> >> > + EmitResolverBlockId,
> >> >> > + EmitTrampolineBlockId,
> >> >> > + EmitTrampolineBlockResponseId,
> >> >> > + GetSymbolAddressId,
> >> >> > + GetSymbolAddressResponseId,
> >> >> > + GetRemoteInfoId,
> >> >> > + GetRemoteInfoResponseId,
> >> >> > + ReadMemId,
> >> >> > + ReadMemResponseId,
> >> >> > + ReserveMemId,
> >> >> > + ReserveMemResponseId,
> >> >> > + RequestCompileId,
> >> >> > + RequestCompileResponseId,
> >> >> > + SetProtectionsId,
> >> >> > + TerminateSessionId,
> >> >> > + WriteMemId,
> >> >> > + WritePtrId
> >> >> > + };
> >> >> > +
> >> >> > + static const char *getJITProcIdName(JITProcId Id);
> >> >> > +
> >> >> > + typedef Procedure<CallIntVoidId, TargetAddress /* FnAddr */>
> >> >> > CallIntVoid;
> >> >> > +
> >> >> > + typedef Procedure<CallIntVoidResponseId, int /* Result */>
> >> >> > + CallIntVoidResponse;
> >> >> > +
> >> >> > + typedef Procedure<CallMainId, TargetAddress /* FnAddr */,
> >> >> > + std::vector<std::string> /* Args */>
> >> >> > + CallMain;
> >> >> > +
> >> >> > + typedef Procedure<CallMainResponseId, int /* Result */>
> >> >> > CallMainResponse;
> >> >> > +
> >> >> > + typedef Procedure<CallVoidVoidId, TargetAddress /* FnAddr */>
> >> >> > CallVoidVoid;
> >> >> > +
> >> >> > + typedef Procedure<CallVoidVoidResponseId> CallVoidVoidResponse;
> >> >> > +
> >> >> > + typedef Procedure<CreateRemoteAllocatorId,
> >> >> > + ResourceIdMgr::ResourceId /* Allocator ID */>
> >> >> > + CreateRemoteAllocator;
> >> >> > +
> >> >> > + typedef Procedure<CreateIndirectStubsOwnerId,
> >> >> > + ResourceIdMgr::ResourceId /* StubsOwner ID */>
> >> >> > + CreateIndirectStubsOwner;
> >> >> > +
> >> >> > + typedef Procedure<DestroyRemoteAllocatorId,
> >> >> > + ResourceIdMgr::ResourceId /* Allocator ID */>
> >> >> > + DestroyRemoteAllocator;
> >> >> > +
> >> >> > + typedef Procedure<DestroyIndirectStubsOwnerId,
> >> >> > + ResourceIdMgr::ResourceId /* StubsOwner ID */>
> >> >> > + DestroyIndirectStubsOwner;
> >> >> > +
> >> >> > + typedef Procedure<EmitIndirectStubsId,
> >> >> > + ResourceIdMgr::ResourceId /* StubsOwner ID */,
> >> >> > + uint32_t /* NumStubsRequired */>
> >> >> > + EmitIndirectStubs;
> >> >> > +
> >> >> > + typedef Procedure<
> >> >> > + EmitIndirectStubsResponseId, TargetAddress /* StubsBaseAddr
> >> >> > */,
> >> >> > + TargetAddress /* PtrsBaseAddr */, uint32_t /*
> NumStubsEmitted
> >> >> > */>
> >> >> > + EmitIndirectStubsResponse;
> >> >> > +
> >> >> > + typedef Procedure<EmitResolverBlockId> EmitResolverBlock;
> >> >> > +
> >> >> > + typedef Procedure<EmitTrampolineBlockId> EmitTrampolineBlock;
> >> >> > +
> >> >> > + typedef Procedure<EmitTrampolineBlockResponseId,
> >> >> > + TargetAddress /* BlockAddr */,
> >> >> > + uint32_t /* NumTrampolines */>
> >> >> > + EmitTrampolineBlockResponse;
> >> >> > +
> >> >> > + typedef Procedure<GetSymbolAddressId, std::string
> /*SymbolName*/>
> >> >> > + GetSymbolAddress;
> >> >> > +
> >> >> > + typedef Procedure<GetSymbolAddressResponseId, uint64_t /*
> >> >> > SymbolAddr
> >> >> > */>
> >> >> > + GetSymbolAddressResponse;
> >> >> > +
> >> >> > + typedef Procedure<GetRemoteInfoId> GetRemoteInfo;
> >> >> > +
> >> >> > + typedef Procedure<GetRemoteInfoResponseId, std::string /* Triple
> >> >> > */,
> >> >> > + uint32_t /* PointerSize */, uint32_t /*
> PageSize
> >> >> > */,
> >> >> > + uint32_t /* TrampolineSize */,
> >> >> > + uint32_t /* IndirectStubSize */>
> >> >> > + GetRemoteInfoResponse;
> >> >> > +
> >> >> > + typedef Procedure<ReadMemId, TargetAddress /* Src */, uint64_t
> /*
> >> >> > Size */>
> >> >> > + ReadMem;
> >> >> > +
> >> >> > + typedef Procedure<ReadMemResponseId> ReadMemResponse;
> >> >> > +
> >> >> > + typedef Procedure<ReserveMemId, ResourceIdMgr::ResourceId /* Id
> >> >> > */,
> >> >> > + uint64_t /* Size */, uint32_t /* Align */>
> >> >> > + ReserveMem;
> >> >> > +
> >> >> > + typedef Procedure<ReserveMemResponseId, TargetAddress /* Addr
> */>
> >> >> > + ReserveMemResponse;
> >> >> > +
> >> >> > + typedef Procedure<RequestCompileId, TargetAddress /*
> >> >> > TrampolineAddr
> >> >> > */>
> >> >> > + RequestCompile;
> >> >> > +
> >> >> > + typedef Procedure<RequestCompileResponseId, TargetAddress /*
> >> >> > ImplAddr
> >> >> > */>
> >> >> > + RequestCompileResponse;
> >> >> > +
> >> >> > + typedef Procedure<SetProtectionsId, ResourceIdMgr::ResourceId /*
> >> >> > Id
> >> >> > */,
> >> >> > + TargetAddress /* Dst */, uint32_t /* ProtFlags
> >> >> > */>
> >> >> > + SetProtections;
> >> >> > +
> >> >> > + typedef Procedure<TerminateSessionId> TerminateSession;
> >> >> > +
> >> >> > + typedef Procedure<WriteMemId, TargetAddress /* Dst */, uint64_t
> /*
> >> >> > Size */
> >> >> > + /* Data should follow */>
> >> >> > + WriteMem;
> >> >> > +
> >> >> > + typedef Procedure<WritePtrId, TargetAddress /* Dst */,
> >> >> > + TargetAddress /* Val */>
> >> >> > + WritePtr;
> >> >> > +};
> >> >> > +
> >> >> > +} // end namespace remote
> >> >> > +} // end namespace orc
> >> >> > +} // end namespace llvm
> >> >> > +
> >> >> > +#endif
> >> >> >
> >> >> > Added:
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
> >> >> > URL:
> >> >> >
> >> >> >
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h?rev=257305&view=auto
> >> >> >
> >> >> >
> >> >> >
> ==============================================================================
> >> >> > ---
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
> >> >> > (added)
> >> >> > +++
> >> >> > llvm/trunk/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h
> >> >> > Sun Jan 10 19:40:11 2016
> >> >> > @@ -0,0 +1,479 @@
> >> >> > +//===---- OrcRemoteTargetServer.h - Orc Remote-target Server
> ----*-
> >> >> > C++
> >> >> > -*-===//
> >> >> > +//
> >> >> > +// The LLVM Compiler Infrastructure
> >> >> > +//
> >> >> > +// This file is distributed under the University of Illinois Open
> >> >> > Source
> >> >> > +// License. See LICENSE.TXT for details.
> >> >> > +//
> >> >> >
> >> >> >
> >> >> >
> +//===----------------------------------------------------------------------===//
> >> >> > +//
> >> >> > +// This file defines the OrcRemoteTargetServer class. It can be
> used
> >> >> > to
> >> >> > build a
> >> >> > +// JIT server that can execute code sent from an
> >> >> > OrcRemoteTargetClient.
> >> >> > +//
> >> >> >
> >> >> >
> >> >> >
> +//===----------------------------------------------------------------------===//
> >> >> > +
> >> >> > +#ifndef LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETSERVER_H
> >> >> > +#define LLVM_EXECUTIONENGINE_ORC_ORCREMOTETARGETSERVER_H
> >> >> > +
> >> >> > +#include "OrcRemoteTargetRPCAPI.h"
> >> >> > +#include "llvm/ExecutionEngine/RTDyldMemoryManager.h"
> >> >> > +#include "llvm/Support/Debug.h"
> >> >> > +#include "llvm/Support/Format.h"
> >> >> > +#include "llvm/Support/Process.h"
> >> >> > +#include "llvm/Support/raw_ostream.h"
> >> >> > +#include <map>
> >> >> > +
> >> >> > +#define DEBUG_TYPE "orc-remote"
> >> >> > +
> >> >> > +namespace llvm {
> >> >> > +namespace orc {
> >> >> > +namespace remote {
> >> >> > +
> >> >> > +template <typename ChannelT, typename TargetT>
> >> >> > +class OrcRemoteTargetServer : public OrcRemoteTargetRPCAPI {
> >> >> > +public:
> >> >> > + typedef std::function<TargetAddress(const std::string &Name)>
> >> >> > + SymbolLookupFtor;
> >> >> > +
> >> >> > + OrcRemoteTargetServer(ChannelT &Channel, SymbolLookupFtor
> >> >> > SymbolLookup)
> >> >> > + : Channel(Channel), SymbolLookup(std::move(SymbolLookup)) {}
> >> >> > +
> >> >> > + std::error_code getNextProcId(JITProcId &Id) {
> >> >> > + return deserialize(Channel, Id);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code handleKnownProcedure(JITProcId Id) {
> >> >> > + DEBUG(dbgs() << "Handling known proc: " <<
> getJITProcIdName(Id)
> >> >> > <<
> >> >> > "\n");
> >> >> > +
> >> >> > + switch (Id) {
> >> >> > + case CallIntVoidId:
> >> >> > + return handleCallIntVoid();
> >> >> > + case CallMainId:
> >> >> > + return handleCallMain();
> >> >> > + case CallVoidVoidId:
> >> >> > + return handleCallVoidVoid();
> >> >> > + case CreateRemoteAllocatorId:
> >> >> > + return handleCreateRemoteAllocator();
> >> >> > + case CreateIndirectStubsOwnerId:
> >> >> > + return handleCreateIndirectStubsOwner();
> >> >> > + case DestroyRemoteAllocatorId:
> >> >> > + return handleDestroyRemoteAllocator();
> >> >> > + case EmitIndirectStubsId:
> >> >> > + return handleEmitIndirectStubs();
> >> >> > + case EmitResolverBlockId:
> >> >> > + return handleEmitResolverBlock();
> >> >> > + case EmitTrampolineBlockId:
> >> >> > + return handleEmitTrampolineBlock();
> >> >> > + case GetSymbolAddressId:
> >> >> > + return handleGetSymbolAddress();
> >> >> > + case GetRemoteInfoId:
> >> >> > + return handleGetRemoteInfo();
> >> >> > + case ReadMemId:
> >> >> > + return handleReadMem();
> >> >> > + case ReserveMemId:
> >> >> > + return handleReserveMem();
> >> >> > + case SetProtectionsId:
> >> >> > + return handleSetProtections();
> >> >> > + case WriteMemId:
> >> >> > + return handleWriteMem();
> >> >> > + case WritePtrId:
> >> >> > + return handleWritePtr();
> >> >> > + default:
> >> >> > + return orcError(OrcErrorCode::UnexpectedRPCCall);
> >> >> > + }
> >> >> > +
> >> >> > + llvm_unreachable("Unhandled JIT RPC procedure Id.");
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code requestCompile(TargetAddress &CompiledFnAddr,
> >> >> > + TargetAddress TrampolineAddr) {
> >> >> > + if (auto EC = call<RequestCompile>(Channel, TrampolineAddr))
> >> >> > + return EC;
> >> >> > +
> >> >> > + while (1) {
> >> >> > + JITProcId Id = InvalidId;
> >> >> > + if (auto EC = getNextProcId(Id))
> >> >> > + return EC;
> >> >> > +
> >> >> > + switch (Id) {
> >> >> > + case RequestCompileResponseId:
> >> >> > + return handle<RequestCompileResponse>(Channel,
> >> >> > +
> >> >> > readArgs(CompiledFnAddr));
> >> >> > + default:
> >> >> > + if (auto EC = handleKnownProcedure(Id))
> >> >> > + return EC;
> >> >> > + }
> >> >> > + }
> >> >> > +
> >> >> > + llvm_unreachable("Fell through request-compile command
> loop.");
> >> >> > + }
> >> >> > +
> >> >> > +private:
> >> >> > + struct Allocator {
> >> >> > + Allocator() = default;
> >> >> > + Allocator(Allocator &&) = default;
> >> >> > + Allocator &operator=(Allocator &&) = default;
> >> >> > +
> >> >> > + ~Allocator() {
> >> >> > + for (auto &Alloc : Allocs)
> >> >> > + sys::Memory::releaseMappedMemory(Alloc.second);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code allocate(void *&Addr, size_t Size, uint32_t
> >> >> > Align)
> >> >> > {
> >> >> > + std::error_code EC;
> >> >> > + sys::MemoryBlock MB = sys::Memory::allocateMappedMemory(
> >> >> > + Size, nullptr, sys::Memory::MF_READ |
> >> >> > sys::Memory::MF_WRITE,
> >> >> > EC);
> >> >> > + if (EC)
> >> >> > + return EC;
> >> >> > +
> >> >> > + Addr = MB.base();
> >> >> > + assert(Allocs.find(MB.base()) == Allocs.end() && "Duplicate
> >> >> > alloc");
> >> >> > + Allocs[MB.base()] = std::move(MB);
> >> >> > + return std::error_code();
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code setProtections(void *block, unsigned Flags) {
> >> >> > + auto I = Allocs.find(block);
> >> >> > + if (I == Allocs.end())
> >> >> > + return
> >> >> > orcError(OrcErrorCode::RemoteMProtectAddrUnrecognized);
> >> >> > + return sys::Memory::protectMappedMemory(I->second, Flags);
> >> >> > + }
> >> >> > +
> >> >> > + private:
> >> >> > + std::map<void *, sys::MemoryBlock> Allocs;
> >> >> > + };
> >> >> > +
> >> >> > + static std::error_code doNothing() { return std::error_code(); }
> >> >> > +
> >> >> > + static TargetAddress reenter(void *JITTargetAddr, void
> >> >> > *TrampolineAddr) {
> >> >> > + TargetAddress CompiledFnAddr = 0;
> >> >> > +
> >> >> > + auto T = static_cast<OrcRemoteTargetServer *>(JITTargetAddr);
> >> >> > + auto EC = T->requestCompile(
> >> >> > + CompiledFnAddr, static_cast<TargetAddress>(
> >> >> > +
> >> >> > reinterpret_cast<uintptr_t>(TrampolineAddr)));
> >> >> > + assert(!EC && "Compile request failed");
> >> >> > + return CompiledFnAddr;
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code handleCallIntVoid() {
> >> >> > + typedef int (*IntVoidFnTy)();
> >> >> > +
> >> >> > + IntVoidFnTy Fn = nullptr;
> >> >> > + if (auto EC = handle<CallIntVoid>(Channel, [&](TargetAddress
> >> >> > Addr)
> >> >> > {
> >> >> > + Fn =
> >> >> > reinterpret_cast<IntVoidFnTy>(static_cast<uintptr_t>(Addr));
> >> >> > + return std::error_code();
> >> >> > + }))
> >> >> > + return EC;
> >> >> > +
> >> >> > + DEBUG(dbgs() << " Calling " << reinterpret_cast<void *>(Fn)
> <<
> >> >> > "\n");
> >> >> > + int Result = Fn();
> >> >> > + DEBUG(dbgs() << " Result = " << Result << "\n");
> >> >> > +
> >> >> > + return call<CallIntVoidResponse>(Channel, Result);
> >> >> > + }
> >> >> > +
> >> >> > + std::error_code handleCallMain() {
> >> >> > + typedef int (*MainFnTy)(int, const char *[]);
> >> >> > +
> >> >> > + MainFnTy Fn = nullptr;
> >> >> > + std::vector<std::string> Args;
> >> >> > + if (auto EC = handle<CallMain>(
> >> >> > + Channel, [&](TargetAddress Addr,
> >> >> > std::vector<std::string>
> >> >> > &A) {
> >> >> > + Fn =
> >> >> > reinterpret_cast<MainFnTy>(static_cast<uintptr_t>(Addr));
> >> >> > + Args = std::move(A);
> >> >> > + return std::error_code();
> >> >> > + }))
> >> >> > + return EC;
> >> >> > +
> >> >> > + int ArgC = Args.size() + 1;
> >> >> > + int Idx = 1;
> >> >> > + std::unique_ptr<const char *[]> ArgV(new const char *[ArgC +
> >> >> > 1]);
> >> >> > + ArgV[0] = "<jit process>";
> >> >> > + for (auto &Arg : Args)
> >> >> > + ArgV[Idx++] = Arg.c_str();
> >> >> > +
> >> >> > + DEBUG(dbgs() << " Calling " << reinterpret_cast<void *>(Fn)
> <<
> >> >> > "\n");
> >> >>
> >> >> This is causing a warning:
> >> >>
> >> >>
> >> >>
> >> >>
> /opt/llvm/build-llvm/src/llvm/include/llvm/ExecutionEngine/Orc/OrcRemoteTargetServer.h:215:5:
> >> >> warning: ISO C++ forbids casting between pointer-to-function and
> >> >> pointer-to-object [enabled by default]
> >> >
> >> >
> >> > I'm not seeing this on a clang self-host. Which compiler is warning
> >> > about
> >> > this? (should we enable this warning when building with clang, or
> >> > disable it
> >> > when building with whatever is firing it?)
> >>
> >> I'm not certain of the compiler version information. This is on our
> >> attribute documentation bot, which I *think* is GCC. Tanya would know
> >> more if we need exact details.
> >>
> >> I think it's a reasonable diagnostic to keep for portability reasons.
> >> We usually do pretty good about keeping this one clean, though it does
> >> crop up from time to time.
> >
> >
> > Hmm, interesting - Clang does have this warning and we do build with
> > -pedantic by default, but Clang does /not/ fire this warning when
> building
> > in C++11 mode...
> >
> > The implementation in Clang (SemaCast.cpp:2022) mentions that this
> behavior
> > is "conditionally supported" - which is, implementations may allow it
> and if
> > they allow the conversion in both directions then it must yield the
> original
> > pointer value (it must roundtrip).
> >
> > The comment also mentions that compilers generally support it even in
> C++03
> > mode, because you need it to use dlsym and GetProcAddress. So it seems we
> > probably require this of any implementation we use anyway.
>
> That is true -- we do require that behavior to be supported anyway,
> and if it's not supported we're likely to find out very quickly. So
> perhaps we can simply disable that diagnostic for the various
> compilers.
>
Thing is it's a bit of a broad flag (-Wpedantic covers many things) and
probably still want the Clang version, just not the GCC version. I suppose
we probably have somewhere that handles GCC-specific warning
configuration...
>
> ~Aaron
>
> >
> >>
> >>
> >> > If it's worth keeping, the easy/reasonable fix is to cast the function
> >> > pointer to intptr_t first, then to void* to print. (or just to
> intptr_t
> >> > then
> >> > hex format with llvm::format)
> >>
> >> Yes, I just wasn't certain whether Lang wanted to keep the DEBUG()
> >> statements or not, since another easy fix is to remove the debug
> >> logging if it's not required.
> >>
> >> ~Aaron
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160115/9785094c/attachment-0001.html>
More information about the llvm-commits
mailing list