[LLVMdev] ORC and relocations

[Lang Hames]

Hi Eugene,

Returning null addresses to the linker is unusual enough that I'd prefer to
keep it as an error.

Could you return a non-zero marker value, e.g. all-ones? If not, we should
add an option to RuntimeDyld like "AllowUnresolvedSymbols" and change the
error condition to if '(!Addr && !AllowUnresolvedSymbols)'.

Cheers,
Lang.


On Thu, Jun 25, 2015 at 6:35 PM, Eugene Rozenfeld <
Eugene.Rozenfeld at microsoft.com> wrote:

>  Hi Lang,
>
>
>
> Can you please let me know you think it would be right to modify
> RuntimeDyldImpl::resolveExternalSymbols to allow resolvers to return 0
> addresses? Something like this would be ideal for me:
>
>
>
> void RuntimeDyldImpl::resolveExternalSymbols() {
>
>   while (!ExternalSymbolRelocations.empty()) {
>
>     StringMap<RelocationList>::iterator
>  i = ExternalSymbolRelocations.begin();
>
>
>
>     StringRef Name = i->first();
>
>     if (Name.size() == 0) {
>
>       // This is an absolute symbol, use an address of zero.
>
>       DEBUG(dbgs() << "Resolving absolute relocations."
>
>                    << "\n");
>
>       RelocationList &Relocs = i->second;
>
>       resolveRelocationList(Relocs, 0);
>
>     } else {
>
>       uint64_t Addr = 0;
>
>       RTDyldSymbolTable::const_iterator
>  Loc = GlobalSymbolTable.find(Name);
>
>       if (Loc == GlobalSymbolTable.end()) {
>
>
> // This is an external symbol, try to get its address from the symbol
>
>         // resolver.
>
>         Addr = Resolver.findSymbol(Name.data()).getAddress();
>
>
> // The call to getSymbolAddress may have caused additional modules to
>
>         // be loaded, which may have added new entries to the
>
>
> // ExternalSymbolRelocations map.  Consquently, we need to update our
>
>         // iterator.  This is also why retrieval of the relocation list
>
>         // associated with this symbol is deferred until below this point.
>
>         // New entries may have been added to the relocation list.
>
>         i = ExternalSymbolRelocations.find(Name);
>
>       } else {
>
>         // We found the symbol in our global table.  It was probably in a
>
>         // Module that we loaded previously.
>
>         const auto &SymInfo = Loc->second;
>
>         Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
>
>                SymInfo.getOffset();
>
>       }
>
>
>
>       if (Addr) {
>
>         DEBUG(dbgs() << "Resolving relocations Name: " << Name << "\t"
>
>           << format("0x%lx", Addr) << "\n");
>
>
> // This list may have been updated when we called getSymbolAddress, so
>
>         // don't change this code to get the list earlier.
>
>         RelocationList &Relocs = i->second;
>
>         resolveRelocationList(Relocs, Addr);
>
>       }
>
>     }
>
>
>
>     ExternalSymbolRelocations.erase(i);
>
>   }
>
> }
>
>
>
> Thanks,
>
>
>
> Eugene
>
> *From:* Eugene Rozenfeld
> *Sent:* Wednesday, June 24, 2015 4:47 PM
> *To:* 'Lang Hames'
> *Cc:* llvmdev at cs.uiuc.edu
> *Subject:* RE: ORC and relocations
>
>
>
> Hi Lang,
>
>
>
> Thank you for your reply. I tried to supply a custom Resolver that just
> returns 0 for the symbols I want to handle manually, but that results in a fatal
> error below. If that can be changed, then yes,
>
> it would be sufficient for my use case.
>
>
>
> Thanks,
>
>
>
> Eugene
>
>
>
> void RuntimeDyldImpl::resolveExternalSymbols() {
>
>   while (!ExternalSymbolRelocations.empty()) {
>
>     StringMap<RelocationList>::iterator
>  i = ExternalSymbolRelocations.begin();
>
>
>
>     StringRef Name = i->first();
>
>     if (Name.size() == 0) {
>
>       // This is an absolute symbol, use an address of zero.
>
>       DEBUG(dbgs() << "Resolving absolute relocations."
>
>                    << "\n");
>
>       RelocationList &Relocs = i->second;
>
>       resolveRelocationList(Relocs, 0);
>
>     } else {
>
>       uint64_t Addr = 0;
>
>       RTDyldSymbolTable::const_iterator
>  Loc = GlobalSymbolTable.find(Name);
>
>       if (Loc == GlobalSymbolTable.end()) {
>
>
> // This is an external symbol, try to get its address from the symbol
>
>         // resolver.
>
>         Addr = Resolver.findSymbol(Name.data()).getAddress();
>
>
> // The call to getSymbolAddress may have caused additional modules to
>
>         // be loaded, which may have added new entries to the
>
>
> // ExternalSymbolRelocations map.  Consquently, we need to update our
>
>         // iterator.  This is also why retrieval of the relocation list
>
>         // associated with this symbol is deferred until below this point.
>
>         // New entries may have been added to the relocation list.
>
>         i = ExternalSymbolRelocations.find(Name);
>
>       } else {
>
>         // We found the symbol in our global table.  It was probably in a
>
>         // Module that we loaded previously.
>
>         const auto &SymInfo = Loc->second;
>
>         Addr = getSectionLoadAddress(SymInfo.getSectionID()) +
>
>                SymInfo.getOffset();
>
>       }
>
>
>
>
> // FIXME: Implement error handling that doesn't kill the host program!
>
>       if (!Addr) {
>
>         report_fatal_error("Program used external function '" + Name +
>
>                            "' which could not be resolved!");
>
>       }
>
>
>
>       DEBUG(dbgs() << "Resolving relocations Name: " << Name << "\t"
>
>         << format("0x%lx", Addr) << "\n");
>
>
> // This list may have been updated when we called getSymbolAddress, so
>
>       // don't change this code to get the list earlier.
>
>       RelocationList &Relocs = i->second;
>
>       resolveRelocationList(Relocs, Addr);
>
>     }
>
>
>
>     ExternalSymbolRelocations.erase(i);
>
>   }
>
> }
>
>
>
>
>
> *From:* Lang Hames [mailto:lhames at gmail.com <lhames at gmail.com>]
> *Sent:* Wednesday, June 24, 2015 4:42 PM
> *To:* Eugene Rozenfeld
> *Cc:* llvmdev at cs.uiuc.edu
> *Subject:* Re: ORC and relocations
>
>
>
> Hi Eugene,
>
>
>
> There's no way to 'skip' application of a relocation.
>
>
>
> You could use the NotifyLoadedFtor and libObject to scan the object ahead
> of time and record the relocations that you'd like to override, then you
> could supply a custom Resolver that just returns '0' for the symbols that
> you want to handle manually.
>
>
>
> There's no way to handle distinct relocations for the same symbol
> differently: For performance reasons we only look up each symbol once and
> re-use the supplied address to apply every relocation for that symbol.
>
>
>
> Is that sufficient to support your use-case?
>
>
>
> Cheers,
>
> Lang.
>
>
>
> On Wed, Jun 24, 2015 at 12:03 AM, Eugene Rozenfeld <
> Eugene.Rozenfeld at microsoft.com> wrote:
>
>   Hello,
>
>
>
> I’m working on LLILC (a jit for the CoreCLR built on ORC), in particular,
> on using LLILC as an ngen jit. I would like to have an ability to be
> notified of relocations that ObjectLinkingLayer  is applying and to be able
> to tell the linking layer not to resolve certain relocations for external
> symbols (so that the client can do some custom resolutions later). The only
> way I found of looking at relocations in the client is via NotifyLoadedFtor
> notifications but I couldn’t find a way of blocking a relocation resolution.
>
>
>
> One way to achieve that is to change the Resolver::findSymbol api to allow
> clients to indicate that the relocations for the symbol shouldn’t be
> resolved and update RuntimeDyldImpl::resolveExternalSymbols accordingly.
> Would this be a reasonable approach?
>
>
>
> Thanks,
>
>
>
> Eugene
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150626/cee1c861/attachment.html>