[llvm] r204398 - Add an option to MCJIT to have it forward all sections to the

Lang Hames lhames at gmail.com
Fri Mar 21 10:36:43 PDT 2014


Hi Philip, Andy,

I was just playing it safe with the decision to keep the existing behavior
by default. It's good to hear that it was justified.

If/when I move on to implementing generic filters for sections I'll talk to
the debugger guys and make sure our defaults Do The Right Thing for debug
info sections.

Andy - Thanks very much for keeping an eye on this. My understanding of ELF
use-cases is improving, but still patchy.

Cheers,
Lang.


On Thu, Mar 20, 2014 at 5:34 PM, Philip Reames <listmail at philipreames.com>wrote:

> This is a good reason for not changing the default.  :)
>
> Philip
>
>
> On 03/20/2014 05:13 PM, Kaylor, Andrew wrote:
>
>> One reason the non-executable sections weren't being loaded is that when
>> dynamic code is registered with GDB for debugging GDB expects the debug
>> sections not to have been loaded.
>>
>> As I recall, there is some flag you can set in the ELF header to tell GDB
>> that the debug sections have been loaded, but I don't believe I was ever
>> able to get everything working that way.
>>
>> -Andy
>>
>> -----Original Message-----
>> From: llvm-commits-bounces at cs.uiuc.edu [mailto:llvm-commits-bounces@
>> cs.uiuc.edu] On Behalf Of Philip Reames
>> Sent: Thursday, March 20, 2014 4:58 PM
>> To: Lang Hames; llvm-commits at cs.uiuc.edu
>> Subject: Re: [llvm] r204398 - Add an option to MCJIT to have it forward
>> all sections to the
>>
>> In this case, I might vote for changing the default.  Given the old
>> behaviour is unlikely to be correct, I don't see the benefit in keeping it
>> as the default.
>>
>> Philip
>>
>> On 03/20/2014 02:06 PM, Lang Hames wrote:
>>
>>> Author: lhames
>>> Date: Thu Mar 20 16:06:46 2014
>>> New Revision: 204398
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=204398&view=rev
>>> Log:
>>> Add an option to MCJIT to have it forward all sections to the
>>> RTDyldMemoryManager, regardless of whether it thinks they're "required
>>> for execution".
>>>
>>> Currently, RuntimeDyld only passes sections that are "required for
>>> execution"
>>> to the RTDyldMemoryManager, and takes "required for execution" to mean
>>> exactly "contains symbols or relocations". There are two problems with
>>> this:
>>> (1) It can drop sections with anonymous data that is referenced by code.
>>> (2) It leaves the JIT client no way to inspect interesting sections that
>>> aren't
>>>       actually required to run the program (e.g dwarf sections).
>>>
>>> A test case is still in the works.
>>>
>>> Future work: We may want to replace this with a generic section
>>> filtering mechanism, but that will require more consideration. For
>>> now, this flag at least allows clients to volunteer to do the filtering
>>> themselves.
>>>
>>> Fixes <rdar://problem/15177691>.
>>>
>>>
>>> Modified:
>>>       llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
>>>       llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h
>>>       llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h
>>>       llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
>>>       llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
>>>
>>> Modified: llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE
>>> ngine/ExecutionEngine.h?rev=204398&r1=204397&r2=204398&view=diff
>>> ======================================================================
>>> ========
>>> --- llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
>>> (original)
>>> +++ llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h Thu Mar
>>> +++ 20 16:06:46 2014
>>> @@ -462,6 +462,21 @@ public:
>>>        llvm_unreachable("No support for an object cache");
>>>      }
>>>    +  /// setProcessAllSections (MCJIT Only): By default, only sections
>>> + that are  /// "required for execution" are passed to the
>>> + RTDyldMemoryManager, and other  /// sections are discarded. Passing
>>> + 'true' to this method will cause  /// RuntimeDyld to pass all
>>> + sections to its RTDyldMemoryManager regardless  /// of whether they
>>> are "required to execute" in the usual sense.
>>> +  ///
>>> +  /// Rationale: Some MCJIT clients want to be able to inspect
>>> + metadata  /// sections (e.g. Dwarf, Stack-maps) to enable
>>> + functionality or analyze  /// performance. Passing these sections to
>>> + the memory manager allows the  /// client to make policy about the
>>> + relevant sections, rather than having  /// MCJIT do it.
>>> +  virtual void setProcessAllSections(bool ProcessAllSections) {
>>> +    llvm_unreachable("No support for ProcessAllSections option");  }
>>> +
>>>      /// Return the target machine (if available).
>>>      virtual TargetMachine *getTargetMachine() { return NULL; }
>>>
>>> Modified: llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE
>>> ngine/RuntimeDyld.h?rev=204398&r1=204397&r2=204398&view=diff
>>> ======================================================================
>>> ========
>>> --- llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h (original)
>>> +++ llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h Thu Mar 20
>>> +++ 16:06:46 2014
>>> @@ -36,6 +36,7 @@ class RuntimeDyld {
>>>      // interface.
>>>      RuntimeDyldImpl *Dyld;
>>>      RTDyldMemoryManager *MM;
>>> +  bool ProcessAllSections;
>>>    protected:
>>>      // Change the address associated with a section when resolving
>>> relocations.
>>>      // Any relocations already associated with the symbol will be
>>> re-resolved.
>>> @@ -84,6 +85,19 @@ public:
>>>      void deregisterEHFrames();
>>>         StringRef getErrorString();
>>> +
>>> +  /// By default, only sections that are "required for execution" are
>>> + passed to  /// the RTDyldMemoryManager, and other sections are
>>> discarded. Passing 'true'
>>> +  /// to this method will cause RuntimeDyld to pass all sections to
>>> + its  /// memory manager regardless of whether they are "required to
>>> + execute" in the  /// usual sense. This is useful for inspecting
>>> + metadata sections that may not  /// contain relocations, E.g. Debug
>>> info, stackmaps.
>>> +  ///
>>> +  /// Must be called before the first object file is loaded.
>>> +  void setProcessAllSections(bool ProcessAllSections) {
>>> +    assert(!Dyld && "setProcessAllSections must be called before
>>> loadObject.");
>>> +    this->ProcessAllSections = ProcessAllSections;  }
>>>    };
>>>       } // end namespace llvm
>>>
>>> Modified: llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/MCJ
>>> IT/MCJIT.h?rev=204398&r1=204397&r2=204398&view=diff
>>> ======================================================================
>>> ========
>>> --- llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h (original)
>>> +++ llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h Thu Mar 20 16:06:46
>>> +++ 2014
>>> @@ -251,6 +251,10 @@ public:
>>>      /// Sets the object manager that MCJIT should use to avoid
>>> compilation.
>>>      void setObjectCache(ObjectCache *manager) override;
>>>    +  void setProcessAllSections(bool ProcessAllSections) override {
>>> +    Dyld.setProcessAllSections(ProcessAllSections);
>>> +  }
>>> +
>>>      void generateCodeForModule(Module *M) override;
>>>         /// finalizeObject - ensure the module is fully processed and is
>>> usable.
>>>
>>> Modified: llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run
>>> timeDyld/RuntimeDyld.cpp?rev=204398&r1=204397&r2=204398&view=diff
>>> ======================================================================
>>> ========
>>> --- llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
>>> (original)
>>> +++ llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp Thu Mar
>>> +++ 20 16:06:46 2014
>>> @@ -161,25 +161,22 @@ ObjectImage* RuntimeDyldImpl::loadObject
>>>      DEBUG(dbgs() << "Parse relocations:\n");
>>>      for (section_iterator SI = Obj->begin_sections(), SE =
>>> Obj->end_sections();
>>>           SI != SE; ++SI) {
>>> -    bool IsFirstRelocation = true;
>>>        unsigned SectionID = 0;
>>>        StubMap Stubs;
>>>        section_iterator RelocatedSection = SI->getRelocatedSection();
>>>    -    for (const RelocationRef &Reloc : SI->relocations()) {
>>> -      // If it's the first relocation in this section, find its
>>> SectionID
>>> -      if (IsFirstRelocation) {
>>> -        bool IsCode = false;
>>> -        Check(RelocatedSection->isText(IsCode));
>>> -        SectionID =
>>> -            findOrEmitSection(*Obj, *RelocatedSection, IsCode,
>>> LocalSections);
>>> -        DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");
>>> -        IsFirstRelocation = false;
>>> -      }
>>> +    if ((SI->relocation_begin() != SI->relocation_end()) ||
>>> +        ProcessAllSections) {
>>> +      bool IsCode = false;
>>> +      Check(RelocatedSection->isText(IsCode));
>>> +      SectionID =
>>> +        findOrEmitSection(*Obj, *RelocatedSection, IsCode,
>>> LocalSections);
>>> +      DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");
>>> +    }
>>>    +    for (const RelocationRef &Reloc : SI->relocations())
>>>          processRelocationRef(SectionID, Reloc, *Obj, LocalSections,
>>> LocalSymbols,
>>>                               Stubs);
>>> -    }
>>>      }
>>>         // Give the subclasses a chance to tie-up any loose ends.
>>> @@ -665,23 +662,40 @@ RuntimeDyld::RuntimeDyld(RTDyldMemoryMan
>>>      // permissions are applied.
>>>      Dyld = 0;
>>>      MM = mm;
>>> +  ProcessAllSections = false;
>>>    }
>>>       RuntimeDyld::~RuntimeDyld() {
>>>      delete Dyld;
>>>    }
>>>    +static std::unique_ptr<RuntimeDyldELF> createRuntimeDyldELF(
>>> +                                                   RTDyldMemoryManager
>>> *MM,
>>> +                                                   bool
>>> +ProcessAllSections) {
>>> +  std::unique_ptr<RuntimeDyldELF> Dyld(new RuntimeDyldELF(MM));
>>> +  Dyld->setProcessAllSections(ProcessAllSections);
>>> +  return Dyld;
>>> +}
>>> +
>>> +static std::unique_ptr<RuntimeDyldMachO> createRuntimeDyldMachO(
>>> +                                                   RTDyldMemoryManager
>>> *MM,
>>> +                                                   bool
>>> +ProcessAllSections) {
>>> +  std::unique_ptr<RuntimeDyldMachO> Dyld(new RuntimeDyldMachO(MM));
>>> +  Dyld->setProcessAllSections(ProcessAllSections);
>>> +  return Dyld;
>>> +}
>>> +
>>>    ObjectImage *RuntimeDyld::loadObject(ObjectFile *InputObject) {
>>>      std::unique_ptr<ObjectImage> InputImage;
>>>         if (InputObject->isELF()) {
>>>        InputImage.reset(RuntimeDyldELF::createObjectImageFromFile(
>>> InputObject));
>>>        if (!Dyld)
>>> -      Dyld = new RuntimeDyldELF(MM);
>>> +      Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();
>>>      } else if (InputObject->isMachO()) {
>>>        InputImage.reset(RuntimeDyldMachO::createObjectImageFromFile(
>>> InputObject));
>>>        if (!Dyld)
>>> -      Dyld = new RuntimeDyldMachO(MM);
>>> +      Dyld = createRuntimeDyldMachO(MM,
>>> + ProcessAllSections).release();
>>>      } else
>>>        report_fatal_error("Incompatible object format!");
>>>    @@ -704,7 +718,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj
>>>      case sys::fs::file_magic::elf_core:
>>>        InputImage.reset(RuntimeDyldELF::createObjectImage(InputBuffer));
>>>        if (!Dyld)
>>> -      Dyld = new RuntimeDyldELF(MM);
>>> +      Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();
>>>        break;
>>>      case sys::fs::file_magic::macho_object:
>>>      case sys::fs::file_magic::macho_executable:
>>> @@ -718,7 +732,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj
>>>      case sys::fs::file_magic::macho_dsym_companion:
>>>        InputImage.reset(RuntimeDyldMachO::createObjectImage(InputBuffer)
>>> );
>>>        if (!Dyld)
>>> -      Dyld = new RuntimeDyldMachO(MM);
>>> +      Dyld = createRuntimeDyldMachO(MM,
>>> + ProcessAllSections).release();
>>>        break;
>>>      case sys::fs::file_magic::unknown:
>>>      case sys::fs::file_magic::bitcode:
>>>
>>> Modified: llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run
>>> timeDyld/RuntimeDyldImpl.h?rev=204398&r1=204397&r2=204398&view=diff
>>> ======================================================================
>>> ========
>>> --- llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
>>> (original)
>>> +++ llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h Thu
>>> +++ Mar 20 16:06:46 2014
>>> @@ -187,6 +187,10 @@ protected:
>>>      Triple::ArchType Arch;
>>>      bool IsTargetLittleEndian;
>>>    +  // True if all sections should be passed to the memory manager,
>>> + false if only  // sections containing relocations should be. Defaults
>>> to 'false'.
>>> +  bool ProcessAllSections;
>>> +
>>>      // This mutex prevents simultaneously loading objects from two
>>> different
>>>      // threads.  This keeps us from having to protect individual data
>>> structures
>>>      // and guarantees that section allocation requests to the memory
>>> manager @@ -320,10 +324,15 @@ protected:
>>>      unsigned computeSectionStubBufSize(ObjectImage &Obj, const
>>> SectionRef &Section);
>>>       public:
>>> -  RuntimeDyldImpl(RTDyldMemoryManager *mm) : MemMgr(mm),
>>> HasError(false) {}
>>> +  RuntimeDyldImpl(RTDyldMemoryManager *mm)
>>> +    : MemMgr(mm), ProcessAllSections(false), HasError(false) {}
>>>         virtual ~RuntimeDyldImpl();
>>>    +  void setProcessAllSections(bool ProcessAllSections) {
>>> +    this->ProcessAllSections = ProcessAllSections;  }
>>> +
>>>      ObjectImage* loadObject(ObjectImage* InputObject);
>>>         void *getSymbolAddress(StringRef Name) {
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140321/8eee870d/attachment.html>


More information about the llvm-commits mailing list