[llvm] r204398 - Add an option to MCJIT to have it forward all sections to the

Philip Reames listmail at philipreames.com
Thu Mar 20 17:34:15 PDT 2014


This is a good reason for not changing the default.  :)

Philip

On 03/20/2014 05:13 PM, Kaylor, Andrew wrote:
> One reason the non-executable sections weren't being loaded is that when dynamic code is registered with GDB for debugging GDB expects the debug sections not to have been loaded.
>
> As I recall, there is some flag you can set in the ELF header to tell GDB that the debug sections have been loaded, but I don't believe I was ever able to get everything working that way.
>
> -Andy
>
> -----Original Message-----
> From: llvm-commits-bounces at cs.uiuc.edu [mailto:llvm-commits-bounces at cs.uiuc.edu] On Behalf Of Philip Reames
> Sent: Thursday, March 20, 2014 4:58 PM
> To: Lang Hames; llvm-commits at cs.uiuc.edu
> Subject: Re: [llvm] r204398 - Add an option to MCJIT to have it forward all sections to the
>
> In this case, I might vote for changing the default.  Given the old behaviour is unlikely to be correct, I don't see the benefit in keeping it as the default.
>
> Philip
>
> On 03/20/2014 02:06 PM, Lang Hames wrote:
>> Author: lhames
>> Date: Thu Mar 20 16:06:46 2014
>> New Revision: 204398
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=204398&view=rev
>> Log:
>> Add an option to MCJIT to have it forward all sections to the
>> RTDyldMemoryManager, regardless of whether it thinks they're "required
>> for execution".
>>
>> Currently, RuntimeDyld only passes sections that are "required for execution"
>> to the RTDyldMemoryManager, and takes "required for execution" to mean
>> exactly "contains symbols or relocations". There are two problems with this:
>> (1) It can drop sections with anonymous data that is referenced by code.
>> (2) It leaves the JIT client no way to inspect interesting sections that aren't
>>       actually required to run the program (e.g dwarf sections).
>>
>> A test case is still in the works.
>>
>> Future work: We may want to replace this with a generic section
>> filtering mechanism, but that will require more consideration. For
>> now, this flag at least allows clients to volunteer to do the filtering themselves.
>>
>> Fixes <rdar://problem/15177691>.
>>
>>
>> Modified:
>>       llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
>>       llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h
>>       llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h
>>       llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
>>       llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
>>
>> Modified: llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE
>> ngine/ExecutionEngine.h?rev=204398&r1=204397&r2=204398&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
>> (original)
>> +++ llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h Thu Mar
>> +++ 20 16:06:46 2014
>> @@ -462,6 +462,21 @@ public:
>>        llvm_unreachable("No support for an object cache");
>>      }
>>    
>> +  /// setProcessAllSections (MCJIT Only): By default, only sections
>> + that are  /// "required for execution" are passed to the
>> + RTDyldMemoryManager, and other  /// sections are discarded. Passing
>> + 'true' to this method will cause  /// RuntimeDyld to pass all
>> + sections to its RTDyldMemoryManager regardless  /// of whether they are "required to execute" in the usual sense.
>> +  ///
>> +  /// Rationale: Some MCJIT clients want to be able to inspect
>> + metadata  /// sections (e.g. Dwarf, Stack-maps) to enable
>> + functionality or analyze  /// performance. Passing these sections to
>> + the memory manager allows the  /// client to make policy about the
>> + relevant sections, rather than having  /// MCJIT do it.
>> +  virtual void setProcessAllSections(bool ProcessAllSections) {
>> +    llvm_unreachable("No support for ProcessAllSections option");  }
>> +
>>      /// Return the target machine (if available).
>>      virtual TargetMachine *getTargetMachine() { return NULL; }
>>    
>>
>> Modified: llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE
>> ngine/RuntimeDyld.h?rev=204398&r1=204397&r2=204398&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h (original)
>> +++ llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h Thu Mar 20
>> +++ 16:06:46 2014
>> @@ -36,6 +36,7 @@ class RuntimeDyld {
>>      // interface.
>>      RuntimeDyldImpl *Dyld;
>>      RTDyldMemoryManager *MM;
>> +  bool ProcessAllSections;
>>    protected:
>>      // Change the address associated with a section when resolving relocations.
>>      // Any relocations already associated with the symbol will be re-resolved.
>> @@ -84,6 +85,19 @@ public:
>>      void deregisterEHFrames();
>>    
>>      StringRef getErrorString();
>> +
>> +  /// By default, only sections that are "required for execution" are
>> + passed to  /// the RTDyldMemoryManager, and other sections are discarded. Passing 'true'
>> +  /// to this method will cause RuntimeDyld to pass all sections to
>> + its  /// memory manager regardless of whether they are "required to
>> + execute" in the  /// usual sense. This is useful for inspecting
>> + metadata sections that may not  /// contain relocations, E.g. Debug info, stackmaps.
>> +  ///
>> +  /// Must be called before the first object file is loaded.
>> +  void setProcessAllSections(bool ProcessAllSections) {
>> +    assert(!Dyld && "setProcessAllSections must be called before loadObject.");
>> +    this->ProcessAllSections = ProcessAllSections;  }
>>    };
>>    
>>    } // end namespace llvm
>>
>> Modified: llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/MCJ
>> IT/MCJIT.h?rev=204398&r1=204397&r2=204398&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h (original)
>> +++ llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h Thu Mar 20 16:06:46
>> +++ 2014
>> @@ -251,6 +251,10 @@ public:
>>      /// Sets the object manager that MCJIT should use to avoid compilation.
>>      void setObjectCache(ObjectCache *manager) override;
>>    
>> +  void setProcessAllSections(bool ProcessAllSections) override {
>> +    Dyld.setProcessAllSections(ProcessAllSections);
>> +  }
>> +
>>      void generateCodeForModule(Module *M) override;
>>    
>>      /// finalizeObject - ensure the module is fully processed and is usable.
>>
>> Modified: llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run
>> timeDyld/RuntimeDyld.cpp?rev=204398&r1=204397&r2=204398&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
>> (original)
>> +++ llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp Thu Mar
>> +++ 20 16:06:46 2014
>> @@ -161,25 +161,22 @@ ObjectImage* RuntimeDyldImpl::loadObject
>>      DEBUG(dbgs() << "Parse relocations:\n");
>>      for (section_iterator SI = Obj->begin_sections(), SE = Obj->end_sections();
>>           SI != SE; ++SI) {
>> -    bool IsFirstRelocation = true;
>>        unsigned SectionID = 0;
>>        StubMap Stubs;
>>        section_iterator RelocatedSection = SI->getRelocatedSection();
>>    
>> -    for (const RelocationRef &Reloc : SI->relocations()) {
>> -      // If it's the first relocation in this section, find its SectionID
>> -      if (IsFirstRelocation) {
>> -        bool IsCode = false;
>> -        Check(RelocatedSection->isText(IsCode));
>> -        SectionID =
>> -            findOrEmitSection(*Obj, *RelocatedSection, IsCode, LocalSections);
>> -        DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");
>> -        IsFirstRelocation = false;
>> -      }
>> +    if ((SI->relocation_begin() != SI->relocation_end()) ||
>> +        ProcessAllSections) {
>> +      bool IsCode = false;
>> +      Check(RelocatedSection->isText(IsCode));
>> +      SectionID =
>> +        findOrEmitSection(*Obj, *RelocatedSection, IsCode, LocalSections);
>> +      DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");
>> +    }
>>    
>> +    for (const RelocationRef &Reloc : SI->relocations())
>>          processRelocationRef(SectionID, Reloc, *Obj, LocalSections, LocalSymbols,
>>                               Stubs);
>> -    }
>>      }
>>    
>>      // Give the subclasses a chance to tie-up any loose ends.
>> @@ -665,23 +662,40 @@ RuntimeDyld::RuntimeDyld(RTDyldMemoryMan
>>      // permissions are applied.
>>      Dyld = 0;
>>      MM = mm;
>> +  ProcessAllSections = false;
>>    }
>>    
>>    RuntimeDyld::~RuntimeDyld() {
>>      delete Dyld;
>>    }
>>    
>> +static std::unique_ptr<RuntimeDyldELF> createRuntimeDyldELF(
>> +                                                   RTDyldMemoryManager *MM,
>> +                                                   bool
>> +ProcessAllSections) {
>> +  std::unique_ptr<RuntimeDyldELF> Dyld(new RuntimeDyldELF(MM));
>> +  Dyld->setProcessAllSections(ProcessAllSections);
>> +  return Dyld;
>> +}
>> +
>> +static std::unique_ptr<RuntimeDyldMachO> createRuntimeDyldMachO(
>> +                                                   RTDyldMemoryManager *MM,
>> +                                                   bool
>> +ProcessAllSections) {
>> +  std::unique_ptr<RuntimeDyldMachO> Dyld(new RuntimeDyldMachO(MM));
>> +  Dyld->setProcessAllSections(ProcessAllSections);
>> +  return Dyld;
>> +}
>> +
>>    ObjectImage *RuntimeDyld::loadObject(ObjectFile *InputObject) {
>>      std::unique_ptr<ObjectImage> InputImage;
>>    
>>      if (InputObject->isELF()) {
>>        InputImage.reset(RuntimeDyldELF::createObjectImageFromFile(InputObject));
>>        if (!Dyld)
>> -      Dyld = new RuntimeDyldELF(MM);
>> +      Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();
>>      } else if (InputObject->isMachO()) {
>>        InputImage.reset(RuntimeDyldMachO::createObjectImageFromFile(InputObject));
>>        if (!Dyld)
>> -      Dyld = new RuntimeDyldMachO(MM);
>> +      Dyld = createRuntimeDyldMachO(MM,
>> + ProcessAllSections).release();
>>      } else
>>        report_fatal_error("Incompatible object format!");
>>    
>> @@ -704,7 +718,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj
>>      case sys::fs::file_magic::elf_core:
>>        InputImage.reset(RuntimeDyldELF::createObjectImage(InputBuffer));
>>        if (!Dyld)
>> -      Dyld = new RuntimeDyldELF(MM);
>> +      Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();
>>        break;
>>      case sys::fs::file_magic::macho_object:
>>      case sys::fs::file_magic::macho_executable:
>> @@ -718,7 +732,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj
>>      case sys::fs::file_magic::macho_dsym_companion:
>>        InputImage.reset(RuntimeDyldMachO::createObjectImage(InputBuffer));
>>        if (!Dyld)
>> -      Dyld = new RuntimeDyldMachO(MM);
>> +      Dyld = createRuntimeDyldMachO(MM,
>> + ProcessAllSections).release();
>>        break;
>>      case sys::fs::file_magic::unknown:
>>      case sys::fs::file_magic::bitcode:
>>
>> Modified: llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run
>> timeDyld/RuntimeDyldImpl.h?rev=204398&r1=204397&r2=204398&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
>> (original)
>> +++ llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h Thu
>> +++ Mar 20 16:06:46 2014
>> @@ -187,6 +187,10 @@ protected:
>>      Triple::ArchType Arch;
>>      bool IsTargetLittleEndian;
>>    
>> +  // True if all sections should be passed to the memory manager,
>> + false if only  // sections containing relocations should be. Defaults to 'false'.
>> +  bool ProcessAllSections;
>> +
>>      // This mutex prevents simultaneously loading objects from two different
>>      // threads.  This keeps us from having to protect individual data structures
>>      // and guarantees that section allocation requests to the memory
>> manager @@ -320,10 +324,15 @@ protected:
>>      unsigned computeSectionStubBufSize(ObjectImage &Obj, const
>> SectionRef &Section);
>>    
>>    public:
>> -  RuntimeDyldImpl(RTDyldMemoryManager *mm) : MemMgr(mm),
>> HasError(false) {}
>> +  RuntimeDyldImpl(RTDyldMemoryManager *mm)
>> +    : MemMgr(mm), ProcessAllSections(false), HasError(false) {}
>>    
>>      virtual ~RuntimeDyldImpl();
>>    
>> +  void setProcessAllSections(bool ProcessAllSections) {
>> +    this->ProcessAllSections = ProcessAllSections;  }
>> +
>>      ObjectImage* loadObject(ObjectImage* InputObject);
>>    
>>      void *getSymbolAddress(StringRef Name) {
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits




More information about the llvm-commits mailing list