[llvm] r204398 - Add an option to MCJIT to have it forward all sections to the

[Kaylor, Andrew]

One reason the non-executable sections weren't being loaded is that when dynamic code is registered with GDB for debugging GDB expects the debug sections not to have been loaded.

As I recall, there is some flag you can set in the ELF header to tell GDB that the debug sections have been loaded, but I don't believe I was ever able to get everything working that way.

-Andy

-----Original Message-----
From: llvm-commits-bounces at cs.uiuc.edu [mailto:llvm-commits-bounces at cs.uiuc.edu] On Behalf Of Philip Reames
Sent: Thursday, March 20, 2014 4:58 PM
To: Lang Hames; llvm-commits at cs.uiuc.edu
Subject: Re: [llvm] r204398 - Add an option to MCJIT to have it forward all sections to the

In this case, I might vote for changing the default.  Given the old behaviour is unlikely to be correct, I don't see the benefit in keeping it as the default.

Philip

On 03/20/2014 02:06 PM, Lang Hames wrote:
> Author: lhames
> Date: Thu Mar 20 16:06:46 2014
> New Revision: 204398
>
> URL: http://llvm.org/viewvc/llvm-project?rev=204398&view=rev
> Log:
> Add an option to MCJIT to have it forward all sections to the 
> RTDyldMemoryManager, regardless of whether it thinks they're "required 
> for execution".
>
> Currently, RuntimeDyld only passes sections that are "required for execution"
> to the RTDyldMemoryManager, and takes "required for execution" to mean 
> exactly "contains symbols or relocations". There are two problems with this:
> (1) It can drop sections with anonymous data that is referenced by code.
> (2) It leaves the JIT client no way to inspect interesting sections that aren't
>      actually required to run the program (e.g dwarf sections).
>
> A test case is still in the works.
>
> Future work: We may want to replace this with a generic section 
> filtering mechanism, but that will require more consideration. For 
> now, this flag at least allows clients to volunteer to do the filtering themselves.
>
> Fixes <rdar://problem/15177691>.
>
>
> Modified:
>      llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
>      llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h
>      llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h
>      llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
>      llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
>
> Modified: llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h
> URL: 
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE
> ngine/ExecutionEngine.h?rev=204398&r1=204397&r2=204398&view=diff
> ======================================================================
> ========
> --- llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h 
> (original)
> +++ llvm/trunk/include/llvm/ExecutionEngine/ExecutionEngine.h Thu Mar 
> +++ 20 16:06:46 2014
> @@ -462,6 +462,21 @@ public:
>       llvm_unreachable("No support for an object cache");
>     }
>   
> +  /// setProcessAllSections (MCJIT Only): By default, only sections 
> + that are  /// "required for execution" are passed to the 
> + RTDyldMemoryManager, and other  /// sections are discarded. Passing 
> + 'true' to this method will cause  /// RuntimeDyld to pass all 
> + sections to its RTDyldMemoryManager regardless  /// of whether they are "required to execute" in the usual sense.
> +  ///
> +  /// Rationale: Some MCJIT clients want to be able to inspect 
> + metadata  /// sections (e.g. Dwarf, Stack-maps) to enable 
> + functionality or analyze  /// performance. Passing these sections to 
> + the memory manager allows the  /// client to make policy about the 
> + relevant sections, rather than having  /// MCJIT do it.
> +  virtual void setProcessAllSections(bool ProcessAllSections) {
> +    llvm_unreachable("No support for ProcessAllSections option");  }
> +
>     /// Return the target machine (if available).
>     virtual TargetMachine *getTargetMachine() { return NULL; }
>   
>
> Modified: llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h
> URL: 
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE
> ngine/RuntimeDyld.h?rev=204398&r1=204397&r2=204398&view=diff
> ======================================================================
> ========
> --- llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h (original)
> +++ llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h Thu Mar 20 
> +++ 16:06:46 2014
> @@ -36,6 +36,7 @@ class RuntimeDyld {
>     // interface.
>     RuntimeDyldImpl *Dyld;
>     RTDyldMemoryManager *MM;
> +  bool ProcessAllSections;
>   protected:
>     // Change the address associated with a section when resolving relocations.
>     // Any relocations already associated with the symbol will be re-resolved.
> @@ -84,6 +85,19 @@ public:
>     void deregisterEHFrames();
>   
>     StringRef getErrorString();
> +
> +  /// By default, only sections that are "required for execution" are 
> + passed to  /// the RTDyldMemoryManager, and other sections are discarded. Passing 'true'
> +  /// to this method will cause RuntimeDyld to pass all sections to 
> + its  /// memory manager regardless of whether they are "required to 
> + execute" in the  /// usual sense. This is useful for inspecting 
> + metadata sections that may not  /// contain relocations, E.g. Debug info, stackmaps.
> +  ///
> +  /// Must be called before the first object file is loaded.
> +  void setProcessAllSections(bool ProcessAllSections) {
> +    assert(!Dyld && "setProcessAllSections must be called before loadObject.");
> +    this->ProcessAllSections = ProcessAllSections;  }
>   };
>   
>   } // end namespace llvm
>
> Modified: llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h
> URL: 
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/MCJ
> IT/MCJIT.h?rev=204398&r1=204397&r2=204398&view=diff
> ======================================================================
> ========
> --- llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h (original)
> +++ llvm/trunk/lib/ExecutionEngine/MCJIT/MCJIT.h Thu Mar 20 16:06:46 
> +++ 2014
> @@ -251,6 +251,10 @@ public:
>     /// Sets the object manager that MCJIT should use to avoid compilation.
>     void setObjectCache(ObjectCache *manager) override;
>   
> +  void setProcessAllSections(bool ProcessAllSections) override {
> +    Dyld.setProcessAllSections(ProcessAllSections);
> +  }
> +
>     void generateCodeForModule(Module *M) override;
>   
>     /// finalizeObject - ensure the module is fully processed and is usable.
>
> Modified: llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
> URL: 
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run
> timeDyld/RuntimeDyld.cpp?rev=204398&r1=204397&r2=204398&view=diff
> ======================================================================
> ========
> --- llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp 
> (original)
> +++ llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp Thu Mar 
> +++ 20 16:06:46 2014
> @@ -161,25 +161,22 @@ ObjectImage* RuntimeDyldImpl::loadObject
>     DEBUG(dbgs() << "Parse relocations:\n");
>     for (section_iterator SI = Obj->begin_sections(), SE = Obj->end_sections();
>          SI != SE; ++SI) {
> -    bool IsFirstRelocation = true;
>       unsigned SectionID = 0;
>       StubMap Stubs;
>       section_iterator RelocatedSection = SI->getRelocatedSection();
>   
> -    for (const RelocationRef &Reloc : SI->relocations()) {
> -      // If it's the first relocation in this section, find its SectionID
> -      if (IsFirstRelocation) {
> -        bool IsCode = false;
> -        Check(RelocatedSection->isText(IsCode));
> -        SectionID =
> -            findOrEmitSection(*Obj, *RelocatedSection, IsCode, LocalSections);
> -        DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");
> -        IsFirstRelocation = false;
> -      }
> +    if ((SI->relocation_begin() != SI->relocation_end()) ||
> +        ProcessAllSections) {
> +      bool IsCode = false;
> +      Check(RelocatedSection->isText(IsCode));
> +      SectionID =
> +        findOrEmitSection(*Obj, *RelocatedSection, IsCode, LocalSections);
> +      DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");
> +    }
>   
> +    for (const RelocationRef &Reloc : SI->relocations())
>         processRelocationRef(SectionID, Reloc, *Obj, LocalSections, LocalSymbols,
>                              Stubs);
> -    }
>     }
>   
>     // Give the subclasses a chance to tie-up any loose ends.
> @@ -665,23 +662,40 @@ RuntimeDyld::RuntimeDyld(RTDyldMemoryMan
>     // permissions are applied.
>     Dyld = 0;
>     MM = mm;
> +  ProcessAllSections = false;
>   }
>   
>   RuntimeDyld::~RuntimeDyld() {
>     delete Dyld;
>   }
>   
> +static std::unique_ptr<RuntimeDyldELF> createRuntimeDyldELF(
> +                                                   RTDyldMemoryManager *MM,
> +                                                   bool 
> +ProcessAllSections) {
> +  std::unique_ptr<RuntimeDyldELF> Dyld(new RuntimeDyldELF(MM));
> +  Dyld->setProcessAllSections(ProcessAllSections);
> +  return Dyld;
> +}
> +
> +static std::unique_ptr<RuntimeDyldMachO> createRuntimeDyldMachO(
> +                                                   RTDyldMemoryManager *MM,
> +                                                   bool 
> +ProcessAllSections) {
> +  std::unique_ptr<RuntimeDyldMachO> Dyld(new RuntimeDyldMachO(MM));
> +  Dyld->setProcessAllSections(ProcessAllSections);
> +  return Dyld;
> +}
> +
>   ObjectImage *RuntimeDyld::loadObject(ObjectFile *InputObject) {
>     std::unique_ptr<ObjectImage> InputImage;
>   
>     if (InputObject->isELF()) {
>       InputImage.reset(RuntimeDyldELF::createObjectImageFromFile(InputObject));
>       if (!Dyld)
> -      Dyld = new RuntimeDyldELF(MM);
> +      Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();
>     } else if (InputObject->isMachO()) {
>       InputImage.reset(RuntimeDyldMachO::createObjectImageFromFile(InputObject));
>       if (!Dyld)
> -      Dyld = new RuntimeDyldMachO(MM);
> +      Dyld = createRuntimeDyldMachO(MM, 
> + ProcessAllSections).release();
>     } else
>       report_fatal_error("Incompatible object format!");
>   
> @@ -704,7 +718,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj
>     case sys::fs::file_magic::elf_core:
>       InputImage.reset(RuntimeDyldELF::createObjectImage(InputBuffer));
>       if (!Dyld)
> -      Dyld = new RuntimeDyldELF(MM);
> +      Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();
>       break;
>     case sys::fs::file_magic::macho_object:
>     case sys::fs::file_magic::macho_executable:
> @@ -718,7 +732,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj
>     case sys::fs::file_magic::macho_dsym_companion:
>       InputImage.reset(RuntimeDyldMachO::createObjectImage(InputBuffer));
>       if (!Dyld)
> -      Dyld = new RuntimeDyldMachO(MM);
> +      Dyld = createRuntimeDyldMachO(MM, 
> + ProcessAllSections).release();
>       break;
>     case sys::fs::file_magic::unknown:
>     case sys::fs::file_magic::bitcode:
>
> Modified: llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
> URL: 
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run
> timeDyld/RuntimeDyldImpl.h?rev=204398&r1=204397&r2=204398&view=diff
> ======================================================================
> ========
> --- llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h 
> (original)
> +++ llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h Thu 
> +++ Mar 20 16:06:46 2014
> @@ -187,6 +187,10 @@ protected:
>     Triple::ArchType Arch;
>     bool IsTargetLittleEndian;
>   
> +  // True if all sections should be passed to the memory manager, 
> + false if only  // sections containing relocations should be. Defaults to 'false'.
> +  bool ProcessAllSections;
> +
>     // This mutex prevents simultaneously loading objects from two different
>     // threads.  This keeps us from having to protect individual data structures
>     // and guarantees that section allocation requests to the memory 
> manager @@ -320,10 +324,15 @@ protected:
>     unsigned computeSectionStubBufSize(ObjectImage &Obj, const 
> SectionRef &Section);
>   
>   public:
> -  RuntimeDyldImpl(RTDyldMemoryManager *mm) : MemMgr(mm), 
> HasError(false) {}
> +  RuntimeDyldImpl(RTDyldMemoryManager *mm)
> +    : MemMgr(mm), ProcessAllSections(false), HasError(false) {}
>   
>     virtual ~RuntimeDyldImpl();
>   
> +  void setProcessAllSections(bool ProcessAllSections) {
> +    this->ProcessAllSections = ProcessAllSections;  }
> +
>     ObjectImage* loadObject(ObjectImage* InputObject);
>   
>     void *getSymbolAddress(StringRef Name) {
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits

_______________________________________________
llvm-commits mailing list
llvm-commits at cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits