<div dir="ltr">Hi Philip, Andy,<div><br></div><div>I was just playing it safe with the decision to keep the existing behavior by default. It's good to hear that it was justified.</div><div><br></div><div>If/when I move on to implementing generic filters for sections I'll talk to the debugger guys and make sure our defaults Do The Right Thing for debug info sections.</div>
<div><br></div><div>Andy - Thanks very much for keeping an eye on this. My understanding of ELF use-cases is improving, but still patchy.</div><div><br></div><div>Cheers,</div><div>Lang. </div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Thu, Mar 20, 2014 at 5:34 PM, Philip Reames <span dir="ltr"><<a href="mailto:listmail@philipreames.com" target="_blank">listmail@philipreames.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This is a good reason for not changing the default. :)<span class="HOEnZb"><font color="#888888"><br>
<br>
Philip</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
On 03/20/2014 05:13 PM, Kaylor, Andrew wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
One reason the non-executable sections weren't being loaded is that when dynamic code is registered with GDB for debugging GDB expects the debug sections not to have been loaded.<br>
<br>
As I recall, there is some flag you can set in the ELF header to tell GDB that the debug sections have been loaded, but I don't believe I was ever able to get everything working that way.<br>
<br>
-Andy<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:llvm-commits-bounces@cs.uiuc.edu" target="_blank">llvm-commits-bounces@cs.uiuc.<u></u>edu</a> [mailto:<a href="mailto:llvm-commits-bounces@cs.uiuc.edu" target="_blank">llvm-commits-bounces@<u></u>cs.uiuc.edu</a>] On Behalf Of Philip Reames<br>
Sent: Thursday, March 20, 2014 4:58 PM<br>
To: Lang Hames; <a href="mailto:llvm-commits@cs.uiuc.edu" target="_blank">llvm-commits@cs.uiuc.edu</a><br>
Subject: Re: [llvm] r204398 - Add an option to MCJIT to have it forward all sections to the<br>
<br>
In this case, I might vote for changing the default. Given the old behaviour is unlikely to be correct, I don't see the benefit in keeping it as the default.<br>
<br>
Philip<br>
<br>
On 03/20/2014 02:06 PM, Lang Hames wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Author: lhames<br>
Date: Thu Mar 20 16:06:46 2014<br>
New Revision: 204398<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=204398&view=rev" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project?rev=204398&view=rev</a><br>
Log:<br>
Add an option to MCJIT to have it forward all sections to the<br>
RTDyldMemoryManager, regardless of whether it thinks they're "required<br>
for execution".<br>
<br>
Currently, RuntimeDyld only passes sections that are "required for execution"<br>
to the RTDyldMemoryManager, and takes "required for execution" to mean<br>
exactly "contains symbols or relocations". There are two problems with this:<br>
(1) It can drop sections with anonymous data that is referenced by code.<br>
(2) It leaves the JIT client no way to inspect interesting sections that aren't<br>
actually required to run the program (e.g dwarf sections).<br>
<br>
A test case is still in the works.<br>
<br>
Future work: We may want to replace this with a generic section<br>
filtering mechanism, but that will require more consideration. For<br>
now, this flag at least allows clients to volunteer to do the filtering themselves.<br>
<br>
Fixes <rdar://problem/15177691>.<br>
<br>
<br>
Modified:<br>
llvm/trunk/include/llvm/<u></u>ExecutionEngine/<u></u>ExecutionEngine.h<br>
llvm/trunk/include/llvm/<u></u>ExecutionEngine/RuntimeDyld.h<br>
llvm/trunk/lib/<u></u>ExecutionEngine/MCJIT/MCJIT.h<br>
llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyld.cpp<br>
llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyldImpl.h<br>
<br>
Modified: llvm/trunk/include/llvm/<u></u>ExecutionEngine/<u></u>ExecutionEngine.h<br>
URL:<br>
<a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project/llvm/trunk/include/<u></u>llvm/ExecutionE</a><br>
ngine/ExecutionEngine.h?rev=<u></u>204398&r1=204397&r2=204398&<u></u>view=diff<br>
==============================<u></u>==============================<u></u>==========<br>
========<br>
--- llvm/trunk/include/llvm/<u></u>ExecutionEngine/<u></u>ExecutionEngine.h<br>
(original)<br>
+++ llvm/trunk/include/llvm/<u></u>ExecutionEngine/<u></u>ExecutionEngine.h Thu Mar<br>
+++ 20 16:06:46 2014<br>
@@ -462,6 +462,21 @@ public:<br>
llvm_unreachable("No support for an object cache");<br>
}<br>
+ /// setProcessAllSections (MCJIT Only): By default, only sections<br>
+ that are /// "required for execution" are passed to the<br>
+ RTDyldMemoryManager, and other /// sections are discarded. Passing<br>
+ 'true' to this method will cause /// RuntimeDyld to pass all<br>
+ sections to its RTDyldMemoryManager regardless /// of whether they are "required to execute" in the usual sense.<br>
+ ///<br>
+ /// Rationale: Some MCJIT clients want to be able to inspect<br>
+ metadata /// sections (e.g. Dwarf, Stack-maps) to enable<br>
+ functionality or analyze /// performance. Passing these sections to<br>
+ the memory manager allows the /// client to make policy about the<br>
+ relevant sections, rather than having /// MCJIT do it.<br>
+ virtual void setProcessAllSections(bool ProcessAllSections) {<br>
+ llvm_unreachable("No support for ProcessAllSections option"); }<br>
+<br>
/// Return the target machine (if available).<br>
virtual TargetMachine *getTargetMachine() { return NULL; }<br>
<br>
Modified: llvm/trunk/include/llvm/<u></u>ExecutionEngine/RuntimeDyld.h<br>
URL:<br>
<a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ExecutionE" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project/llvm/trunk/include/<u></u>llvm/ExecutionE</a><br>
ngine/RuntimeDyld.h?rev=<u></u>204398&r1=204397&r2=204398&<u></u>view=diff<br>
==============================<u></u>==============================<u></u>==========<br>
========<br>
--- llvm/trunk/include/llvm/<u></u>ExecutionEngine/RuntimeDyld.h (original)<br>
+++ llvm/trunk/include/llvm/<u></u>ExecutionEngine/RuntimeDyld.h Thu Mar 20<br>
+++ 16:06:46 2014<br>
@@ -36,6 +36,7 @@ class RuntimeDyld {<br>
// interface.<br>
RuntimeDyldImpl *Dyld;<br>
RTDyldMemoryManager *MM;<br>
+ bool ProcessAllSections;<br>
protected:<br>
// Change the address associated with a section when resolving relocations.<br>
// Any relocations already associated with the symbol will be re-resolved.<br>
@@ -84,6 +85,19 @@ public:<br>
void deregisterEHFrames();<br>
StringRef getErrorString();<br>
+<br>
+ /// By default, only sections that are "required for execution" are<br>
+ passed to /// the RTDyldMemoryManager, and other sections are discarded. Passing 'true'<br>
+ /// to this method will cause RuntimeDyld to pass all sections to<br>
+ its /// memory manager regardless of whether they are "required to<br>
+ execute" in the /// usual sense. This is useful for inspecting<br>
+ metadata sections that may not /// contain relocations, E.g. Debug info, stackmaps.<br>
+ ///<br>
+ /// Must be called before the first object file is loaded.<br>
+ void setProcessAllSections(bool ProcessAllSections) {<br>
+ assert(!Dyld && "setProcessAllSections must be called before loadObject.");<br>
+ this->ProcessAllSections = ProcessAllSections; }<br>
};<br>
} // end namespace llvm<br>
<br>
Modified: llvm/trunk/lib/<u></u>ExecutionEngine/MCJIT/MCJIT.h<br>
URL:<br>
<a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/MCJ" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project/llvm/trunk/lib/<u></u>ExecutionEngine/MCJ</a><br>
IT/MCJIT.h?rev=204398&r1=<u></u>204397&r2=204398&view=diff<br>
==============================<u></u>==============================<u></u>==========<br>
========<br>
--- llvm/trunk/lib/<u></u>ExecutionEngine/MCJIT/MCJIT.h (original)<br>
+++ llvm/trunk/lib/<u></u>ExecutionEngine/MCJIT/MCJIT.h Thu Mar 20 16:06:46<br>
+++ 2014<br>
@@ -251,6 +251,10 @@ public:<br>
/// Sets the object manager that MCJIT should use to avoid compilation.<br>
void setObjectCache(ObjectCache *manager) override;<br>
+ void setProcessAllSections(bool ProcessAllSections) override {<br>
+ Dyld.setProcessAllSections(<u></u>ProcessAllSections);<br>
+ }<br>
+<br>
void generateCodeForModule(Module *M) override;<br>
/// finalizeObject - ensure the module is fully processed and is usable.<br>
<br>
Modified: llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyld.cpp<br>
URL:<br>
<a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project/llvm/trunk/lib/<u></u>ExecutionEngine/Run</a><br>
timeDyld/RuntimeDyld.cpp?rev=<u></u>204398&r1=204397&r2=204398&<u></u>view=diff<br>
==============================<u></u>==============================<u></u>==========<br>
========<br>
--- llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyld.cpp<br>
(original)<br>
+++ llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyld.cpp Thu Mar<br>
+++ 20 16:06:46 2014<br>
@@ -161,25 +161,22 @@ ObjectImage* RuntimeDyldImpl::loadObject<br>
DEBUG(dbgs() << "Parse relocations:\n");<br>
for (section_iterator SI = Obj->begin_sections(), SE = Obj->end_sections();<br>
SI != SE; ++SI) {<br>
- bool IsFirstRelocation = true;<br>
unsigned SectionID = 0;<br>
StubMap Stubs;<br>
section_iterator RelocatedSection = SI->getRelocatedSection();<br>
- for (const RelocationRef &Reloc : SI->relocations()) {<br>
- // If it's the first relocation in this section, find its SectionID<br>
- if (IsFirstRelocation) {<br>
- bool IsCode = false;<br>
- Check(RelocatedSection-><u></u>isText(IsCode));<br>
- SectionID =<br>
- findOrEmitSection(*Obj, *RelocatedSection, IsCode, LocalSections);<br>
- DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");<br>
- IsFirstRelocation = false;<br>
- }<br>
+ if ((SI->relocation_begin() != SI->relocation_end()) ||<br>
+ ProcessAllSections) {<br>
+ bool IsCode = false;<br>
+ Check(RelocatedSection-><u></u>isText(IsCode));<br>
+ SectionID =<br>
+ findOrEmitSection(*Obj, *RelocatedSection, IsCode, LocalSections);<br>
+ DEBUG(dbgs() << "\tSectionID: " << SectionID << "\n");<br>
+ }<br>
+ for (const RelocationRef &Reloc : SI->relocations())<br>
processRelocationRef(<u></u>SectionID, Reloc, *Obj, LocalSections, LocalSymbols,<br>
Stubs);<br>
- }<br>
}<br>
// Give the subclasses a chance to tie-up any loose ends.<br>
@@ -665,23 +662,40 @@ RuntimeDyld::RuntimeDyld(<u></u>RTDyldMemoryMan<br>
// permissions are applied.<br>
Dyld = 0;<br>
MM = mm;<br>
+ ProcessAllSections = false;<br>
}<br>
RuntimeDyld::~RuntimeDyld() {<br>
delete Dyld;<br>
}<br>
+static std::unique_ptr<<u></u>RuntimeDyldELF> createRuntimeDyldELF(<br>
+ RTDyldMemoryManager *MM,<br>
+ bool<br>
+ProcessAllSections) {<br>
+ std::unique_ptr<<u></u>RuntimeDyldELF> Dyld(new RuntimeDyldELF(MM));<br>
+ Dyld->setProcessAllSections(<u></u>ProcessAllSections);<br>
+ return Dyld;<br>
+}<br>
+<br>
+static std::unique_ptr<<u></u>RuntimeDyldMachO> createRuntimeDyldMachO(<br>
+ RTDyldMemoryManager *MM,<br>
+ bool<br>
+ProcessAllSections) {<br>
+ std::unique_ptr<<u></u>RuntimeDyldMachO> Dyld(new RuntimeDyldMachO(MM));<br>
+ Dyld->setProcessAllSections(<u></u>ProcessAllSections);<br>
+ return Dyld;<br>
+}<br>
+<br>
ObjectImage *RuntimeDyld::loadObject(<u></u>ObjectFile *InputObject) {<br>
std::unique_ptr<ObjectImage> InputImage;<br>
if (InputObject->isELF()) {<br>
InputImage.reset(<u></u>RuntimeDyldELF::<u></u>createObjectImageFromFile(<u></u>InputObject));<br>
if (!Dyld)<br>
- Dyld = new RuntimeDyldELF(MM);<br>
+ Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();<br>
} else if (InputObject->isMachO()) {<br>
InputImage.reset(<u></u>RuntimeDyldMachO::<u></u>createObjectImageFromFile(<u></u>InputObject));<br>
if (!Dyld)<br>
- Dyld = new RuntimeDyldMachO(MM);<br>
+ Dyld = createRuntimeDyldMachO(MM,<br>
+ ProcessAllSections).release();<br>
} else<br>
report_fatal_error("<u></u>Incompatible object format!");<br>
@@ -704,7 +718,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj<br>
case sys::fs::file_magic::elf_core:<br>
InputImage.reset(<u></u>RuntimeDyldELF::<u></u>createObjectImage(InputBuffer)<u></u>);<br>
if (!Dyld)<br>
- Dyld = new RuntimeDyldELF(MM);<br>
+ Dyld = createRuntimeDyldELF(MM, ProcessAllSections).release();<br>
break;<br>
case sys::fs::file_magic::macho_<u></u>object:<br>
case sys::fs::file_magic::macho_<u></u>executable:<br>
@@ -718,7 +732,7 @@ ObjectImage *RuntimeDyld::loadObject(Obj<br>
case sys::fs::file_magic::macho_<u></u>dsym_companion:<br>
InputImage.reset(<u></u>RuntimeDyldMachO::<u></u>createObjectImage(InputBuffer)<u></u>);<br>
if (!Dyld)<br>
- Dyld = new RuntimeDyldMachO(MM);<br>
+ Dyld = createRuntimeDyldMachO(MM,<br>
+ ProcessAllSections).release();<br>
break;<br>
case sys::fs::file_magic::unknown:<br>
case sys::fs::file_magic::bitcode:<br>
<br>
Modified: llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyldImpl.h<br>
URL:<br>
<a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/Run" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project/llvm/trunk/lib/<u></u>ExecutionEngine/Run</a><br>
timeDyld/RuntimeDyldImpl.h?<u></u>rev=204398&r1=204397&r2=<u></u>204398&view=diff<br>
==============================<u></u>==============================<u></u>==========<br>
========<br>
--- llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyldImpl.h<br>
(original)<br>
+++ llvm/trunk/lib/<u></u>ExecutionEngine/RuntimeDyld/<u></u>RuntimeDyldImpl.h Thu<br>
+++ Mar 20 16:06:46 2014<br>
@@ -187,6 +187,10 @@ protected:<br>
Triple::ArchType Arch;<br>
bool IsTargetLittleEndian;<br>
+ // True if all sections should be passed to the memory manager,<br>
+ false if only // sections containing relocations should be. Defaults to 'false'.<br>
+ bool ProcessAllSections;<br>
+<br>
// This mutex prevents simultaneously loading objects from two different<br>
// threads. This keeps us from having to protect individual data structures<br>
// and guarantees that section allocation requests to the memory<br>
manager @@ -320,10 +324,15 @@ protected:<br>
unsigned computeSectionStubBufSize(<u></u>ObjectImage &Obj, const<br>
SectionRef &Section);<br>
public:<br>
- RuntimeDyldImpl(<u></u>RTDyldMemoryManager *mm) : MemMgr(mm),<br>
HasError(false) {}<br>
+ RuntimeDyldImpl(<u></u>RTDyldMemoryManager *mm)<br>
+ : MemMgr(mm), ProcessAllSections(false), HasError(false) {}<br>
virtual ~RuntimeDyldImpl();<br>
+ void setProcessAllSections(bool ProcessAllSections) {<br>
+ this->ProcessAllSections = ProcessAllSections; }<br>
+<br>
ObjectImage* loadObject(ObjectImage* InputObject);<br>
void *getSymbolAddress(StringRef Name) {<br>
<br>
<br>
______________________________<u></u>_________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu" target="_blank">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/<u></u>mailman/listinfo/llvm-commits</a><br>
</blockquote>
______________________________<u></u>_________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu" target="_blank">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/<u></u>mailman/listinfo/llvm-commits</a><br>
</blockquote>
<br>
</div></div></blockquote></div><br></div>