[PATCH} [Review Request] MCJIT PIC support for x86-64

[Kaylor, Andrew]

Hi Eli,

Thanks for the suggestions regarding documentation.  I'll see what I can put together.  I believe I still have the diagrams you referred to, and I can put together something more to describe the control flow.

BTW, I wanted to let you know that your old blog posts describing the PIC relocation model were extremely helpful as I was putting this patch together.  Yours was the clearest description of PIC handling that I was able to find.

Also, I meant to ask what ever happened to the lit subtest patch that you submitted about a year and a half ago.  As far as I can tell it was never committed, but I couldn't tell why not.  The "new" tests I'm introducing are just duplications of other tests with variations in invocation arguments.  It would be nice to be able to simplify that ExecutionEngine testing tree.

-Andy

From: Eli Bendersky [mailto:eliben at google.com]
Sent: Wednesday, August 14, 2013 9:28 AM
To: Kaylor, Andrew
Cc: Commit Messages and Patches for LLVM; Jim Grosbach; Rafael Espíndola
Subject: Re: [PATCH} [Review Request] MCJIT PIC support for x86-64



On Tue, Aug 13, 2013 at 11:38 AM, Kaylor, Andrew <andrew.kaylor at intel.com<mailto:andrew.kaylor at intel.com>> wrote:
Hi all,

The attached patch implements PIC relocation support in MCJIT for small and large memory models on x86-64 architecture targets with ELF (plus a few tests).

It seemed like a good idea to solicit some feedback about the choices I've made in my implementation before I commit it.  I'm directly addressing a few people who I think might know something about this stuff, but opinions are welcome from everyone.

I expect that the tests will fail on several non-x86-64 platforms, but I don't know which ones.  If someone can take the time to try them out (only the test patch need be applied for this purpose) I'll mark them as XFAIL in advance.  Otherwise, I'm going to have to rely on the buildbots to tell me what won't work.

I've tried to make the GOT support platform independent where possible, but it only gets executed for x86-64 targets.

There were just a few minor tweaks necessary to support large code model PIC, notably handling of the R_X86_64_PC64 relocation type and recognition of ST_Data symbols.

For small code model, I had to handle R_X86_64_PLT32 and R_X86_64_GOTPCREL relocations and thus some sort of PLT and GOT support.  This is more interesting so I'll describe what I did in a bit of detail here.  As background, since MC emits a relocatable object to be loaded by MCJIT we need to do both linking and loading.  An important implication of this is that the emitted object does not contain '.plt' or '.got' sections.  It's left to RuntimeDyld to allocate those as needed.

For R_X86_64_PLT32 it seemed that if the symbol associated with the relocation was a function defined in the module being loaded I could just remap this as a R_X86_64_PC32 relocation which would result in a PC-relative call directly to the function.  I could have done this with external symbols as well, but since there is no way to guarantee in that case that the external function will be within 2^32 bytes of the relocation target I decided it was preferable to implement something like normal PLT support for that case.

I say "something like normal PLT support" because rather than actually allocate a true PLT section/segment I reused the existing stub mechanism in the RuntimeDyld, and I did not implement any sort of lazy binding mechanism.  I did associate a GOT entry with each stub and implemented the stub as a jump using a RIP-relative reference to the address provided in GOT entry.  However, I am always mapping the GOT entry directly to the function at load time.  It should be fairly straight-forward to implement lazy binding in the future if it is needed.

For the GOT support, I added a SmallVector to RuntimeDyldELF which gets populated with RelocationValueRef objects as GOT-based relocations are processed.  I also added a 'finalizeLoad' method in the base RuntimeDyldImpl class to give the ELF implementation a handy place to allocate memory for the GOT, which it does using RuntimeDyldMemoryManager::allocateDataSection().  Initially, I'm leaving all of the GOT entries initialized to zero.  When a relocation is resolved that references a GOT entry I am using the information provided by RuntimeDyldELF::resolveRelocation() (which now knows the actual address of the symbol) to populate the GOT entry.  Finally, the resolveRelocation handler calculates the PC-relative offset to the GOT entry and fills it in.

The GOT implementation was a little messy because the existing ELF relocation processing code was combining the symbol offset and the relocation addend.  Rather than revise all of the code that relied on this usually reasonable behavior I just added a new field to keep the symbol offset and used that to calculate the original addend in the one case where I needed it.

Hi Andy,

Recently, as the LLVM documentation infrastructure was improved, there was a push for more documentation - which is great. Perhaps at this stage of the evolution of MCJIT, and seeing that there's still interest in pushing it forward - it would make sense to throw together a documentation page that describes its design in broad terms? I really you had a nice class diagram back in the time when you did some major refactoring. Such documentation would definitely help me ramp up on the code (having not touched it for more than a year now...) :-)

This is not to say I'm gating the review with documentation. You can certainly commit this if you like. I'll do my best to find time to review the patch in depth, but it may take... time.

Eli




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130816/3b52b6a8/attachment.html>