[LLVMdev] Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

[Kaylor, Andrew]

It's probably best to open a bug.

-Andy

From: Alexey Samsonov [mailto:samsonov at google.com]
Sent: Thursday, January 31, 2013 12:27 AM
To: Kaylor, Andrew
Cc: LLVM Developers Mailing List
Subject: Re: Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests


On Wed, Jan 30, 2013 at 8:34 PM, Kaylor, Andrew <andrew.kaylor at intel.com<mailto:andrew.kaylor at intel.com>> wrote:
Yes, at some point we definitely should introduce stubs as a last resort for x86-64 relocations when the sections are too far apart, but I'd like to avoid it whenever possible.

What I meant in my previous message was that I'd have RecordingMemoryManager use something other than malloc (such as the memory API used by SectionMemoryManager) to keep section near one another.

Ok, I see your point. Should I open the bug to track this, or you'll have a chance to look at this issue soon?


-Andy

From: Alexey Samsonov [mailto:samsonov at google.com<mailto:samsonov at google.com>]
Sent: Wednesday, January 30, 2013 3:59 AM
To: Kaylor, Andrew
Cc: LLVM Developers Mailing List
Subject: Re: Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

Hi Andrew,

Looks like RecordingMemoryManager in lli just calls malloc() and it would be strange to make assumptions (or enforce) that the difference between two returned pointers in 64-bit
virtual address space will be fit into 32 bits. Can we do smth similar to what Adhemerval proposed (see the special case in processRelocationRef for ELF::R_PPC64_REL24 relocations)?

On Tue, Jan 29, 2013 at 9:40 PM, Kaylor, Andrew <andrew.kaylor at intel.com<mailto:andrew.kaylor at intel.com>> wrote:
Hi Alexey,

I think the most likely way to resolve this is to have the RecordingMemoryManager do something more complex to manage its allocations in such a way as to guarantee that they are all within proximity of one another.  The code that is asserting is handling a relocation where code was generated to use a 32-bit relative offset in 64-bit code.  If the two sections involved really are more than a 32-bit offset apart then the generated code will not work.

Alternatively, we could have MCJIT use whatever code generation option will prevent 32-bit relative relocations from being generated in the first place.  That would probably be preferable, but I haven't had success trying to do that in limited efforts up to now.

As it happens, I'm working with the '-use-remote' option for lli this week trying to add support for actual out-of-process execution.  As I do, I'll take a look at the allocation scheme in RecordingMemoryManager and see if there is something reasonable I can do there.

In the meantime, is there any way that you can mark these tests as XFAIL in the sanitizer case?

Thanks,
Andy

From: Alexey Samsonov [mailto:samsonov at google.com<mailto:samsonov at google.com>]
Sent: Tuesday, January 29, 2013 3:36 AM
To: LLVM Developers Mailing List
Cc: Kaylor, Andrew
Subject: Assertions in RuntimeDyldELF in ExecutionEngine/MCJIT tests

Hi!

I'm trying to run LLVM test suite under AddressSanitizer and get test failures in:
    LLVM :: ExecutionEngine/MCJIT/simpletest-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-data-align-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-fp-no-external-funcs-remote.ll
    LLVM :: ExecutionEngine/MCJIT/test-global-init-nonzero-remote.ll

All of them fail with assertion:
lli: /usr/local/google/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp:230: void llvm::RuntimeDyldELF::resolveX86_64Relocation(const llvm::SectionEntry &, uint64_t, uint64_t, uint32_t, int64_t): Assertion `RealOffset <= (2147483647) && RealOffset >= (-2147483647-1)' failed.

The reason is that AddressSanitizer replaces system malloc with its own allocator, which
allocates memory at "unusual" parts of heap and the difference between pointers can be significant
(and doesn't fit in 32 bytes).

I add debug output to calculation of RealOffset in resolveX86_64Relocation:

    uint64_t  FinalAddress = Section.LoadAddress + Offset;
    int64_t RealOffset = *Placeholder + Value + Addend - FinalAddress;
    fprintf(stderr, "%x + %lx + %lx - %lx = %lx\n",
            *Placeholder, Value, Addend, FinalAddress, RealOffset);
    assert(RealOffset <= INT32_MAX && RealOffset >= INT32_MIN);




This is what I get for system malloc:



0 + 7fec9867a000 + 0 - 7fec9867a040 = ffffffffffffffc0
This is what I get for ASan allocator (which results in assert failure):



0 + 600c0000a8a0 + 0 - 6018000090a0 = fffffff400001800

--
Alexey Samsonov, MSK



--
Alexey Samsonov, MSK



--
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130131/f5d4baa5/attachment.html>