[llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow when compiling for x86_64

Tue Nov 24 02:12:08 PST 2015

I just tried going along path (1), but I end up with the same issue for a
different symbol again, so it seems that (3) is my only option.
Will take a look around dyld to see if this problem has already been solved
anywhere, but if not, I'll see if I can write a more general solution that
handles overflowing 32 bit relocations automatically and upstream it.
I'll also need to learn some LLVM IR and how to inject this bit of code.
Should be a lot of ... fun ?

I have no idea what a scratch register is.

Thanks very much for your help again!

--
Joshua Gerrard
JUCE Software Developer

*ROLI’s **award-winning*
<http://www.telegraph.co.uk/luxury/design/31520/the-seaboard-grand-piano-wins-designs-of-the-year-2014-award.html>*
Seaboard
GRAND, celebrated as the “**piano of the future*
<http://edition.cnn.com/2013/09/27/tech/innovation/hans-zimmer-seaboard-future-piano/>*”,
is now joined by the **Seaboard RISE*
<https://www.youtube.com/watch?v=fGr7VbDiRNw>*, “**every bit as slimline
and attractive as its bigger brother*
<http://www.soundonsound.com/news?NewsID=18726>*”. The press is hailing the
Seaboard RISE as “**innovative*
<http://www.wired.co.uk/news/archive/2015-09/10/seaboard-rise-digital-keyboard-launch-uk-price>*”,
“**expressive*
<http://createdigitalmusic.com/2015/09/new-roli-instrument-wants-make-expressive-control-mainstream/>*”,
“**accessible*
<http://createdigitalmusic.com/2015/09/new-roli-instrument-wants-make-expressive-control-mainstream/>*”,
and “**a keyboard controller that does to piano keys what 3D touch does to
the iPhone*
<http://www.slashgear.com/roli-seaboard-rise-is-like-3d-touch-for-musicians-11404216/>*”.
Now available for preorder at **www.roli.com* <http://www.roli.com/>*.*

On 23 November 2015 at 19:28, Andy Ayers <andya at microsoft.com> wrote:

> Yes, that’s what case (3) is....
>
>
>
> Currently you have something like:
>
>
>
> @foo()
>
>      ...
>
>      call RIP + 32-bit disp to __fperrraise
>
>     ...
>
>
>
> That only works if __fperrraise is sufficiently close to the call.
>
>
>
> You can leave some space in the .text section that contains @foo, and when
> you load that section, if __fperrraise is too far away, you can create a
> bit of code there to jump to _fperrraise with a 64 bit disp (whose value
> you know, so it will be a literal), and call that bit of code from @foo.
> Since the stub is in the same section it will definitely be reachable.
>
>
>
> It should work pretty generally. The jmp from the stub will be
> transparent, though there might be some trickiness if you need a scratch
> register. You can compute worst-case how many stubs you might need (note
> you just need one per target, not one per call site) and leave yourself
> enough space.
>
>
>
> There may already be some support for this in dyld.  I haven’t needed it
> so I haven’t looked that closely.
>
>
>
> *From:* Joshua Gerrard [mailto:joshua.gerrard at roli.com]
> *Sent:* Monday, November 23, 2015 8:58 AM
> *To:* Andy Ayers <andya at microsoft.com>
> *Cc:* llvm-dev at lists.llvm.org
> *Subject:* Re: [llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow
> when compiling for x86_64
>
>
>
> Thanks Andy, helpful as always!
>
>
>
> 1 is a possibility, but not ideal for us.
>
>
>
> Could you elaborate a little on 3? I don't really know what a jump stub
> is, but am guessing it's a kind of "alternative symbol" which would just
> act as a middle man to invoke the "real" symbol in the static library.
>
> If that's the case, I can think of a way to implement it for specific
> symbols, but not for the more general case.
>
>
> --
> Joshua Gerrard
> JUCE Software Developer
>
> *ROLI’s **award-winning*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.telegraph.co.uk%2fluxury%2fdesign%2f31520%2fthe-seaboard-grand-piano-wins-designs-of-the-year-2014-award.html&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=%2fgxyxsJ5WUDfyHudFXYN1XAy9eKjwz8bnZujH0PuL6Q%3d>* Seaboard
> GRAND, celebrated as the “**piano of the future*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fedition.cnn.com%2f2013%2f09%2f27%2ftech%2finnovation%2fhans-zimmer-seaboard-future-piano%2f&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=bHjzyL%2f86kMolRXlL5GLLyuT5y81SB7bVi%2bsncAU1xA%3d>*”,
> is now joined by the **Seaboard RISE*
> <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.youtube.com%2fwatch%3fv%3dfGr7VbDiRNw&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=sj1TMdJ3rgWyoZUx358B5kqd7VJniAtuHNwC2crO5mE%3d>
> *, “**every bit as slimline and attractive as its bigger brother*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.soundonsound.com%2fnews%3fNewsID%3d18726&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=bENWYQoSd5yz8pCcdcoIBACTgmlIr8Yr3JTL7SLR0aE%3d>*”.
> The press is hailing the Seaboard RISE as “**innovative*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.wired.co.uk%2fnews%2farchive%2f2015-09%2f10%2fseaboard-rise-digital-keyboard-launch-uk-price&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=i9Z%2flXTTQ%2btkiQpWpLwDOQhqZlgq64dVrXEP5c8G804%3d>*”,
> “**expressive*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=6Px83pvi1oWT0sgQ2kGuCkfINmFV4ViyNjboMN7%2fklU%3d>*”,
> “**accessible*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=6Px83pvi1oWT0sgQ2kGuCkfINmFV4ViyNjboMN7%2fklU%3d>*”,
> and “**a keyboard controller that does to piano keys what 3D touch does
> to the iPhone*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.slashgear.com%2froli-seaboard-rise-is-like-3d-touch-for-musicians-11404216%2f&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=CRz4T2P3rw9MYmcoV6T0HVPDzRGaBcjx4u461jZuFtM%3d>*”.
> Now available for preorder at **www.roli.com*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.roli.com%2f&data=01%7c01%7candya%40microsoft.com%7ca21458767056455f1af408d2f42736b5%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=99XH7mWP1JtbRpodnpbra3wUfYze9aWDHkDoOOcGSlo%3d>
> *.*
>
>
>
> On 23 November 2015 at 16:27, Andy Ayers <andya at microsoft.com> wrote:
>
> Microsoft compilers have for quite a while now assumed the code you
> compile is going to be linked into PE images, which are limited to 4GB. So
> they assume a small memory model and use 32 bit relocations. If at link
> time it turns out your export is from a DLL the linker will insert a jump
> stub / dllimport into the image for you which can handle larger distances.
>
>
>
> So you can’t straightforwardly load code from a static CRT library into
> the Dyld-hosted process, since the latter assumes a large memory model.
>
>
>
> Your choices are:
>
>
>
> 1.      Dynamically link whatever you’re compiling against the CRT
> (compile with /MD or /MDd as appropriate)
>
> 2.      I think there has been some work on supporting small memory
> models in Dyld, you could try that out
>
> 3.      Implement a jump stub that is “nearby” the code you’ve compiled
> that can branch to the target (that is, emulate what the linker does)
>
>
>
>
>
> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of *Joshua
> Gerrard via llvm-dev
> *Sent:* Monday, November 23, 2015 3:50 AM
> *To:* llvm-dev <llvm-dev at lists.llvm.org>
> *Subject:* [llvm-dev] COFF::IMAGE_REL_AMD64_REL32 relocation overflow
> when compiling for x86_64
>
>
>
> Some time ago I posted here regarding a relocation overflow on Windows
> (among other things), but the issue disappeared and so the thread got left.
> I've started this new thread because a) I didn't want to necro the old one
> and b) it felt like its own.
>
> I've now encountered the issue again and am noting down all the
> information I can get about it whilst it's happening.
>
>
>
> The issues is that I am getting a relocation overflow assertion inside
> RuntimeDyldCOFFX86_64.h inside the COFF::IMAGE_REL_AMD64_REL32 case.
>
> However, the other thread left me with the impression that I shouldn't be
> getting such relocation when I'm compiling for 64 bit. The only reason I
> can think of for this that I'm not supposed to get 32 bit relocations in
> the code I'm building rather than all the code being loaded.
>
>
>
> The LLVM side of the call stack looks like this:
>
>
>
> _wassert(const wchar_t * expr, const wchar_t * filename, unsigned int
> lineno) Line 369            C
>
> llvm::RuntimeDyldCOFFX86_64::resolveRelocation(const llvm::RelocationEntry
> & RE, unsigned __int64 Value) Line 81         C++
>
> llvm::RuntimeDyldImpl::resolveRelocationList(const
> llvm::SmallVector<llvm::RelocationEntry,64> & Relocs, unsigned __int64
> Value) Line 796            C++
>
> llvm::RuntimeDyldImpl::resolveExternalSymbols() Line 849            C++
>
> llvm::RuntimeDyldImpl::resolveRelocations() Line 95          C++
>
> llvm::RuntimeDyld::resolveRelocations() Line 961    C++
>
> llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::ConcreteLinkedObjectSet<std::shared_ptr<llvm::SectionMemoryManager>,ClangClasses::LLVMExecutionEngine::LinkingResolver
> * __ptr64>::Finalize() Line 112           C++
>
> llvm::orc::ObjectLinkingLayer<llvm::orc::DoNothingOnNotifyLoaded>::findSymbolIn::__l19::<lambda>()
> Line 246            C++
>
> std::_Callable_obj<unsigned __int64 <lambda>(void),0>::_ApplyX<unsigned
> __int64>() Line 284     C++
>
> std::_Func_impl<std::_Callable_obj<unsigned __int64
> <lambda>(void),0>,std::allocator<std::_Func_class<unsigned __int64>
> >,unsigned __int64>::_Do_call() Line 229      C++
>
> std::_Func_class<unsigned __int64>::operator()() Line 316   C++
>
> llvm::orc::JITSymbol::getAddress() Line 62   C++
>
>
>
> RelType is 4 (IMAGE_REL_AMD64_REL32).
>
> Value is 139830239098107.
>
> Addend is 0.
>
>
>
> The symbol that is currently being resolved is _fperrraise. I did some
> researching and it appears that this symbol resides in libcmtd.lib (for me
> the path is C:\Program Files (x86)\Microsoft Visual Studio
> 12.0\VC\lib\amd64\libcmtd.lib).
>
> The relocation type stated in that library (information gathered from
> dumpbin) is REL32.
>
>
>
> I'm not sure what other information there is for me to gather, could
> somebody please help me resolve this?
>
>
>
> Many thanks in advance!
>
> --
> Joshua Gerrard
> JUCE Software Developer
>
> *ROLI’s **award-winning*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.telegraph.co.uk%2fluxury%2fdesign%2f31520%2fthe-seaboard-grand-piano-wins-designs-of-the-year-2014-award.html&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Ykf6luCK%2f2N%2bWIfoJ2xCjeUPQcAvUo70IsWas%2boRido%3d>* Seaboard
> GRAND, celebrated as the “**piano of the future*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fedition.cnn.com%2f2013%2f09%2f27%2ftech%2finnovation%2fhans-zimmer-seaboard-future-piano%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=F3ZwPreXW3KJcEwCzim4YG%2ftGYGrzT8yCS9lQ7pD4Tw%3d>*”,
> is now joined by the **Seaboard RISE*
> <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.youtube.com%2fwatch%3fv%3dfGr7VbDiRNw&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=xaAx43eo5wz73jydxQaa3Lr6%2bvKAp1ui17tqqTELN9M%3d>
> *, “**every bit as slimline and attractive as its bigger brother*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.soundonsound.com%2fnews%3fNewsID%3d18726&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=J8FxKgDY30hPP68pFtrGMa49OeGyGOitZrPxA5BrH8U%3d>*”.
> The press is hailing the Seaboard RISE as “**innovative*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.wired.co.uk%2fnews%2farchive%2f2015-09%2f10%2fseaboard-rise-digital-keyboard-launch-uk-price&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=OKI6eDu1U0AAN4YqwQczE%2fDAxYA3i%2baOL7Vw31v6ueY%3d>*”,
> “**expressive*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iP67j7MpThqiuuKSX5cVbJDHZFKN8KHnICGWLCulVhw%3d>*”,
> “**accessible*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcreatedigitalmusic.com%2f2015%2f09%2fnew-roli-instrument-wants-make-expressive-control-mainstream%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iP67j7MpThqiuuKSX5cVbJDHZFKN8KHnICGWLCulVhw%3d>*”,
> and “**a keyboard controller that does to piano keys what 3D touch does
> to the iPhone*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.slashgear.com%2froli-seaboard-rise-is-like-3d-touch-for-musicians-11404216%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=P9hBMgrTEZupSsqBdX081ZpH1h%2bccISnlZ7vnBp%2bScU%3d>*”.
> Now available for preorder at **www.roli.com*
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.roli.com%2f&data=01%7c01%7candya%40microsoft.com%7cdea4217b5ead441afcf508d2f3fc3084%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=%2bwHEhValdk0plr5GqFNZVsOS9yNPz9n06qH39rjF2DE%3d>
> *.*
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151124/746f7e05/attachment-0001.html>