[llvm-dev] Clang/LLVM JIT - When to use "registerEHFrames()"
Stefan Gränitz via llvm-dev
llvm-dev at lists.llvm.org
Sun Oct 1 07:16:59 PDT 2017
Hi, I checked last year's mails. Back then Timur and me faced the same
issue Björn reports (nulled memory in _ThrowInfo).
> Back then I tried to solve a related issue: SEH exceptions thrown
> from code loaded with RuntimeDyld had to be caught in statically
> compiled code. It turned out Windows explicitly prohibits this. I
> got in touch with Microsoft people and IIRC it's due to security
> concerns.
>
>
> Stefan -- That's an interesting restriction. :/
> Does it prohibit exceptions thrown in JIT'd code from being caught
> also in JIT'd code, or does it only apply if an exception crosses the
> boundary back into statically compiled code?
> Do you know how they were enforcing that?
Igor Minin explained this was due to Data Execution Prevention (DEP) and
gave pointers to the involved functions in the OS runtime:
http://lists.llvm.org/pipermail/llvm-dev/2016-June/100901.html
The whole thing gets hairy quickly. It seems like a combination of
multiple problems. So let's first collect evidence and clarify the goal.
Joseph Tremoulet from Microsoft partially confirmed Igor's statements:
"I checked with some experts here, and they agree with your/Igor’s
assessment that it’s not possible to have a dynamically generated
handler on x86"
"the llilc code only handles x86_64, not 32-bit x86; I know the
implementations of WinEH are quite different between the two"
First conclusion: No solution for JITed catch handlers on x86.
I think for most of us x86 has little prio anyway, so let's focus on x86_64.
In theory both, JITed and static catch handlers, may be possible here.
I disabled DEP globally on my machine and set MEM_EXECUTE_OPTION_ENABLE
explicitly:
http://www.thewindowsclub.com/disable-data-execution-prevention
https://github.com/Sh1ft0x0EF/metahook/blob/master/sys_launcher.cpp#L65
It didn't fix anything.
Second conclusion: DEP may be one reason, but there must be something else.
My next guess was an incompatibility between Clang (we used for JITing)
and MSVC (we used for static compilation). Joseph:
"One thing that comes to mind is that presumably you’re not trying to
make your language/runtime’s exceptions masquerade as MSVC++ exceptions,
so I guess you’d want your static native code to either have an SEH
__try/__except or compile with clang’s equivalents of MSVC’s /EHa and
catch(…)"
I tried all the combinations without success. I also compared the
compiled code between Clang 3.8 and MSVC, which showed no reasonable
difference.
Second conclusion: No issue with code generation.
At that point we ran out of time we could spend on this. So we postponed
the feature, hoping for Microsoft to come up with a solution at some
point. Giving it another try today, I think this looks promising:
https://github.com/dotnet/llilc/blob/master/lib/Jit/EEMemoryManager.cpp#L250
Processing .pdata section entries, it creates and allocates unwind
blocks and funclets (handlers for catch & finally) for CLR managed
exceptions. It's probably not directly applicable tonative exception
handling, but it may provide useful information. Also there is much more
documentation available now:
https://github.com/dotnet/coreclr/blob/master/Documentation/botr/clr-abi.md#exception-handling
https://llvm.org/docs/ExceptionHandling.html#exception-handling-using-the-windows-runtime
I could imagine setting up minimal tests would be a useful first step to
distinguish issues in a reproducible way.
Best
Stefan
Am 29.09.17 um 19:43 schrieb Lang Hames:
> Hi Bjoern,
>
> I'm trying to make exceptions run. I have an Object file with a
> function, throwing a 1 and a second function which should catch
> the 1. Normal JITTING under Windows showed me, that I have an
> unresolved reference to the virtual table of type_info. Some
> experiments later I was able to load "msvcrt.lib" as an archive
> and could resolve the reference. Nice - but than
> "??_Etype_info@@UEAPEAXI at Z" was missing too. Ufff! I decided to
> ignore this. Because, when I try to load every .lib and .obj
> provided by Visual Studio, I get an assertion failure with
> "Relocation type not implemented yet!".
>
>
> RuntimeDyldCOFF is missing a lot of relocation support. We need a COFF
> expert to fix that and unfortunately I'm not one.
>
> I decided to have a look at "_CxxThrowException". I inserted my
> own function for the JIT and had a look at the parameters. I got
> two of them. The first was the address of the Exception-Object,
> which was correct. The second is the address for the "_ThrowInfo".
> This address was valid too, but all its members - except from
> attributes - are null. So I can't throw this Exception. I tried to
> pass the address of typeid(1) to it, or modificate the call.
> Nothing helped.
>
>
> I don't know windows exception handling well, but if it's anything
> like DWARF EH then I'd be inclined to blame the missing
> relocations/fixups -- the _ThrowInfo struct (or whatever data-source
> ultimately populates it) probably isn't being fixed up.
>
> I have no clue and no idea anymore. So... Do you have an idea?
>
>
> I'm afraid I don't personally. We need some windows linker / system
> experts to take an interest in the JIT.
>
> Back then I tried to solve a related issue: SEH exceptions thrown
> from code loaded with RuntimeDyld had to be caught in statically
> compiled code. It turned out Windows explicitly prohibits this. I
> got in touch with Microsoft people and IIRC it's due to security
> concerns.
>
>
> Stefan -- That's an interesting restriction. :/
> Does it prohibit exceptions thrown in JIT'd code from being caught
> also in JIT'd code, or does it only apply if an exception crosses the
> boundary back into statically compiled code?
> Do you know how they were enforcing that?
>
> Cheers,
> Lang.
>
>
>
>
>
> On Thu, Sep 28, 2017 at 12:37 PM, Stefan Gränitz via llvm-dev
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>
>> I tried loading the "msvcrt.lib" as a archive. That was... a bad
>> idea! I get a Exception while loading:
>> Assertion failed: ((int64_t)Result <= INT32_MAX) && "Relocation
>> overflow", file
>> \lib\executionengine\runtimedyld\Targets/RuntimeDyldCOFFX86_64.h,
>> line 81
> It's a limitation of the COFF/PE format and unrelated to
> exceptions. This patch explains it and shows a workaround:
> https://github.com/weliveindetail/pj-llvm/commit/97cd336d458ae9c73232d1b539ceefcdb2f5eb0f
> <https://github.com/weliveindetail/pj-llvm/commit/97cd336d458ae9c73232d1b539ceefcdb2f5eb0f>
>
>> Is there no hope left?
> Well at least I am not aware of a solution.
>
> Am 28.09.17 um 16:04 schrieb bjoern.gaier at horiba.com
> <mailto:bjoern.gaier at horiba.com>:
>> Hello Stefan,
>>
>> I'm happy someone replied to my problem! Many thanks! To be
>> honest... I didn't understood much of your mail. I'm a beginner
>> with the JIT - so I will explain what I've done.
>>
>> To manage the memory and resolve symbols, I'm using my own
>> Resolver-Class, which overloads the allocation and the findSymbol
>> functions. I've noticed today, that the "registerEHFrames"
>> function of my class gets called automatically, with correct
>> values. I'm remapping my code and the address are still correct.
>> Great! But, what should I do with it? I pass the values to the
>> original function, but my exception won't be caught! It's an
>> exception raised inside the JITTED code and should also catched
>> there.
>>
>> I tried loading the "msvcrt.lib" as a archive. That was... a bad
>> idea! I get a Exception while loading:
>> Assertion failed: ((int64_t)Result <= INT32_MAX) && "Relocation
>> overflow", file
>> \lib\executionengine\runtimedyld\Targets/RuntimeDyldCOFFX86_64.h,
>> line 81
>>
>> Research didn't helped me! My code was compiled with /MD, but it
>> didn't changed. So I'm still stupid D:
>> The JITTED code must be loaded to shared memory later - there
>> aren't libraries, so even if this would work, it wouldn't help
>> me. I tried compiling my code with sjlj-exceptions. Didn't worked...
>>
>> Is there no hope left?
>>
>> Kind regards
>> Björn
>>
>>
>>
>> From: Stefan Gränitz <stefan.graenitz at gmail.com>
>> <mailto:stefan.graenitz at gmail.com>
>> To: bjoern.gaier at horiba.com
>> <mailto:bjoern.gaier at horiba.com>, llvm-dev at lists.llvm.org
>> <mailto:llvm-dev at lists.llvm.org>
>> Date: 27.09.2017 23:09
>> Subject: Re: [llvm-dev] Clang/LLVM JIT - When to use
>> "registerEHFrames()"
>> ------------------------------------------------------------------------
>>
>>
>>
>> Hi Björn
>>
>> To first answer your questionin the subject: For x86
>> registerEHFrames() is only a stub. For x86_64 registerEHFrames()
>> is implemented properly in RuntimeDyldCOFFX86_64, calling
>> MemMgr.registerEHFrames() for each EH frame section. It should be
>> called and work out of the box without your involvement, but
>> unfortunately it won't solve your issue. All the essential
>> information is there in the comments, just check the base classes.
>>
>> This thread from last year helps with your unresolved symbol:_
>> __http://lists.llvm.org/pipermail/llvm-dev/2016-October/106458.html_
>> <http://lists.llvm.org/pipermail/llvm-dev/2016-October/106458.html>
>>
>> Back then I tried to solve a related issue: SEH exceptions thrown
>> from code loaded with RuntimeDyld had to be caught in statically
>> compiled code. It turned out Windows explicitly prohibits this. I
>> got in touch with Microsoft people and IIRC it's due to security
>> concerns.
>>
>> Depending on your specific case, you may want to fall back to
>> vectored exception handling. In my experience this was a dead end
>> though. If you need a solution for arbitrary situations, you just
>> can't jump back to a "safe" place to continue execution. I tried
>> setjump (on each entry point to the dynamically loaded code) /
>> longjmp (in the vectored exception handler), but the address was
>> invalidated when I accessed it. I suspect it's kind of undefined
>> behavior to call longjmp outside a child frame of the function
>> that called setjmp. Anyway it turned all far too hacky.
>>
>> If you are willing to do research, compare implementations and
>> behavior with the MachO and ELF versions. At least one of them
>> works, just not on Windows ;)
>> Also check the LLILC project: _https://github.com/dotnet/llilc_
>> <https://github.com/dotnet/llilc>I heard about some solution that
>> uses trampolines to push exceptions back to dynamically loaded
>> code and handle them there.
>>
>> AND disclaimer: I did not follow recent developments in this
>> area. If there's news please let me know!
>>
>> Cheers & Good Luck!
>> Stefan
>>
>> Am 25.09.17 um 11:31 schrieb via llvm-dev:
>> Hello friendly LLVM-World,
>>
>> because I don't know if I had send my problem to the correct
>> Mailing-List, I will send my problem to this address too. I'm not
>> subscribed to this list, so please add my in CC if you response.
>>
>> Kind regards
>> Björn
>>
>>
>> From: Bjoern Gaier/HE/HORIBA
>> To: Clang Dev _<cfe-dev at lists.llvm.org>_
>> <mailto:cfe-dev at lists.llvm.org>, "cfe-dev"
>> _<cfe-dev-bounces at lists.llvm.org>_
>> <mailto:cfe-dev-bounces at lists.llvm.org>
>> Date: 19.09.2017 08:05
>> Subject: Clang/LLVM JIT - When to use "registerEHFrames()"
>>
>> ------------------------------------------------------------------------
>>
>>
>> Hello friendly Clang-World,
>>
>> I was experimenting with Clang and the JIT capabilities of LLVM.
>> Most of my attempts were successfully but, I still fail miserably
>> at exceptions. Doing research I found the function
>> "registerEHFrames()" which should assist me supporting exceptions
>> - but sadly the documentation I found wasn't helpful.
>> I looked at into the "notifyObjectLoaded" function and discovered
>> that there appear some symbol names starting with "$" - I
>> expected them to be connected to my try and catch block. But what
>> now? As usually, at this point I have there names, but can't get
>> there address to register them with the "registerEHFrames()"
>> function. Also the JITTER still wants an address for
>> "??_7type_info@@6B@" which is the virtual table of the type_info
>> struct.
>>
>> Confusing! So friendly Clang-World, could you please help?
>>
>> Not so important - but has the dragon which decorates clang and
>> LLVM a name?
>>
>> Kind regards
>> Björn
>>
>> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB
>> 9816, USt.ID-Nr. DE 114 165 789
>> Geschäftsführer: Hiroshi Kawamura, Dr Hiroshi Nakamura, Markus
>> Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> _llvm-dev at lists.llvm.org_ <mailto:llvm-dev at lists.llvm.org>
>> _http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev_
>> <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>>
>> --
>> _https://weliveindetail.github.io/blog/_
>> <https://weliveindetail.github.io/blog/>
>> _https://cryptup.org/pub/stefan.graenitz@gmail.com_
>> <https://cryptup.org/pub/stefan.graenitz@gmail.com>
>>
>>
>> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB
>> 9816, USt.ID-Nr. DE 114 165 789
>> Geschäftsführer: Hiroshi Kawamura, Dr Hiroshi Nakamura, Markus
>> Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima.
>
> --
> https://weliveindetail.github.io/blog/
> <https://weliveindetail.github.io/blog/>
> https://cryptup.org/pub/stefan.graenitz@gmail.com
> <https://cryptup.org/pub/stefan.graenitz@gmail.com>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>
>
--
https://weliveindetail.github.io/blog/
https://cryptup.org/pub/stefan.graenitz@gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171001/b35b888d/attachment.html>
More information about the llvm-dev
mailing list