[llvm-dev] C++ JIT Compiler with LLVM on Windows 10 - part 5

Emmanuel Roche via llvm-dev llvm-dev at lists.llvm.org
Tue May 5 23:20:10 PDT 2020 

Hello Lang,

Good work! That was very quick ;-)

> I will file a bug tomorrow to add COMDAT support to ORC. I won't have
time to work on this feature any time soon, but I will try to provide a
sketch of the solution in the bug report in case you or anyone else is
interested in taking up the challenge. :)

=> I suspect this could be an issue that could get in my way more
significantly than I initially thought, so yes, if you could provide some
details in the bug report that would be great indeed, and I might
eventually find enough motivation and energy to try to handle it myself.

=> Please let me know when you have a link to that new bug report
available, and meanwhile thank you very much for your time and efforts on
this issue!

Have a great day!

Cheers,
Emmanuel.

Le mer. 6 mai 2020 à 02:32, Lang Hames <lhames at gmail.com> a écrit :

> Hi Emmanuel,
>
> I see the problem:
>
> $"??_7exception at std@@6B@" = comdat largest
>
> The JIT knows how to honor linkages, but it does not know about
> comdats yet except in limited circumstances, so it sees these as
> conflicting definitions.
>
> Adding support for comdats will be non-trivial as their selection rules
> are more complex than the selection rules for linkage types. For now I
> would stick to using llvm-link. I will file a bug tomorrow to add COMDAT
> support to ORC. I won't have time to work on this feature any time soon,
> but I will try to provide a sketch of the solution in the bug report in
> case you or anyone else is interested in taking up the challenge. :)
>
> Cheers,
> Lang.
>
> On Tue, May 5, 2020 at 11:12 AM Lang Hames <lhames at gmail.com> wrote:
>
>> Hi Emmanuel,
>>
>> Thank you very much for these! I will check them out today and see what's
>> going on.
>>
>> Regards,
>> Lang.
>>
>> On Mon, Apr 27, 2020 at 8:35 PM Emmanuel Roche <roche.emmanuel at gmail.com>
>> wrote:
>>
>>> Hi Lang,
>>>
>>> Thank you for your feedback on the blog post, please find below some
>>> additional inputs from my side on the comments you provided:
>>>
>>> > Regarding emulated TLS deactivation: I've never looking into
>>> how/whether this works on Windows. If it doesn't make sense to have it on
>>> by default there we can change the default for Windows targets.
>>>
>>> I'm really no expert, so you probably want to collect additional
>>> inputs/checks on this point, but yes, from my current perspective, it seems
>>> that it doesn't really work to try to use Emulated TLS in LLJIT on Windows,
>>> but one should keep in mind that this is when using the microsoft crt from
>>> Visual Studio (FYI, I'm using version 2017 for now, maybe it was different
>>> in previous versions too): I would expect things to be different if you are
>>> using another compiler as base (?) [I mean, if you change the default, this
>>> might then break previously working code that was based on mingw for
>>> instance... But not sure if this should be a concern or not for a not yet
>>> released version of LLVM ;-)]
>>>
>>> > I think that anything that llvm-link can merge should, in theory, be
>>> safe to add to the JIT. This actually sounds like a bug. Are you able to
>>> share the full modules that you were merging?
>>>
>>> Initially, the test I made on this point was using the Lest test
>>> framework as described in my post, but your feedback above got me thinking,
>>> and I could actually come up with a more simple setup without any C++
>>> library dependency: this new test is simply based on a couple of C++
>>> functions that are using exceptions. I used the attached lua script
>>> "exception3.lua" to generate the bitcode files and load the corresponding
>>> modules [of course I don't expect you to run lua scripts, but I thought
>>> that this could help understanding the code generation context anyway]. So
>>> with that script I generated the
>>> test_func1.bc/test_func2.bc/test_func1_and_func2.bc and the corresponding
>>> .ll file (generated from the .bc files using llvm-dis): all those files are
>>> attached to this email too.
>>>
>>> => So, I can load for instance the module corresponding to test_func1.bc
>>> in my main JITDylib, but then, if I try to load the test_func2 IR module in
>>> the same Dylib, I get the following error:
>>>
>>> *Duplicate definition of symbol '??_7exception at std@@6B@'*
>>>
>>> In all the .ll files you will find multiple references of that symbol...
>>> but unfortunately, my skills won't get me any further than that [=> ie. I
>>> have no idea what to think about the content of those files :-)].
>>>
>>> But anyway, if you notice anything interesting yourself, or you have an
>>> idea you think I should try, or you need more info on anything here please
>>> let me know.
>>>
>>> PS: I'm thinking maybe the command line arguments I'm providing to the
>>> compilerInvocation might also be important here so, just in case: here is
>>> the list of args I'm using currently:
>>>
>>> {    "-triple=x86_64-pc-windows-msvc19.16.27030",
>>>     "-x", "c++",
>>>     "-mrelax-all",
>>>     "-mincremental-linker-compatible",
>>>     "-disable-free",
>>>     "-discard-value-names",
>>>     "-mrelocation-model", "pic",
>>>     "-pic-level", "2",
>>>     "-mthread-model", "posix",
>>>     "-mframe-pointer=none",
>>>     "-relaxed-aliasing",
>>>     "-fmath-errno",
>>>     "-fno-rounding-math",
>>>     "-mconstructor-aliases",
>>>     "-munwind-tables",
>>>     "-target-cpu", "x86-64",
>>>     "-mllvm",
>>>     "-x86-asm-syntax=intel",
>>>     "-stack-protector","2",
>>>     "-fcxx-exceptions",
>>>     "-fexceptions",
>>>     "-fexternc-nounwind",
>>>     "-fms-volatile",
>>>     "-fdiagnostics-format", "msvc",
>>>     "-dwarf-column-info",
>>>     "-Wall",
>>>     "-std=c++17",
>>>     "-fdeprecated-macro",
>>>     "-ferror-limit","19",
>>>     "-fmessage-length=174",
>>>     "-fms-extensions",
>>>     "-fms-compatibility",
>>>     "-fms-compatibility-version=19.16.27030",
>>>     "-fdelayed-template-parsing",
>>>     "-fcolor-diagnostics",
>>>     "-fobjc-runtime=gcc",
>>>     "-faddrsig",
>>>   }
>>>
>>> Regards,
>>> Manu.
>>>
>>>
>>> Le lun. 27 avr. 2020 à 20:41, Lang Hames <lhames at gmail.com> a écrit :
>>>
>>>> I was just reading the blog post -- very cool!
>>>>
>>>> Regarding Globals construction & destruction: There definitely has been
>>>> a lot of churn in that area. There will probably be more before  LLVM 11 is
>>>> released, but I can see light at the end of the tunnel. I think the Right
>>>> Way to run initializers in a JITDylib is to treat it as equivalent to a
>>>> dlopen operation (with extra allowances for the fact that new initializers
>>>> can be added to a JITDylib at runtime). This is what the LLJIT::initialize
>>>> method is doing. Now we just need to generalize it to support
>>>> out-of-process JITs. That will at least require a new remote-target
>>>> abstraction, and an RPC implementation for testing in-tree.
>>>>
>>>> Regarding emulated TLS deactivation: I've never looking into
>>>> how/whether this works on Windows. If it doesn't make sense to have it on
>>>> by default there we can change the default for Windows targets.
>>>>
>>>> Regarding merging of multiple modules:
>>>>
>>>> "But of course this would not work because as soon as I try to load the
>>>> second script, I get a duplicate symbol error from LLVM (and this
>>>> completely makes sense):
>>>>
>>>> [ERROR]: LLVM error: Duplicate definition of symbol '??_7success at lest
>>>> @@6B@'"
>>>>
>>>> I think that anything that llvm-link can merge should, in theory, be
>>>> safe to add to the JIT. This actually sounds like a bug. Are you able to
>>>> share the full modules that you were merging?
>>>>
>>>> Regards,
>>>> Lang.
>>>>
>>>>
>>>> On Mon, Apr 27, 2020 at 11:11 AM David Blaikie <dblaikie at gmail.com>
>>>> wrote:
>>>>
>>>>> +Lang for LLVM Orc things
>>>>>
>>>>> On Mon, Apr 27, 2020 at 3:34 AM Emmanuel Roche via llvm-dev <
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> Hello LLVM developers!
>>>>>>
>>>>>> So, I'm continuing my journey with my toy C++ JIT compiler
>>>>>> implementation, and I wrote another article on the issues/solutions I've
>>>>>> been working on in the past few days, mainly:
>>>>>>
>>>>>> - Precompiled header handling,
>>>>>> - Emulated TLS desactivation,
>>>>>> - Globals construction & destruction,
>>>>>> - C++ exceptions handling,
>>>>>> - Multi modules linking,
>>>>>>
>>>>>> => In case this could be helpful to anyone, you will find this
>>>>>> article here:
>>>>>> http://wiki.nervtech.org/doku.php?id=blog:2020:0425_jit_compiler_part5_improvements
>>>>>>
>>>>>> And of course, if you have any questions for me, just let me know.
>>>>>>
>>>>>> Meanwhile, happy hacking everyone ;-)!
>>>>>>
>>>>>> Best regards,
>>>>>> Manu.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> llvm-dev at lists.llvm.org
>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200506/0988540d/attachment.html>

More information about the llvm-dev mailing list