[llvm-dev] Get all symbols stored(?)in llvm::orc::ExecutionSession

Sat Aug 22 10:38:47 PDT 2020

Hi Bjoern,

> Symbol names shouldn't change from the moment they're added to the JIT
> (i.e. after being fully mangled). Consistency of symbol names is a
> requirement for ORC to work.
> Right, what I meant was, that some of the symbols I take from the modules
> might disappear… I don’t remember the details anymore, but I think I
> encountered the symbol for an intrinsic version of memcpy but after the
> compilation it became a call to memcpy.
> Does that explain what I mean?

Oh -- You only want to look at definitions, not declarations. It's totally
fine if your code adds or removes *uses* of other symbols -- that only
changes the lookups that the JIT linker will need to perform at the end of
the process. What's really important is that the definitions do not change.

That said: *Sometimes* the compiler does add or remove definitions. E.g.
COFF COMDATS for floating point constants (see
http://lists.llvm.org/pipermail/llvm-dev/2020-January/138584.html). These
cases are rare, but if you hit one please let me know: We probably need to
teach ORC to recognize it.

So I would use the function “setPlatform” of the ”ExecutionSession”? But
> wouldn’t I override something really important with that?

It depends on whether you're using LLJIT or a custom JIT class.
ExecutionSession doesn't set a Platform by default so you're free to
(ab)use the Platform when writing a custom JIT class. LLJIT *does* set up a
Platform, but we could look at moving that into a header so that you could
extend it with your own behavior.

-- Lang.

On Wed, Aug 19, 2020 at 4:29 AM Gaier, Bjoern <Bjoern.Gaier at horiba.com>
wrote:

> Hey Lang,
>
>
>
> > Symbol names shouldn't change from the moment they're added to the JIT
> (i.e. after being fully mangled). Consistency of symbol names is a
> requirement for ORC to work.
>
> Right, what I meant was, that some of the symbols I take from the modules
> might disappear… I don’t remember the details anymore, but I think I
> encountered the symbol for an intrinsic version of memcpy but after the
> compilation it became a call to memcpy.
>
> Does that explain what I mean?
>
>
>
> > You could try hooking into the orc::Platform API for this:
> https://github.com/llvm/llvm-project/blob/ec29538af2e0886a65f479d6a533956a1c478132/llvm/include/llvm/ExecutionEngine/Orc/Core.h#L1062
> <https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fllvm%2fllvm%2dproject%2fblob%2fec29538af2e0886a65f479d6a533956a1c478132%2fllvm%2finclude%2fllvm%2fExecutionEngine%2fOrc%2fCore.h%23L1062&umid=a9f2d13d-7fe0-42bd-b679-39bf94f23b19&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-0818ddd39fac560390288eccb348f4aef06ae077>
>
> So I would use the function “setPlatform” of the ”ExecutionSession”? But
> wouldn’t I override something really important with that?
>
>
>
> Thank you so far Lang :D
>
>
>
> Kind greetings
>
> Björn
>
>
>
>
>
> *From:* Lang Hames <lhames at gmail.com>
> *Sent:* 18 August 2020 18:05
> *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com>
> *Cc:* LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
> *Subject:* Re: [llvm-dev] Get all symbols stored(?)in
> llvm::orc::ExecutionSession
>
>
>
> Hi Bjoern,
>
>
>
> I’m a bit worried about taking symbols from modules, because some of the
> symbols might change later when the code was compiled with the JIT
>
>
>
> Symbol names shouldn't change from the moment they're added to the JIT
> (i.e. after being fully mangled). Consistency of symbol names is a
> requirement for ORC to work.
>
>
>
> You could try hooking into the orc::Platform API for this:
> https://github.com/llvm/llvm-project/blob/ec29538af2e0886a65f479d6a533956a1c478132/llvm/include/llvm/ExecutionEngine/Orc/Core.h#L1062
> <https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fllvm%2fllvm%2dproject%2fblob%2fec29538af2e0886a65f479d6a533956a1c478132%2fllvm%2finclude%2fllvm%2fExecutionEngine%2fOrc%2fCore.h%23L1062&umid=a9f2d13d-7fe0-42bd-b679-39bf94f23b19&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-0818ddd39fac560390288eccb348f4aef06ae077>
>
>
>
> The notifyAdding method will be called for each MaterializationUnit added
> to the JIT, which gives you an opportunity to iterate over the symbols that
> it provides and record them.
>
>
>
> Regards,
>
> Lang.
>
>
>
>
>
>
>
>
>
> On Mon, Aug 17, 2020 at 11:16 PM Gaier, Bjoern <Bjoern.Gaier at horiba.com>
> wrote:
>
> Hey Lang,
>
>
>
> Oh shoot… seems like I oversaw it… Sorry sorry sorry! (I was 5 weeks away
> but I thought I checked all mails correctly >o<)
>
>
>
> I think ORCs assumption is correct, currently I do extract the symbols
> which I care about via the modules, but when writing the code I was like
> “why do I have to do it like that, when the ExecutionSession already knows
> all symbols?” – that is where my question came from.
>
> However… I’m a bit worried about taking symbols from modules, because some
> of the symbols might change later when the code was compiled with the JIT.
> So I felt like that the ExecutionSession would been more reliable.
>
>
>
> But if I reaaaally want such a function, I could basically have a look
> into the dump function… Cause I think my motivation is not convincing xD
>
>
>
> Thank you Lang!
>
>
>
> Kind greetings
>
> Björn
>
>
>
>
>
> *From:* Lang Hames <lhames at gmail.com>
> *Sent:* 17 August 2020 18:40
> *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com>
> *Cc:* LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
> *Subject:* Re: [llvm-dev] Get all symbols stored(?)in
> llvm::orc::ExecutionSession
>
>
>
> Hi Bjoern,
>
>
>
> Did you see my previous reply?
>
>
>
> There's no way to do this currently. ORC assumes you know all the symbols,
> since you added the modules defining them.
> For testing / debugging you can dump the modules to stderr using
> ExecutionSession::dump, but that's about it.
> Do you want the symbols for diagnostic purposes, or some other reason?
>
>
>
> Regards,
>
> Lang.
>
>
>
> On Mon, Aug 17, 2020 at 5:19 AM Gaier, Bjoern via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> This issue is still not solved for me…
>
> What can I improve on my question? :o
>
> Adding more details or something? >_x
>
> Kind greetings
>
> Björn
>
> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> *On Behalf Of *Gaier,
> Bjoern via llvm-dev
> *Sent:* 03 July 2020 13:51
> *To:* LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
> *Subject:* [llvm-dev] Get all symbols stored(?)in
> llvm::orc::ExecutionSession
>
>
>
> Hey everyone,
>
>
>
> is there a way to get the name of all symbols that are stored (“stored” is
> not the right term – is it?) in my current ExecutionSession?
>
> I know by now that you can use "lookup" to get symbols, but this requires
> knowing those names.
>
>
>
> Mhh... I guess that is my shortest so far question xD Thank you in advance
> for any help!
>
>
>
> Kind greetings
>
> Björn
>
>
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
> Cansiz.
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
> Cansiz.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
> Cansiz.
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
> Cansiz.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200822/e1873694/attachment.html>