[llvm-dev] Questions after playing around with KaleidoscopeJIT -Update (With source files)

Tue Oct 1 23:02:43 PDT 2019

Heyho,

I resend this mail, including the attachments to mark the questions that were answered, but also to append a new question. I hope to keep track about the questions - and not to spam...

From: Gaier, Bjoern
Sent: 25 September 2019 11:37
To: llvm-dev <llvm-dev at lists.llvm.org>
Subject: Questions after playing around with KaleidoscopeJIT (With source files)

Hello LLVM people,

after finishing Chapter 1 and 2 of the KaleidoscopeJIT tutorial, I started to play around with the code and now, I have even more questions than before. I hope that the people reading this could help me with it, to improve my understanding about the LLVM and the JIT.
I have also the source code and binaries (Windows) available but because I don't know how the mailing list treats attachments (like .zip) I decided to only attach the source files - hoping it will help >o<

---

1.) Resolving undefined references
I wonder what the best practice is to resolve symbols that are not defined in my llvm::Module.
For that purpose I created "CM_ExternalConstant", that is using "extern const int planschiValue;" and compiled it with Clang. (See CM_ExternalConstant.cpp)

"planschiValue" is not defined in the module, so it will be unreferenced when I jit it. What is the best way to do this? I used 3 different ways:

1.) I went over the llvm::Module before adding it to the layers. I searched for the symbol "?planschiValue@@3HB" and called "replaceAllUsesWith" to replace the symbol with the correct address. (See PlanschbeckenJIT.cpp:31)
2.) I set my own GeneratorFunction and provided addresses for the symbols (See PlanschbeckenJIT.cpp:95)
3.) I can use this->es.getMainJITDylib().define

Each way worked fine. But I felt that way 2.) was the most annoying way, because I had to provide many other addresses. So what way is the best? Or can I 'freely' choose?
Is it possible to have multiple GeneratorFunctions? I would like to use the 'default' DynamicLibrarySearchGenerator plus my own lookup function.
Would it also be possible to replace the uses of "?planschiValue@@3HB" directly with the actuall value of "plaschiValue"? Because it is a constant?

2.) Loading multiple modules
I created "CM_Orc", which has a function "int helloOrc()" - "CM_ExternalConstant" has the same function, but a different implementation. (See CM_Orc.cpp and CM_ExternalConstant.cpp) Adding both modules to the JIT will lead to an error message:
"Failure value returned from cantFail wrapped call
UNREACHABLE executed at c:\program files\llvm\include\llvm\support\error.h:708!"
I always thought that in this situation, one of the functions would be renamed...

3.) Loading multiple modules - controlled resolving
Finally I created "CM_PlanschiValue" this file defines a value for "planschiValue" that was referenced in "CM_ExternalConstant". (See CM_ PlanschiValue.cpp and CM_ExternalConstant.cpp) Surprisingly, if I wrote:
"const int planschiValue = 543210;"
Clang would not generate code for it.
But when writing:
"extern const int planschiValue = 543210;" it worked. Why? Why can Clang optimize away the code, when it could be referenced by a different module? Like it was!

I loaded "CM_ExternalConstant" and "CM_PlanschiValue" together, while using my own GeneratorFunction. But there was no need any more to provide an address for "?planschiValue@@3HB". Obviously, the two modules where linked - which is correct. But what, if I don't want that for whatever reason? Would I have to rename the referenced value in the llvm::Module before adding the second module? Or should I run instead two different JITs?
The idea behind this is... what should I do if I have multiple modules, that are not meant to be used together but should run in the same process.

[Answered] 4.) Clang and JIT optimization
If I compile my modules with Clang and all optimizations turned on - do I still need the TransformLayer in my JIT?

[Answered] 5.) Mangling names
Does Clang or the LLVM provide a function to mangle from the string "const int planschiValue;" to "?planschiValue@@3HB"? Or vice versa?

6.) Delayed resolving
This question goes along with question 1.) - I wondered if there is a way, when emitting code for my module, if I can resolve the undefined values with a "don't know yet value". The idea is to still create code for the module, so that I can already use some functions of it - at a later point I would like to revisit the "don't know values" to resolve them.
Only because function "foo" is missing a certain reference, why shouldn't I be allowed to already use "bar" which is fully resolved? That is the basic idea behind my question.

---

I know that this are a lot questions and that I'm not good in explaining stuff - but I really hope you guys can help me to understand everything better.

Kind greetings and many thanks
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191002/3c163766/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CM_ExternalConstant.cpp
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191002/3c163766/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CM_Orc.cpp
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191002/3c163766/attachment-0001.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CM_PlanschiValue.cpp
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191002/3c163766/attachment-0002.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PlanschbeckenOrcJIT.cpp
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191002/3c163766/attachment-0003.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PlanschbeckenJIT.cpp
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191002/3c163766/attachment-0004.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PlanschbeckenJIT.h
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191002/3c163766/attachment.h>