[llvm-dev] Questions after playing around with KaleidoscopeJIT (With source files)

Thu Sep 26 09:36:07 PDT 2019

Hi Gaier,

(2) - I'd imagine that this is more or less the expected behavior.
Function redefinition like that isn't legal in regular C++ land. If
you'd like to make the second definition shadow the old one though
that's a more interesting problem.

One idea that Patrick Nappa had is to name each function
"functionName_{count}" then keep a map between the function's name,
and its count. When you add a function, increment the count and add it
to the JIT with that name. To look it up, check the map for its most
recent name and look that up.

(4) - AFIK Clang uses the same optimization passes as you might choose
to add to the JIT. There is a really great sackoverflow question that
covers the optimization passes that Clang uses for various
optimization flags here:

https://stackoverflow.com/questions/15548023/clang-optimization-levels

I'd imagine that you'd want to run those passes in the JIT because
that way you can control when things are optimized and likely speed up
compilation, but I'd be interested to see what you end up deciding on.

Also, as a side question, I'd be interested to know how you're using
clang to compile the C++ code into modules. I've been working on a
similar project and have found getting errors out of the parser
properly to be pretty difficult. If you've managed to solve that
problem, I'd love to hear about how you worked it out. Otherwise I
found Cling's incremental parsing code to be pretty helpful:

https://github.com/root-project/cling/blob/master/lib/Interpreter/IncrementalParser.cpp#L741

Good luck,
Zeke

On Wed, Sep 25, 2019 at 2:37 AM Gaier, Bjoern via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hello LLVM people,
>
>
>
> after finishing Chapter 1 and 2 of the KaleidoscopeJIT tutorial, I started to play around with the code and now, I have even more questions than before. I hope that the people reading this could help me with it, to improve my understanding about the LLVM and the JIT.
>
> I have also the source code and binaries (Windows) available but because I don't know how the mailing list treats attachments (like .zip) I decided to only attach the source files - hoping it will help >o<
>
>
>
> ---
>
>
>
> 1.) Resolving undefined references
>
> I wonder what the best practice is to resolve symbols that are not defined in my llvm::Module.
>
> For that purpose I created "CM_ExternalConstant", that is using "extern const int planschiValue;" and compiled it with Clang. (See CM_ExternalConstant.cpp)
>
>
>
> "planschiValue" is not defined in the module, so it will be unreferenced when I jit it. What is the best way to do this? I used 3 different ways:
>
>
>
> 1.) I went over the llvm::Module before adding it to the layers. I searched for the symbol "?planschiValue@@3HB" and called "replaceAllUsesWith" to replace the symbol with the correct address. (See PlanschbeckenJIT.cpp:31)
>
> 2.) I set my own GeneratorFunction and provided addresses for the symbols (See PlanschbeckenJIT.cpp:95)
>
> 3.) I can use this->es.getMainJITDylib().define
>
>
>
> Each way worked fine. But I felt that way 2.) was the most annoying way, because I had to provide many other addresses. So what way is the best? Or can I 'freely' choose?
>
> Is it possible to have multiple GeneratorFunctions? I would like to use the 'default' DynamicLibrarySearchGenerator plus my own lookup function.
>
> Would it also be possible to replace the uses of "?planschiValue@@3HB" directly with the actuall value of "plaschiValue"? Because it is a constant?
>
>
>
> 2.) Loading multiple modules
>
> I created "CM_Orc", which has a function "int helloOrc()" - "CM_ExternalConstant" has the same function, but a different implementation. (See CM_Orc.cpp and CM_ExternalConstant.cpp) Adding both modules to the JIT will lead to an error message:
>
> "Failure value returned from cantFail wrapped call
>
> UNREACHABLE executed at c:\program files\llvm\include\llvm\support\error.h:708!"
>
> I always thought that in this situation, one of the functions would be renamed...
>
>
>
> 3.) Loading multiple modules - controlled resolving
>
> Finally I created "CM_PlanschiValue" this file defines a value for "planschiValue" that was referenced in "CM_ExternalConstant". (See CM_ PlanschiValue.cpp and CM_ExternalConstant.cpp) Surprisingly, if I wrote:
>
> "const int planschiValue = 543210;"
>
> Clang would not generate code for it.
>
> But when writing:
>
> "extern const int planschiValue = 543210;" it worked. Why? Why can Clang optimize away the code, when it could be referenced by a different module? Like it was!
>
>
>
> I loaded "CM_ExternalConstant" and "CM_PlanschiValue" together, while using my own GeneratorFunction. But there was no need any more to provide an address for "?planschiValue@@3HB". Obviously, the two modules where linked - which is correct. But what, if I don't want that for whatever reason? Would I have to rename the referenced value in the llvm::Module before adding the second module? Or should I run instead two different JITs?
>
> The idea behind this is... what should I do if I have multiple modules, that are not meant to be used together but should run in the same process.
>
>
>
> 4.) Clang and JIT optimization
>
> If I compile my modules with Clang and all optimizations turned on - do I still need the TransformLayer in my JIT?
>
>
>
> 5.) Mangling names
>
> Does Clang or the LLVM provide a function to mangle from the string "const int planschiValue;" to "?planschiValue@@3HB"? Or vice versa?
>
>
>
> ---
>
>
>
> I know that this are a lot questions and that I'm not good in explaining stuff - but I really hope you guys can help me to understand everything better.
>
>
>
> Kind greetings and many thanks
>
> Björn
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev