[cfe-users] using Clang for runtime compilation of a class

Sun Oct 4 23:30:27 PDT 2015

Hi Georg, Joel,

Georg - Your question got me curious, so I wrote up a very basic
implementation based on the clang-interpreter demo from clang/examples.
I've tested it on Darwin with LLVM/Clang r249281 and it works, in its very
limited way. I thought I would share it, in case it helps.

It sounds like Joel's project will be an excellent resource for a more
serious approach.

Cheers,
Lang.

On Sun, Oct 4, 2015 at 7:46 AM, Joel Andersson <j.a.e.andersson at gmail.com>
wrote:

> Hi Georg (and others interested in Clang+JIT),
>
> We just implemented this (Clang+JIT) in our open-source project CasADi (
> http://casadi.org). Feel free to have a look and/or reuse components.
>
> We used a plugin-design for using Clang, which is a bit different from the
> examples we could find for using LLVM/Clang. We added a generic "Compiler"
> class and made Clang/JIT a plugin for this Compiler class. Here is the
> implementation of the plugin:
> https://github.com/casadi/casadi/blob/develop/casadi/interfaces/clang/clang_compiler.cpp.
> We also made a second plugin (only for Linux/Mac) which simply calls the
> system compiler using shell commands. Mainly for benchmarking and when
> Clang/JIT isn't available:
> https://github.com/casadi/casadi/blob/develop/casadi/solvers/shell_compiler.cpp
> .
>
> The trickiest was to set up the build system (we use CMake) and in
> particular redistribute the C and C++ standard libraries needed to compile
> any useful C/C++ code. And do it in a way that makes the compiled code
> portable and not relying on e.g. setting environmental variables.
>
> This all works now, also on Windows, although it is not yet completely
> mature (especially when trying to JIT compile C++ code). The solution used
> in the LLVM examples was to use the arguments passed to "main", e.g. to
> locate the "resource", but this didn't work for us since or tool is itself
> a library and we don't have a "main". So the design we chose instead was to
> place the compilation resources (including C/C++ standard library headers)
> in a relative location to the plugin (which is a shared library) since the
> path of the plugin can be retrieved at runtime by "dladdr" in Linux/Mac and
> GetModuleHandleEx+GetModuleFileName on Windows.
>
> Best regards,
> Joel
>
>
> 2015-10-03 20:37 GMT+02:00 David Blaikie via cfe-users <
> cfe-users at lists.llvm.org>:
>
>>
>>
>> On Sat, Oct 3, 2015 at 12:30 AM, Georg Bartels via cfe-users <
>> cfe-users at lists.llvm.org> wrote:
>>
>>> Dear Clang Users,
>>>
>>> I have a particular programming problem, and several discussion forums
>>> suggested Clang as a possible solution. Being totally new to Clang and its
>>> community, is this the right place to ask questions about how to use Clang
>>> for a particular programming task? If not, where should I turn to?
>>>
>>> In case this is the right forum for such questions, I continue
>>> describing my problem. Basically, I want to compile a class which inherits
>>> from a virtual interface class at runtime of my program. The program should
>>> then create an instance of this new class and use it through its interface.
>>>
>>> Let me quickly sketch an example. The static version of my program would
>>> look like this:
>>>
>>> class Base
>>> {
>>>   public:
>>>     virtual int calc() const;
>>> };
>>>
>>> class Derived: public Base
>>> {
>>>   public:
>>>     virtual int calc() const
>>>     {
>>>       return 42;
>>>     }
>>> };
>>>
>>> int main()
>>> {
>>>     std::cout << Derived().calc() << std::endl;
>>>     return 0;
>>> }
>>>
>>>
>>> The dynamic version (with two fuzzy lines of code) shall look like this:
>>>
>>> class Base
>>> {
>>>   public:
>>>     virtual void calc() const;
>>> };
>>>
>>> int main()
>>> {
>>>   std::string code = read_derived_code();
>>>
>>>   // two fuzzy lines for which I am looking for a solution
>>>   SomeCompiledObjectClass o = compile(code);
>>>   Base* d = SomeFactoryClass(o).createInstance();
>>>
>>>   std::cout << d->calc() << std::endl;
>>>   return 0;
>>> }
>>>
>>> Is Clang the right tool for solving my two fuzzy lines of code? If yes,
>>> which class should have a look at? If not, do you maybe know about another
>>> library/compile which can achieve this task?
>>>
>>
>> This migth be the sort of thing that would escalate to cfe-dev and/or
>> llvm-dev.
>>
>> What you're looking to do is JIT (Just In Time) compile some code. That
>> involves using clang to turn the source code into LLVM IR and then LLVM to
>> make machine code from that, put it in memory, make it executable, and call
>> some part of it.
>>
>> Your two fuzzy lines of code would look slightly different (chances are
>> what you'd do is generate source code that included a private function that
>> return an instance, then you would JIT compile that function, indirectly
>> code generating anything in the class that was needed, etc).
>>
>> I'm not sure where the best examples of the Clang half of this problem
>> are. But for the second have, LLVM has the Kaleidoscope tutorials
>> (especially the recently added ORC examples) that show how to use LLVM as a
>> JIT compiler. You shuold be able to wrap up all these APIs into a fairly
>> simple API for your one specific use case.
>>
>> I think /maybe/ out there is an existing C++ rapid development
>> environment using Clang (essentially a C++ interpreter/interactive command
>> prompt like Python) that you might be able to grab the basics of "feed code
>> to compiler, get IR" from.
>>
>> - David
>>
>>
>>>
>>> Thanks a lot for your help!
>>>
>>> Cheers,
>>> Georg.
>>>
>>> _______________________________________________
>>> cfe-users mailing list
>>> cfe-users at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
>>>
>>
>>
>> _______________________________________________
>> cfe-users mailing list
>> cfe-users at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
>>
>>
>
>
> --
> --
> Joel Andersson, PhD
> Ptge. Busquets 11-13, atico 3
> E-08940 Cornella de Llobregat (Barcelona), Spain
> Home: +34-93-6034011
> Mobile: +34-63-4408800 (in Sweden also +46-707-360512)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-users/attachments/20151004/7b665e28/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jit-derived.tgz
Type: application/x-gzip
Size: 3831 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-users/attachments/20151004/7b665e28/attachment.bin>