[LLVMdev] Selectively Jitting using MCJIT

[Jonatan Waern]

Back again with more issues...

Loading the library file and running it works great on a simple test  
file I made (containg just a simple function, printing some result).

I am now trying to get JIT to intercept failed symbol lookups when  
running the library and run the lazyFunctionGenerator I installed to  
generate those functions. This did not work naively by obtaining the  
external entry point using a call to SearchForAddressOfSymbol and then  
running it as you would a regular function pointer. The function gets  
called fine, but when it calls an extern (unimplemented) function it  
just obtains a symbol lookup error, rather than callbacking into the  
lazyFunctionGenerator.

I also tried to "wrap" the execution of the library inside the JIT by  
constructing an entry point in-memory using an IR builder. The entry  
point was a simple function that had the regular "main" signature and  
called the external function, passing along its arguments. This  
however promptly results in a segfault on that instruction. I don't  
know why this is, and I have not yet found a way to debug in-memory  
generated IR that has no source file.

Quoting Willy WOLFF <willy.wolff at etu.unistra.fr>:

>
> On 24 May 2014, at 00:46, Jonatan Waern wrote:
>
>>> As far as I know, ObjectFile is either COFF, ELF or MachO format,  
>>> statically compiled. But the pass manager for applying  
>>> optimisation passes works only on llvm-ir.
>> I would not need to apply optimisations on the parts of the program  
>> in the object file. These are meant to be optimised beforehand. The  
>> only parts I want to run passes on are specific function that I  
>> know beforehand.
>>
>
> Then a simple dlopen (with RTLD_LAZY) /dlsym/dlclose is not sufficient?
>
>>> What you can do is to isolate the function you want to jit by  
>>> extracting it to a new module. Load this module during runtime,  
>>> apply your pass and jitting it.
>> Yes this is exactly what I want to do. The interesting part is to  
>> load it during runtime. It is not sufficient for me to load it  
>> during JIT initialisation, but rather I want to load and JIT it  
>> during program execution. Hence my idea of declaring the function  
>> in the module as extern and load it through the lazy function  
>> creator.
>>
>> Quoting Willy WOLFF <willy.wolff at etu.unistra.fr>:
>>
>>>
>>> On 23 May 2014, at 14:55, Jonatan Waern wrote:
>>>
>>>> Hello, I am a novice at using the llvm api to do much of anything.
>>>>
>>>> I am trying to lazily apply certain optimisation passes on select  
>>>> functions when jitting a large program. It is undesirable to me  
>>>> to load the entire program as IR code and then generate code in  
>>>> memory for it (time concerns). It seems that there is a function  
>>>> in MCJIT "loadObjectFile" that would suit my purposes, allowing  
>>>> me to statically compile the program and then load it into the jit.
>>>>
>>>
>>> As far as I know, ObjectFile is either COFF, ELF or MachO format,  
>>> statically compiled. But the pass manager for applying  
>>> optimisation passes works only on llvm-ir.
>>>
>>>> My question is then what would be a good way to callback into the  
>>>> jit for applying the optimisation passes before calling one of  
>>>> the function? My current idea is to make the function extern in  
>>>> the llvm IR prior to static compile, and then use  
>>>> "installLazyFunctionCreator" to generate this function by loading  
>>>> ircode from an isolated module. Is this a feasible approach?
>>>>
>>>
>>> What you can do is to isolate the function you want to jit by  
>>> extracting it to a new module. Load this module during runtime,  
>>> apply your pass and jitting it.
>>> --
>>> Willy WOLFF
>
> --
> Willy WOLFF



Quoting Willy WOLFF <willy.wolff at etu.unistra.fr>:

>
> On 24 May 2014, at 00:46, Jonatan Waern wrote:
>
>>> As far as I know, ObjectFile is either COFF, ELF or MachO format,  
>>> statically compiled. But the pass manager for applying  
>>> optimisation passes works only on llvm-ir.
>> I would not need to apply optimisations on the parts of the program  
>> in the object file. These are meant to be optimised beforehand. The  
>> only parts I want to run passes on are specific function that I  
>> know beforehand.
>>
>
> Then a simple dlopen (with RTLD_LAZY) /dlsym/dlclose is not sufficient?
>
>>> What you can do is to isolate the function you want to jit by  
>>> extracting it to a new module. Load this module during runtime,  
>>> apply your pass and jitting it.
>> Yes this is exactly what I want to do. The interesting part is to  
>> load it during runtime. It is not sufficient for me to load it  
>> during JIT initialisation, but rather I want to load and JIT it  
>> during program execution. Hence my idea of declaring the function  
>> in the module as extern and load it through the lazy function  
>> creator.
>>
>> Quoting Willy WOLFF <willy.wolff at etu.unistra.fr>:
>>
>>>
>>> On 23 May 2014, at 14:55, Jonatan Waern wrote:
>>>
>>>> Hello, I am a novice at using the llvm api to do much of anything.
>>>>
>>>> I am trying to lazily apply certain optimisation passes on select  
>>>> functions when jitting a large program. It is undesirable to me  
>>>> to load the entire program as IR code and then generate code in  
>>>> memory for it (time concerns). It seems that there is a function  
>>>> in MCJIT "loadObjectFile" that would suit my purposes, allowing  
>>>> me to statically compile the program and then load it into the jit.
>>>>
>>>
>>> As far as I know, ObjectFile is either COFF, ELF or MachO format,  
>>> statically compiled. But the pass manager for applying  
>>> optimisation passes works only on llvm-ir.
>>>
>>>> My question is then what would be a good way to callback into the  
>>>> jit for applying the optimisation passes before calling one of  
>>>> the function? My current idea is to make the function extern in  
>>>> the llvm IR prior to static compile, and then use  
>>>> "installLazyFunctionCreator" to generate this function by loading  
>>>> ircode from an isolated module. Is this a feasible approach?
>>>>
>>>
>>> What you can do is to isolate the function you want to jit by  
>>> extracting it to a new module. Load this module during runtime,  
>>> apply your pass and jitting it.
>>> --
>>> Willy WOLFF
>
> --
> Willy WOLFF