[LLVMdev] Follow-up on: Dynamic updates of current executed code

Tue May 11 06:26:02 PDT 2004

Hello!

I am coming back to the below discussion again, regarding the LLVM support of Ruby dynamics. The initial problem description is as follow, to refresh your memory :) After that my questions come:

-----------BEGIN Initial problem description---------------

Problem is though, that the Ruby compiler is integrated in the compilation of the program being executed, to be able to parse & compile dynamic code at run-time. Therefore the calls to ExecutionEngine::getPointerToGlobal(F) need to be made in LLVM code. Here is a detailed simplistic example in pseudocode of what we want to do:

First Ruby code is entered at run-time, received as a string, parsed and compiled into the following code:

%--------------
; External function
declare int %printf(sbyte*, ...)

; Custom function
int %puts_kernel( sbyte* %string )
{
%tmp.0 = call int (sbyte*, ...)* %printf( sbyte* %string )
ret int 0
}
%-------------

This code is represented in the string variable 
%dynamically_compiled_function_code below:

%----------
%dynamically_compiled_function_code = internal constant [LENGTH x sbyte] c"--String with the function code--\0A\00"

; Table of function pointer(s)
%kernel = type { int ( sbyte* )* }

int %main() {

; Create the kernel in memory, and get pointer to first function pointer
%theKernel = malloc %kernel
%FirstFunctionPTR = getelementptr %kernel* %theKernel, long 0, ubyte 0
;Load code
%myNewFunction = %getPointerToGlobal(%dynamically_compiled_function_code)

; Write memory address of myNewFunction() into kernel struct
store RETURNTYPE (PARAMTYPE*)* %myNewFunction, RETURNTYPE (PARAMTYPE*)** %FirstFunctionPTR

;Any code using first function element in %kernel is now_
;using dynamically updated function!?

ret int 0
}

%-----------

The questionmark is at this pseudocode row:
%myNewFunction = %getPointerToGlobal(%dynamically_compiled_function_code)

Is there an llvm version of the getPointerToGlobal() function as outlined, and can the %myNewFunction pointer be used as described? 
Also, does the getPointerToGlobal() take human readable code (.ll) or only binary byte code (.bc)? Is there a specification of how to write binary byte code directly, so we do not have to externally call the llvm-as utility?

Best regards
Anders
-----------END Initial problem description---------------

-------------------Previous Discussion:

>> Is there an llvm version of the getPointerToGlobal() function as
>> outlined, and can the %myNewFunction pointer be used as described? 

>Yes, see llvm/lib/ExecutionEngine/ExecutionEngine.cpp . It will compile
>the given Function* to machine code, and return a pointer to you. In
>your LLVM code, you will have to first cast your function pointer before
>storing it, but it should work as you have it.

1) Can a modified version of  ParseAssemblyFile(InputFilename) that takes a pointer to LLVM code in memory instead be used to generate a Function* for use by the getPointerToGlobal()? I understand that it currently returns a Module*. Is it possible to make it return an F* instead, and if so how would the implementation roughly look like? 

2) Can this function be compiled into LLVM independently and used as a stand-alone function by the run-time compiler?

>One thing is that you need to have a pointer to the currently-running
>instance of ExecutionEngine to call methods on it... a easy quick hack
>would be to create a static variable that the ExecutionEngine
>constructor writes its own pointer (this) into, and have a C function
>(in ExecutionEngine.cpp) that returns that pointer. Your code can then
>access the currently-running ExecutionEngine through that pointer. There
>should be a cleaner solution to this...

We aim for an integrated binary solution with compiler-application-LLVMjitter, (if at all possible), so my question is: 

1) How is it possible to make LLVM calls to C++ methods? Can the Ruby-LLVM compiler, applied on itself thus residing in LLVM-form, call the getPointerToGlobal() using the pointer to the currently-running instance of ExecutionEngine from the 
fast hack
 above? :) In this process it is passing the F* from the modified ParseAssemblyFile(InputFilename), finally getting a pointer to the function, which can be called using standard LLVM syntax?

(&)

>Here's an idea:

>In tools/llvm-as/llvm-as.cpp, you can see how the assembly parser really
>does its job: it calls ParseAssemblyFile(InputFilename) which returns it
>a Module*. 

>ParseAssemblyFile is defined in llvm/lib/AsmParser/Parser.cpp; something
>similar to it, but one that accepts a string instead of a filename could
>be implemented.

>However, there is a catch (as I see it, Chris may correct me):

>The entry to the parser is to parse a whole Module.  The new function
>that you write may reference other functions. When you parse a Module,
>it has to be complete, so functions that are not defined in the module
>are "external". Having a definition for those functions will make those
>external functions different (in terms of their location in memory) and
>hence different value for Function*, which will mess with the
>ExecutionEngine as it uses a map Function* => memory address of
>translation. Other problems would be with types, same idea.

>(Is this making sense?)

>A solution would be perhaps to use a different entry point into a parser
>that would just parse a Function instead of a whole Module. It should
>then resolve the types it finds to the already-parsed Module in memory.
>I don't think this has been done yet.

>-- 
>Misha Brukman :: http://misha.brukman.net :: http://llvm.cs.uiuc.edu

I hope you are able to roughly describe this scenario! We do not aim for implementing it at this point, but only to know if it is possible, and roughly how.

Best regards 
Anders Alexandersson
RubyComp project

----------------------------------------------------------------
Anders Alexandersson
Masters student at the special year of Software Engineering, HTU Trollhättan
E-mail: anders.alexandersson at student.htu.se