[LLVMdev] Fail to load a pointer to a function inside MCJIT-ed code when it is reload from ObjectCache

[Lang Hames]

Hi Cheng,

Thanks for attaching the object file and bit code - it's very helpful.

I think the problem is that there is a pointer address hard-coded into your
IR. If the data that pointer references is stored at a different address on
your second run (which is highly likely) you get the failure that you're
seeing.

In your definition of JittedOpExpr, the first call instruction (to
IRExprGetValue1) is:

%lhs = call i64 @IrExprGetValue1(%struct.ExprState* inttoptr (i64
140715076067816 to %struct.ExprState*), %struct.ExprContext* %econtext, i8*
%isNull, i32* %isDone)

Note the hard coded pointer address 140715076067816 (0x7ffac81fa1e8). That
should probably have been a reference to the %expr argument of
JittedOpExpr. If you change this call to use %expr rather than the
hard-coded address I think it will fix your issue.

I hope this helps.

Regards,
Lang.



On Thu, Sep 11, 2014 at 2:45 PM, Cheng Zhu <chengzhu at gmail.com> wrote:

> Thank you Lang. I attached the ELF object file here for your reference.
> Here is the IR dump of JittedOpExpr LLVM function. IrExprGetValue1 LLVM
> function calls to external function expr->evalfunc(expr, econtext, isNull,
> isDone); which should be pointed by 0x7fe4801fa1f8. However, only the first
> time MCJIT generated object point to expr->evalfunc but second time when
> program load from object cache does not. In the second time, before program
> load object cache, brand new llvm::Module and llvm::ExecutionEngine are
> created. IrExprGetValue is saved in file as IR format and will be loaded
> into module every run. JittedOpExpr and other LLVM function are created and
> associated with the new Module.
>
> ; Function Attrs: uwtable
> define i64 @IrExprGetValue1(%struct.ExprState* %expr, %struct.ExprContext*
> %econtext, i8* %isNull, i32* %isDone) #0 {
> entry:
>   %expr.addr = alloca %struct.ExprState*, align 8
>   %econtext.addr = alloca %struct.ExprContext*, align 8
>   %isNull.addr = alloca i8*, align 8
>   %isDone.addr = alloca i32*, align 8
>   store %struct.ExprState* %expr, %struct.ExprState** %expr.addr, align 8
>   store %struct.ExprContext* %econtext, %struct.ExprContext**
> %econtext.addr, align 8
>   store i8* %isNull, i8** %isNull.addr, align 8
>   store i32* %isDone, i32** %isDone.addr, align 8
>   %0 = load %struct.ExprState** %expr.addr, align 8
>   %evalfunc = getelementptr inbounds %struct.ExprState* %0, i32 0, i32 2
>   %evalfunc1 = bitcast {}** %evalfunc to i64 (%struct.ExprState*,
> %struct.ExprContext*, i8*, i32*)**
>   %1 = load i64 (%struct.ExprState*, %struct.ExprContext*, i8*, i32*)**
> %evalfunc1, align 8
>   %2 = load %struct.ExprState** %expr.addr, align 8
>   %3 = load %struct.ExprContext** %econtext.addr, align 8
>   %4 = load i8** %isNull.addr, align 8
>   %5 = load i32** %isDone.addr, align 8
>   %call = call i64 %1(%struct.ExprState* %2, %struct.ExprContext* %3, i8*
> %4, i32* %5)
>   ret i64 %call
> }
>
> define i64 @JittedIntLit() {
> entry:
>   ret i64 5
> }
>
> define i64 @JittedOpExpr(%struct.ExprState* %expr, %struct.ExprContext*
> %econtext, i8* %isNull, i32* %isDone) {
> entry:
>   %lhs = call i64 @IrExprGetValue1(%struct.ExprState* inttoptr (i64
> 140715076067816 to %struct.ExprState*), %struct.ExprContext* %econtext, i8*
> %isNull, i32* %isDone)
>   %rhs = call i64 @JittedIntLit()
>   %tmp_add = add i64 %lhs, %rhs
>   ret i64 %tmp_add
> }
>
> and /samba/data/gDB2/src/ptcompiler/compiler/llvm_ir/GaussDB.ir contains
>
> extern "C"
> Datum IrExprGetValue(ExprState* expr, ExprContext *econtext, bool *isNull,
> ExprDoneCond *isDone) {
>   return expr->evalfunc(expr, econtext, isNull, isDone);
> }
>
> Cheng
>
> On Thu, Sep 11, 2014 at 1:26 PM, Lang Hames <lhames at gmail.com> wrote:
>
>> Hi Cheng,
>>
>> It sounds like the either (1) 0x7fe4801fa1f8 is part of the environment,
>> and it's not being configured the same way the 2nd time around, or (2) The
>> JIT is handling on-disk objects differently from in-memory ones. The first
>> option is more likely.
>>
>> It would be helpful if you could attach the object that was stored in
>> your cache.
>>
>> Cheers,
>> Lang.
>>
>> On Thu, Sep 11, 2014 at 10:58 AM, Cheng Zhu <chengzhu at gmail.com> wrote:
>>
>>> Hi, All
>>>
>>> I have a problem to reuse mcjit jitted code loaded from ObjectCache from
>>> a file. In the first run, I use MCJIT generate function JittedOpExpr object
>>> code as following and it runs OK. 0x7fe4801fa1f8 at instruction
>>> 0x00007fe4cc6c2014 points to 0x69382E which is the beginning of ExecEvalVar
>>> function. Then I save the object code into a file after implementing
>>> notifyObjectCompiled method.
>>>
>>>
>>>                     IrExprGetValue:
>>> 0x00007fe4cc6c2000:   push %rbp
>>> 0x00007fe4cc6c2001:   mov %rsp,%rbp
>>> 0x00007fe4cc6c2004:   mov 0x10(%rdi),%rax
>>> 0x00007fe4cc6c2008:   pop %rbp
>>> 0x00007fe4cc6c2009:   jmpq *%rax
>>> 0x00007fe4cc6c200b:   nopl 0x0(%rax,%rax,1)
>>>                     JittedOpExpr:
>>> 0x00007fe4cc6c2010:   push %rbp
>>> 0x00007fe4cc6c2011:   mov %rsp,%rbp
>>> *0x00007fe4cc6c2014:   movabs $0x7fe4801fa1f8,%rax*
>>> 0x00007fe4cc6c201e:   movabs $0x7fe4801fa1e8,%rdi
>>> 0x00007fe4cc6c2028:   callq *(%rax)
>>> 0x00007fe4cc6c202a:   add $0x5,%rax
>>> 0x00007fe4cc6c202e:   pop %rbp
>>> 0x00007fe4cc6c202f:   retq
>>> 0x00007fe4cc6c2030:   adc $0x0,%al
>>> 0x00007fe4cc6c2032:   add %al,(%rax)
>>> 0x00007fe4cc6c2034:   add %al,(%rax)
>>>
>>> *0x7fe4801fa1f8 -> 0x69382E*
>>> *                    ExecEvalVar*(ExprState*, ExprContext*, bool*,
>>> ExprDoneCond*):
>>> 0x000000000069382e:   push %rbp
>>> 0x000000000069382f:   mov %rsp,%rbp
>>> 0x0000000000693832:   push %r12
>>>
>>> In the next run, I buildedunction of JittedOpExpr again and loaded
>>> compiled object from that saved binary file using getObject and went
>>> through getPointertoFunction from MCJIT execution engine, I got the
>>> following object code in memory. But this time 0x7fe4801fa1f8 points to
>>> 0x0, so when callq *(%rax) it couldn't find the ExecEvalVar anymore. I
>>> followed the blog "Object Caching with the Kaleidoscope Example Problem"
>>> written by Andy Kaylor by implementing my own MCJITObjectCache class. Did I
>>> miss anything here? Thank you very much.
>>>
>>>                     IrExprGetValue:
>>> 0x00007fe4cc6c2000:   push %rbp
>>> 0x00007fe4cc6c2001:   mov %rsp,%rbp
>>> 0x00007fe4cc6c2004:   mov 0x10(%rdi),%rax
>>> 0x00007fe4cc6c2008:   pop %rbp
>>> 0x00007fe4cc6c2009:   jmpq *%rax
>>> 0x00007fe4cc6c200b:   nopl 0x0(%rax,%rax,1)
>>>                     JittedOpExpr:
>>> 0x00007fe4cc6c2010:   push %rbp
>>> 0x00007fe4cc6c2011:   mov %rsp,%rbp
>>> *0x00007fe4cc6c2014:   movabs $0x7fe4801fa1f8,%rax*
>>> 0x00007fe4cc6c201e:   movabs $0x7fe4801fa1e8,%rdi
>>> 0x00007fe4cc6c2028:   callq *(%rax)
>>> 0x00007fe4cc6c202a:   add $0x5,%rax
>>> 0x00007fe4cc6c202e:   pop %rbp
>>> 0x00007fe4cc6c202f:   retq
>>>
>>> *but 0x7fe4801fa1f8 -> 000000000*
>>>
>>> --
>>> Best regards
>>>
>>> Cheng
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>>
>>
>
>
> --
> Best regards
>
> Cheng
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140912/480cd6c6/attachment.html>