[LLVMdev] bug in the JIT global variable emitter

[Dale Johannesen]

On Oct 14, 2008, at 4:47 AMPDT, Nuno Lopes wrote:

>>> Today I found a nice bug in the JIT global variable emitter.
>>> The problem may lead to an assert() failure when doing the  
>>> following:
>>> 1) compile some function
>>> 2) emit a global variable
>>> 3) compile another function. an assert() may trigger in the JIT  
>>> memory
>>> manager
>>>
>>> This happens because the JIT global variable emitter is using the
>>> MachineCodeEmitter::allocate() function, which uses memory allocated
>>> by the
>>> JIT memory manager (which should be used for functions only).
>>
>> No, this was a deliberate change, 54442.  We have a situation where a
>> user
>> wants to emit JIT code on one machine, then send it off to another
>> machine to
>> execute.  Putting statically allocated data in the same buffer as  
>> code
>> is the
>> easiest approach to make this work, although there may be others.
>
> Ok, thanks for the explanation. So my first patch doesn't work.  
> Also, to be
> clear, this bug has nothing to do with overflowing the JIT memory  
> buffer.
> I made another one that takes keeps the allocation of global  
> variables in
> the JIT buffer, but it creates a new mem block if it doesn't exist  
> (i.e.
> when dumping a global variable out of the scope of a function  
> compilation).
> The patch is at:
> http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt

OK, I think I finally understand this:  the allocation in  
JIT::getOrEmitGlobalVariable doesn't call into JITEmitter but uses the  
MachineCodeEmitter directly.  Thus, if you call it from outside the  
JITEmitter, it won't update the data structures the JITEmitter relies  
on.   In the usual case, the JITEmitter calls into JIT and updates its  
own data structures, so that case works fine.

I think adding a smarter allocateSpace to JITEmitter is the right  
general idea, and your patch looks OK.   I'd like to get another pair  
of eyes on it though, as I find this area not straightfoward:)  Please  
run the testsuite before committing.

Thanks for analyzing this and the test case.

> The problem only happens when calling
> ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm  
> program. If
> the function is called when JITing a function, it works, since it  
> will dump
> the global variable to the memory reserved to the function being  
> JITed,
> which raises a question: shouldn't it generate the GV to some non- 
> executable
> memory block?? My patch doesn't attempt to fix this last concern.
>
> Thanks,
> Nuno
>
> P.S.: the control flow of this bug is quite complex, so feel free to  
> ask if
> you don't get what the problem is.
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev