[LLVMdev] bug in the JIT global variable emitter

[Nuno Lopes]

>>>> Today I found a nice bug in the JIT global variable emitter.
>>>> The problem may lead to an assert() failure when doing the
>>>> following:
>>>> 1) compile some function
>>>> 2) emit a global variable
>>>> 3) compile another function. an assert() may trigger in the JIT
>>>> memory
>>>> manager
>>>>
>>>> This happens because the JIT global variable emitter is using the
>>>> MachineCodeEmitter::allocate() function, which uses memory allocated
>>>> by the
>>>> JIT memory manager (which should be used for functions only).
>>>
>>> No, this was a deliberate change, 54442.  We have a situation where a
>>> user
>>> wants to emit JIT code on one machine, then send it off to another
>>> machine to
>>> execute.  Putting statically allocated data in the same buffer as
>>> code
>>> is the
>>> easiest approach to make this work, although there may be others.
>>
>> Ok, thanks for the explanation. So my first patch doesn't work.
>> Also, to be
>> clear, this bug has nothing to do with overflowing the JIT memory
>> buffer.
>> I made another one that takes keeps the allocation of global
>> variables in
>> the JIT buffer, but it creates a new mem block if it doesn't exist
>> (i.e.
>> when dumping a global variable out of the scope of a function
>> compilation).
>> The patch is at:
>> http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt
>
> Sorry, I am still not able to understand the problem. Is there a bug
> in the default memory manager? From your patch it seems like there is
> a real bug. What is the assertion that you ran into?

This bug is a problem in the design of the JIT GV emitter, that appeared 
with the revision 54442, as Dale pointed.
I was trying to avoid to make a test case, but I did one now. Please find it 
in attach (I hope the ML doesn't strip it). The source code has enough 
comments to explain what's going on.
The module.bc is just a container to have 2 functions, but it's irrelevant 
to the problem. I just picked some random .bc file that had 2 functions.


>> The problem only happens when calling
>> ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm
>> program. If
>> the function is called when JITing a function, it works, since it
>> will dump
>> the global variable to the memory reserved to the function being
>> JITed,
>> which raises a question: shouldn't it generate the GV to some non-
>> executable
>> memory block?? My patch doesn't attempt to fix this last concern.
>
> There are two ways to go about this. Either we enhance memory manager
> interface to create non-executable memory for GVs etc. Or we keep the
> memory manager simple and let the JIT change the privilege as it sees
> fit. I prefer the later unless there is a good argument for an
> alternative.

I didn't check, but I think the JIT engine is mapping the whole function's 
memory as executable. As now most global variables end up being created when 
JITing functions, those GV end up in executable memory.

Nuno 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: llvm-jit-memmanager.zip
Type: application/x-zip-compressed
Size: 1886 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081015/66bb60c8/attachment.bin>