[LLVMdev] MC-JIT

Tue Jul 20 13:36:03 PDT 2010

> In the context of the JIT, there really is no such thing as a
> relocation, just fixups. I'm not completely sure what the right
> approach is, but the JIT should be able to fully resolve all of the
> symbols that are being used in the module. We may need some extra
> interfaces to allow the JIT to tell the MCAssembler about the address
> of some external symbols though.

OK, I'm still experimenting with this. But it seems that I just need
to modify the "FixedValue" parameter of the RecordRelocation method.
More on this in future patches. :)

> There seems to be an ownership problem of the MCJITObjectWriter.  If I
> understand the code correctly, the assembler Finish method takes
> ownership of the Writer parameter, which presumably is needed to JIT
> two functions.

Ok, it's not really clear, but with the patch :
- If no Writer is provided to the Finish method, the MCAssembler takes
ownership of the Writer it creates
- If a Writer is provided, the ownership is not taken by MCAssembler

> Ok. We could also make the streamer setup the right ObjectWriter,
> since the ObjectStreamer should always know what that is.

Yep ! It will clarify the ownership thing. But my patch was to disturb
as little as possible the other streamers. I can implement it like
this if you prefer.

> +1 for separate patches, or alternatively, can we just commit this and
> begin developing this in-tree?  Then other people can hack on it
> before it's considered "ready".  :)

No problem for me and Jan, we can work out or in tree, even if it is
probably more convenient in tree.

> - Try to reuse the support classes whenever possible.

This was planned like this.

> Regarding the memory management strategy, I'd like to take this
> opportunity to try to do something more sane than the current JIT.
> Instead of streaming the code directly into RWX memory, it would be
> better to stream to some other resizeable in-memory data structure,
> and allocate the RWX memory once the size is known.  You can probably
> wait on this one, but I'd like to see this eventually.

The plan was to allocate just the necessary size of memory, as after
the layout, memory size needed for the generated function should be
known.

> Seems reasonable, but I haven't looked at the code yet. I would
> suggest trying to split your work up into separate patches which
> implement incremental pieces of functionality, then submitting them to
> llvm-commits for review. That is much easier for us to deal with than
> large monolithic patches out of tree.

Ok. Sorry for the too big patch. Attached is the first patch adding
only 2 hooks on TargetMachine and on MCAssembler. Style should be LLVM
compliant. Apply it with "patch -p0".

> You shouldn't need to initialize all targets to use MC, you should be
> able to just use InitializeHostTarget(). If not, its probably a bug
> somewhere.

Ah. Sorry never mind ! I finally found the "InitializeNativeTarget"
method which do what I need.

Olivier.

On Tue, Jul 20, 2010 at 7:06 PM, Reid Kleckner <reid.kleckner at gmail.com> wrote:
> Some boring style comments:
> - whack trailing whitespace
> - spaces, not tabs
> - the methods in MCJITStreamer.cpp should probably have blank lines between them
>
> There seems to be an ownership problem of the MCJITObjectWriter.  If I
> understand the code correctly, the assembler Finish method takes
> ownership of the Writer parameter, which presumably is needed to JIT
> two functions.
>
> +1 for separate patches, or alternatively, can we just commit this and
> begin developing this in-tree?  Then other people can hack on it
> before it's considered "ready".  :)
>
> There's also a lot of copying stuff out of the JIT going on.  The
> MCJIT doesn't need to reimplement things like the JITEventListener
> interface, for example.  I think you should either:
> - Maybe make a svn branch and replace the JIT in-place.
> - Try to reuse the support classes whenever possible.
>
> Regarding the memory management strategy, I'd like to take this
> opportunity to try to do something more sane than the current JIT.
> Instead of streaming the code directly into RWX memory, it would be
> better to stream to some other resizeable in-memory data structure,
> and allocate the RWX memory once the size is known.  You can probably
> wait on this one, but I'd like to see this eventually.
>
> Reid
>
> On Mon, Jul 19, 2010 at 5:14 AM, Olivier Meurant
> <meurant.olivier at gmail.com> wrote:
>> Together with Jan Sjodin (in copy of this email), we begin an
>> implementation of the JIT with MC. The idea, suggested by Jan, is to
>> develop a MCJIT in parallel of the current JIT and to keep the two
>> implementations until (at least) the new MC one is mature enough.
>> Currently code is kept on gitorious
>> (http://gitorious.org/llvm-mc-jit/llvm-mc-jit).
>>
>> Following this, a boolean "bool MCJIT = false" has been introduced to
>> the "static ExecutionEngine *create(...)" method, allowing the lli
>> tool to use the MCJIT with the optional flag "-mcjit".
>>
>> The MCJIT can now execute little functions with no relocation (like
>> add(a,b) => a+b), to do that some modifications have been made :
>> - The addPassesToEmitMC method has been added to TargetMachine class.
>> It fills the MCContext pointer so we can build the MCJITStreamer.
>> - The Finish method of MCAssembler have a new optional argument
>> "Writer" to allow a custom MCJITObjectWriter to be used.
>>
>> Can you give us some feedbacks on the general idea and on this 2
>> particular hooks ?
>>
>>
>> Currently MCJIT has one unittest and the binary size is quite big
>> (~110MB in debug...) because before using the MC framework we need to
>> call "InitializeAllTargets()" and friends (same as llc does). For the
>> JIT, we need only the "host" backend and "InitializeHostTarget()"-like
>> method doesn't seem to exist. Do you have an opinion on this ?
>>
>> Attached you will find the patch introducing the first MCJIT draft.
>> Can you comment on this ?
>>
>> Jan and Olivier.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mc_jit_01.patch
Type: text/x-patch
Size: 4499 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100720/3226c8b9/attachment.bin>