[PATCH] Rework lli-child-target for portability and correctness

[Kaylor, Andrew]

>Having the lightweight child implementation as a single source is important for embedding and
>deploying the remote JIT if you plan on using it anywhere outside the LLVM source tree.

The lli-child-target should not be seen as a building block for remote MCJIT implementations.  It only exists for testing purposes.  I suppose it has some secondary value as a reference implementation, but I would expect that any client using remote MCJIT capabilities would provide a more robust and sophisticated implementation for allocation and communication.  LLDB, for instance, accomplishes this in a way that doesn't even resemble the lli-child-target implementation.

On the other hand, I do think that there is value in having the memory utility functions that lli-child-target uses available in this sort of a lightweight mode.   The cache invalidation code, in particular, is something that I wouldn't want every client to have to re-invent.  The reason that I pulled them out (and duplicated them) in the first place was to demonstrate that code generated by MCJIT could execute in an external process that did not require any LLVM libraries.

-Andy

-----Original Message-----
From: Alp Toker [mailto:alp at nuanti.com] 
Sent: Tuesday, December 03, 2013 11:26 AM
To: Reid Kleckner
Cc: llvm-commits; Kaylor, Andrew
Subject: Re: [PATCH] Rework lli-child-target for portability and correctness


On 03/12/2013 18:05, Reid Kleckner wrote:
> Carving up Support with #ifdefs to support .cpp file inclusion at a 
> distance doesn't seem right.
>
> Why is lli-child-target so large in the first place?  If it links 
> statically and only calls a few helpers out of Support, the linker 
> should avoid linking in unreferenced object files from the static 
> library.  If the .cpp files in Support reference each other too much, 
> maybe we can find a way to factor them differently.

Hi Reid,

Having the lightweight child implementation as a single source is important for embedding and deploying the remote JIT if you plan on using it anywhere outside the LLVM source tree.

Cross-process (and potentially cross-platform) JIT is an unusual problem to solve so it was bound to have an unusual source setup. In this instance, I think Andy got it mostly right except for the duplicated code, so it'd be a shame to break that model now by introducing a dependency on the host build configuration.

I note also that the inc files were originally made to support this use case. it's just that they've inadvertently grown a web of cross-dependencies in the last few years -- my first patch resolves that sufficiently for the lli use case.


>
> Alternatively, we could bring back System for this kind of low-level 
> stuff.

Or you could just fork() from lli, but both would be throwing out the baby with the bathwater.

Alp.




>
>
> On Tue, Dec 3, 2013 at 9:34 AM, Alp Toker <alp at nuanti.com 
> <mailto:alp at nuanti.com>> wrote:
>
>     This is a bumper patch to fix assorted MCJIT remote issues, split
>     into two parts for ease of review.
>
>     Minor tweaks outside of lli were necessary to support reuse of
>     LLVM's Support implementation by inclusion without having to link
>     to LLVM.
>
>       9 files changed, 157 insertions(+), 253 deletions(-)
>
>     Patch 1:
>
>     Provide LLVM_STANDALONE for lightweight Support embedders
>
>     This is sufficient to build lli-child-target in a standalone
>     configuration
>     without having to link any LLVM libraries.
>
>     Also fixes a problem in AllocateRWX() where
>     process::get_self()->page_size()
>     was being called repeatedly.
>
>     Patch 2:
>
>     Share a single IPC transport implementation between the child and
>     parent
>     instead of duplicating the code.
>
>     Share the existing LLVM memory management and cache invalidation
>     code instead
>     of copy-and-pasting it. This is complicated; we don't want to
>     maintain two
>     versions but at the same time need to keep lli-child-target free of
>     dependencies.
>
>     Make memory management and instruction cache flushing portable in
>     the child
>     application using LLVM's Support headers in standalone mode.
>     Eliminate lots of
>     copy-and-pasted code.
>
>     Fix a typo that caused the instruction cache invalidation never to
>     run on
>     executable pages, which could be a problem on ARM, PowerPC etc.
>
>     Add assertions for piped read/write. This should help cut down on
>     needless
>     assertions at the callsites.
>
>     Build lli-child-target in a standalone configuration, reducing
>     binary size to
>     24KB down from ~500KB.
>
>     These fixes pave the way for a Windows implementation.
>
>     Alp.
>
>     -- 
>     http://www.nuanti.com
>     the browser experts
>
>
>
>     _______________________________________________
>     llvm-commits mailing list
>     llvm-commits at cs.uiuc.edu <mailto:llvm-commits at cs.uiuc.edu>
>     http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>

--
http://www.nuanti.com
the browser experts