[llvm-dev] RFC: Enabling Module passes post-ISel

[Hal Finkel via llvm-dev]

----- Original Message -----

> From: "James Molloy" <james at jamesmolloy.co.uk>
> To: "Justin Bogner" <mail at justinbogner.com>, "James Molloy via
> llvm-dev" <llvm-dev at lists.llvm.org>
> Cc: "Hal Finkel" <hfinkel at anl.gov>, "Chandler Carruth"
> <chandlerc at google.com>, "Matthias Braun" <matze at braunis.de>, "Pete
> Cooper" <peter_cooper at apple.com>
> Sent: Tuesday, July 19, 2016 9:16:02 AM
> Subject: Re: [llvm-dev] RFC: Enabling Module passes post-ISel

> Hi all,

> I like all the ideas so far. Here are my thoughts:

> I think that fundamentally users of LLVM should be able to opt-in to
> more aggressive or intensive computation at compile time if they
> wish. Users' needs differ, and while a 33% increase in clang LTO is
> absolutely out of the question for some people, for those developing
> microcontrollers or HPC applications that may well be irrelevant.
I agree. A 33% increase is absorbable in many environments. 

> Either the volume of code expected is significantly smaller or
> they're happy to trade off compile time for expensive server time.
> That does not mean that we shouldn't strive for a solution that can
> be acceptable by all users. On the other hand making something
> opt-in makes it non-default, and that increases the testing surface.

> Tangentially I think that LLVM currently doesn't have the right
> tuning knobs to allow the user to select their desired tradeoff. We
> have one optimization flag -O{s,z,0,1,2,3} which encodes both
> optimization *goal* (a point on the pareto curve between size and
> speed) and amount of effort to expend at compile time achieving that
> goal. Anyway, that's besides the point.

> I like Justin's idea of removing IR from the backend to free up
> memory. I think it's a very long term project though, one that
> requires significant (re)design; alias analysis access in the
> backend would be completely broken and BasicAA among others depends
> on seeing the IR at query time. We'd need to work out a way of
> providing alias analysis with no IR present. I don't think that is
> feasible for the near future.

> So my suggestion is that we go with Matthias' idea - do the small
> amount of refactoring needed to allow MachineModulePasses on an
> opt-in basis. The knobs to enable that opt-in might need some more
> bikeshedding.
This makes sense to me. I expect that targets will be able to opt-in in some optimization-level-dependent fashion. 

-Hal 

> Cheers,

> James

> On Tue, 19 Jul 2016 at 08:21 Justin Bogner < mail at justinbogner.com >
> wrote:

> > James Molloy via llvm-dev < llvm-dev at lists.llvm.org > writes:
> 
> > > In LLVM it is currently not possible to write a Module-level pass
> > > (a pass that
> 
> > > modifies or analyzes multiple MachineFunctions) after DAG
> > > formation. This
> 
> > > inhibits some optimizations[1] and is something I'd like to see
> > > changed.
> 
> > >
> 
> > > The problem is that in the backend, we emit a function at a time,
> > > from DAG
> 
> > > formation to object emission. So no two MachineFunctions ever
> > > exist
> > > at any one
> 
> > > time. Changing this necessarily means increasing memory usage.
> 
> > >
> 
> > > I've prototyped this change and have measured peak memory usage
> > > in
> > > the worst
> 
> > > case scenario - LTO'ing llc and clang. Without further ado:
> 
> > >
> 
> > > llvm-lto llc: before: 1.44GB maximum resident set size
> 
> > > after: 1.68GB (+17%)
> 
> > >
> 
> > > llvm-lto clang: before: 2.48GB maximum resident set size
> 
> > > after: 3.42GB (+33%)
> 
> > >
> 
> > > The increases are very large. This is worst-case (non-LTO builds
> > > would see the
> 
> > > peak usage of the backend masked by the peak of the midend) but
> > > still - pretty
> 
> > > big. Thoughts? Is this completely no-go? is this something that
> > > we
> > > *just need*
> 
> > > to do? Is crippling the backend architecture to keep memory down
> > > justified? Is
> 
> > > this something we could enable under an option?
> 

> > Personally, I think this price is too high. I think that if we want
> > to
> 
> > enable machine module passes (which we probably do) we need to turn
> 
> > MachineFunction into more of a first class object that isn't just a
> 
> > wrapper around IR.
> 

> > This can and should be designed to work something like Pete's
> > solution,
> 
> > where we get rid of the IR and just have machine level stuff in
> > memory.
> 
> > This way, we may still increase the memory usage here, but it
> > should
> > be
> 
> > far less dramatic.
> 

> > You'll note that doing this also has tangential benefits - it
> > should
> > be
> 
> > helpful for simplifying MIR and generally improving testability of
> > the
> 
> > backends.
> 

-- 

Hal Finkel 
Assistant Computational Scientist 
Leadership Computing Facility 
Argonne National Laboratory 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160727/0269ce26/attachment-0001.html>