[llvm-dev] RFC: Enabling Module passes post-ISel

James Molloy via llvm-dev llvm-dev at lists.llvm.org
Tue Jul 19 07:16:02 PDT 2016

Hi all,

I like all the ideas so far. Here are my thoughts:

I think that fundamentally users of LLVM should be able to opt-in to more
aggressive or intensive computation at compile time if they wish. Users'
needs differ, and while a 33% increase in clang LTO is absolutely out of
the question for some people, for those developing microcontrollers or HPC
applications that may well be irrelevant. Either the volume of code
expected is significantly smaller or they're happy to trade off compile
time for expensive server time. That does not mean that we shouldn't strive
for a solution that can be acceptable by all users. On the other hand
making something opt-in makes it non-default, and that increases the
testing surface.

Tangentially I think that LLVM currently doesn't have the right tuning
knobs to allow the user to select their desired tradeoff. We have one
optimization flag -O{s,z,0,1,2,3} which encodes both optimization *goal* (a
point on the pareto curve between size and speed) and amount of effort to
expend at compile time achieving that goal. Anyway, that's besides the

I like Justin's idea of removing IR from the backend to free up memory. I
think it's a very long term project though, one that requires significant
(re)design; alias analysis access in the backend would be completely broken
and BasicAA among others depends on seeing the IR at query time. We'd need
to work out a way of providing alias analysis with no IR present. I don't
think that is feasible for the near future.

So my suggestion is that we go with Matthias' idea - do the small amount of
refactoring needed to allow MachineModulePasses on an opt-in basis. The
knobs to enable that opt-in might need some more bikeshedding.



On Tue, 19 Jul 2016 at 08:21 Justin Bogner <mail at justinbogner.com> wrote:

> James Molloy via llvm-dev <llvm-dev at lists.llvm.org> writes:
> > In LLVM it is currently not possible to write a Module-level pass (a
> pass that
> > modifies or analyzes multiple MachineFunctions) after DAG formation. This
> > inhibits some optimizations[1] and is something I'd like to see changed.
> >
> > The problem is that in the backend, we emit a function at a time, from
> > formation to object emission. So no two MachineFunctions ever exist at
> any one
> > time. Changing this necessarily means increasing memory usage.
> >
> > I've prototyped this change and have measured peak memory usage in the
> worst
> > case scenario - LTO'ing llc and clang. Without further ado:
> >
> >   llvm-lto llc:   before: 1.44GB maximum resident set size
> >                   after:  1.68GB (+17%)
> >
> >   llvm-lto clang: before: 2.48GB maximum resident set size
> >                   after:  3.42GB (+33%)
> >
> > The increases are very large. This is worst-case (non-LTO builds would
> see the
> > peak usage of the backend masked by the peak of the midend) but still -
> pretty
> > big. Thoughts? Is this completely no-go? is this something that we *just
> need*
> > to do? Is crippling the backend architecture to keep memory down
> justified? Is
> > this something we could enable under an option?
> Personally, I think this price is too high. I think that if we want to
> enable machine module passes (which we probably do) we need to turn
> MachineFunction into more of a first class object that isn't just a
> wrapper around IR.
> This can and should be designed to work something like Pete's solution,
> where we get rid of the IR and just have machine level stuff in memory.
> This way, we may still increase the memory usage here, but it should be
> far less dramatic.
> You'll note that doing this also has tangential benefits - it should be
> helpful for simplifying MIR and generally improving testability of the
> backends.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160719/e214e7d4/attachment.html>

More information about the llvm-dev mailing list