<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Dear All,<br>
<br>
Just to add my two cents to the discussion, we're working on a
project that will require inter-procedural analysis on the
MachineInstr IR that will also use inter-procedural analysis
information from the LLVM IR level. It isn't clear yet whether we
will need to keep the LLVM IR around or if we can encode the LLVM
IR analysis results into the MI IR during code generation and
throw the LLVM IR away, but for now, I'm thinking that we'll need
both LLVM IR and MI IR in memory at once. Memory consumption
isn't a problem for us as we're doing static analysis for
security, so buying big hardware is a reasonable solution for us,
and analysis time can grow large if needed.<br>
<br>
If the MachineModulePass functionality is added, then we will
probably use it. If no one adds it, we will probably just hack
LLVM to not delete the MachineFunction's during code generation.<br>
<br>
Regards,<br>
<br>
John Criswell<br>
<br>
On 7/17/16 2:52 PM, Pete Cooper via llvm-dev wrote:<br>
</div>
<blockquote
cite="mid:8DD620CF-619C-4973-869A-9DF52D1399D9@apple.com"
type="cite">
<meta http-equiv="Context-Type" content="text/html; charset=utf-8">
Hi James, Matthias
<div class=""><br class="">
</div>
<div class="">I recently proposed the idea of deleting the IR for
a function after its reached the AsmPrinter. This was before
the idea of MachineModulePasses. I was seeing peak memory
savings of 20% when LTOing clang itself.</div>
<div class=""><br class="">
</div>
<div class="">So it might be possible to trade the memory so that
IR and MachineIR aren’t live at the same time for all functions.
But that depends on whether a MachineModulePass would want
access to the IR. AA is the typical example of MachineIR
referencing back to IR. I knew it was safe to delete IR when we
hit the AsmPrinter because nothing at (or beyond) that point
needs AA.</div>
<div class=""><br class="">
</div>
<div class="">Cheers,</div>
<div class="">Pete<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Jul 17, 2016, at 10:57 AM, Matthias Braun
via llvm-dev <<a moz-do-not-send="true"
href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">
<blockquote type="cite" class="">
<div class=""><br class="Apple-interchange-newline">
On Jul 17, 2016, at 9:34 AM, James Molloy via
llvm-dev <<a moz-do-not-send="true"
href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="">Hi,</div>
<div class=""><br class="">
</div>
<div class="">[Apologies to those receiving this
mail twice - used the old list address by
accident]</div>
<div class=""><br class="">
</div>
<div class="">In LLVM it is currently not possible
to write a Module-level pass (a pass that
modifies or analyzes multiple MachineFunctions)
after DAG formation. This inhibits some
optimizations[1] and is something I'd like to
see changed.</div>
<div class=""><br class="">
</div>
<div class="">The problem is that in the backend,
we emit a function at a time, from DAG formation
to object emission. So no two MachineFunctions
ever exist at any one time. Changing this
necessarily means increasing memory usage.</div>
<div class=""><br class="">
</div>
<div class="">I've prototyped this change and have
measured peak memory usage in the worst case
scenario - LTO'ing llc and clang. Without
further ado:</div>
<div class=""><br class="">
</div>
<div class=""> llvm-lto llc: before: 1.44GB
maximum resident set size</div>
<div class=""> after: 1.68GB
(+17%)</div>
<div class=""><br class="">
</div>
<div class=""> llvm-lto clang: before: 2.48GB
maximum resident set size</div>
<div class=""> after: 3.42GB
(+33%)</div>
<div class=""><br class="">
</div>
<div class="">The increases are very large. This
is worst-case (non-LTO builds would see the peak
usage of the backend masked by the peak of the
midend) but still - pretty big. Thoughts? Is
this completely no-go? is this something that we
*just need* to do? Is crippling the backend
architecture to keep memory down justified? Is
this something we could enable under an option?</div>
</div>
</div>
</blockquote>
<div class="">I also recently did a prototype that
enables MachineModulePasses (<a moz-do-not-send="true"
href="https://github.com/MatzeB/llvm/tree/MachineModulePass" class="">https://github.com/MatzeB/llvm/tree/MachineModulePass</a>).
I assume your patch looks similar (I did not do any
measurements with mine).</div>
<div class=""><br class="">
</div>
<div class="">I believe this could be done in a way so
the memory stays at current levels if no machine
module pass issued (simply remove the memory used by
the MachineFunction after we AsmPrinted it). So that
would at least make the infrastructure available for
people to experiment with. So maybe that is a good
first step?</div>
<div class=""><br class="">
</div>
<div class="">So far all the uses I have seen did not
convince me that the increased memory consumption is
worth it (by default). But IMO if we can provide by
infrastructure for MachineModulePasses with nearly
zero cost in case no machinemodulepass is used, then
I’d say we should go for it.</div>
<div class=""><br class="">
</div>
<div class="">- Matthias</div>
<div class=""><br class="">
</div>
</div>
<span class="">_______________________________________________</span><br
class="">
<span class="">LLVM Developers mailing list</span><br
class="">
<a moz-do-not-send="true"
href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br
class="">
<a moz-do-not-send="true"
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div>
</blockquote>
</div>
<br class="">
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
LLVM Developers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>
<a class="moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
<br>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
<a class="moz-txt-link-freetext" href="http://www.cs.rochester.edu/u/criswell">http://www.cs.rochester.edu/u/criswell</a></pre>
</body>
</html>