[llvm-dev] RFC: Enabling Module passes post-ISel

[Pete Cooper via llvm-dev]

Hi James, Matthias

I recently proposed the idea of deleting the IR for a function after its reached the AsmPrinter.  This was before the idea of MachineModulePasses.  I was seeing peak memory savings of 20% when LTOing clang itself.

So it might be possible to trade the memory so that IR and MachineIR aren’t live at the same time for all functions.  But that depends on whether a MachineModulePass would want access to the IR.  AA is the typical example of MachineIR referencing back to IR.  I knew it was safe to delete IR when we hit the AsmPrinter because nothing at (or beyond) that point needs AA.

Cheers,
Pete
> On Jul 17, 2016, at 10:57 AM, Matthias Braun via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
>> 
>> On Jul 17, 2016, at 9:34 AM, James Molloy via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> Hi,
>> 
>> [Apologies to those receiving this mail twice - used the old list address by accident]
>> 
>> In LLVM it is currently not possible to write a Module-level pass (a pass that modifies or analyzes multiple MachineFunctions) after DAG formation. This inhibits some optimizations[1] and is something I'd like to see changed.
>> 
>> The problem is that in the backend, we emit a function at a time, from DAG formation to object emission. So no two MachineFunctions ever exist at any one time. Changing this necessarily means increasing memory usage.
>> 
>> I've prototyped this change and have measured peak memory usage in the worst case scenario - LTO'ing llc and clang. Without further ado:
>> 
>>   llvm-lto llc:   before: 1.44GB maximum resident set size
>>                   after:  1.68GB (+17%)
>> 
>>   llvm-lto clang: before: 2.48GB maximum resident set size
>>                   after:  3.42GB (+33%)
>> 
>> The increases are very large. This is worst-case (non-LTO builds would see the peak usage of the backend masked by the peak of the midend) but still - pretty big. Thoughts? Is this completely no-go? is this something that we *just need* to do? Is crippling the backend architecture to keep memory down justified? Is this something we could enable under an option?
> I also recently did a prototype that enables MachineModulePasses (https://github.com/MatzeB/llvm/tree/MachineModulePass <https://github.com/MatzeB/llvm/tree/MachineModulePass>). I assume your patch looks similar (I did not do any measurements with mine).
> 
> I believe this could be done in a way so the memory stays at current levels if no machine module pass issued (simply remove the memory used by the MachineFunction after we AsmPrinted it). So that would at least make the infrastructure available for people to experiment with. So maybe that is a good first step?
> 
> So far all the uses I have seen did not convince me that the increased memory consumption is worth it (by default). But IMO if we can provide by infrastructure for MachineModulePasses with nearly zero cost in case no machinemodulepass is used, then I’d say we should go for it.
> 
> - Matthias
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160717/7b67c0f9/attachment.html>