[LLVMdev] Function-at-a-time Processing

[David Greene]

We process very large programs and it is not unusual for the IR for some
compilation unit to exceed system memory.  With some hacking in LLVM 2.5
I was able to coax LLVM to generate asm for each functioin as it was
processed and then completely forget about it (i.e. delete it) and move
on to the next function.

This required a bit of hackery.  I had to create two pass managers, one
for the module and one for each function.  I called doInitialization on
the module pass manager to convince the AsmPrinter to dump out global
information needed at the top of the asm file.

Then I would run each function though the function pass manager as they
were encountered.  I could construct, run and destroy the function pass
manager for each function.

Finally I would run the module pass manager and doFinalization on the
module pass manager to dump out the stuff needed at the end of the asm
file.

This was not at all pretty but I could at least get it to work.

I'm in the last stages of upgrading to LLVM 2.7 and I'm finding that the
new MCStreamer system is making this sort of model much more difficult.
MCContexts are somehow shared between the separate module and function
pass managers in ways I don't understand.  I think that somehow the
LLVMTargetMachine is shared between the pass managers.

I think this sort of sharing is actually correct since we want the
MCContext for the function pass manager to be the same as the one for
the module pass manager.  That way globals, etc. have consistent
representations in both.  Unfortunately, when the first function pass
manager gets deleted, it destroys the MCContext alonmg with it.
Actually, the AsmPrinter destroys the MCContext through a reference (!)
which makes things all the more confusing.  When the module pass manager
comes along and tries of access an MCSection, everything blows up.

I had to re-add addAssemblyEmitter to LLVMTargetMachine to try to make
this work.  It essentially takes the AsmPrinter creation bits from
addPassesToEmitFile and packages them into a separate function.

[As a side note, I don't understand how the two AsmPrinters (one for the
module pass manager, one for the function pass manager) end up pointing
to the same MCContext as LLVMTargetMachine::addPassesToEmitFile()
creates a new MCContext to pass to Target::createAsmPrinter.  Insights
welcome.  :)]

Perhaps there is a better way of doing this that's officially supported.
Is this model of function processing supported at all?  If not, has
anyone thought about how we might support it?  I think this is a model
that is not all that uncommon so I would hope we could figure out a
reasonable solution.

Thanks!

                     -Dave