[LLVMdev] Function-at-a-time Processing
criswell at illinois.edu
Wed Jul 28 13:45:21 PDT 2010
Dumb question: Would it be easier to write an LLVM tool that took the
large IR file and divided it into several smaller IR files that could
then be code-generated separately?
-- John T.
David Greene wrote:
> We process very large programs and it is not unusual for the IR for some
> compilation unit to exceed system memory. With some hacking in LLVM 2.5
> I was able to coax LLVM to generate asm for each functioin as it was
> processed and then completely forget about it (i.e. delete it) and move
> on to the next function.
> This required a bit of hackery. I had to create two pass managers, one
> for the module and one for each function. I called doInitialization on
> the module pass manager to convince the AsmPrinter to dump out global
> information needed at the top of the asm file.
> Then I would run each function though the function pass manager as they
> were encountered. I could construct, run and destroy the function pass
> manager for each function.
> Finally I would run the module pass manager and doFinalization on the
> module pass manager to dump out the stuff needed at the end of the asm
> This was not at all pretty but I could at least get it to work.
> I'm in the last stages of upgrading to LLVM 2.7 and I'm finding that the
> new MCStreamer system is making this sort of model much more difficult.
> MCContexts are somehow shared between the separate module and function
> pass managers in ways I don't understand. I think that somehow the
> LLVMTargetMachine is shared between the pass managers.
> I think this sort of sharing is actually correct since we want the
> MCContext for the function pass manager to be the same as the one for
> the module pass manager. That way globals, etc. have consistent
> representations in both. Unfortunately, when the first function pass
> manager gets deleted, it destroys the MCContext alonmg with it.
> Actually, the AsmPrinter destroys the MCContext through a reference (!)
> which makes things all the more confusing. When the module pass manager
> comes along and tries of access an MCSection, everything blows up.
> I had to re-add addAssemblyEmitter to LLVMTargetMachine to try to make
> this work. It essentially takes the AsmPrinter creation bits from
> addPassesToEmitFile and packages them into a separate function.
> [As a side note, I don't understand how the two AsmPrinters (one for the
> module pass manager, one for the function pass manager) end up pointing
> to the same MCContext as LLVMTargetMachine::addPassesToEmitFile()
> creates a new MCContext to pass to Target::createAsmPrinter. Insights
> welcome. :)]
> Perhaps there is a better way of doing this that's officially supported.
> Is this model of function processing supported at all? If not, has
> anyone thought about how we might support it? I think this is a model
> that is not all that uncommon so I would hope we could figure out a
> reasonable solution.
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
More information about the llvm-dev