[LLVMdev] "Ran out of registers during register allocation" bug affecting ffmpeg

[Jakob Stoklund Olesen]

On Aug 21, 2010, at 5:27 PM, Eli Friedman wrote:

> See http://llvm.org/bugs/show_bug.cgi?id=4668 and
> http://llvm.org/bugs/show_bug.cgi?id=5010.  The basic description of
> the issue (from http://llvm.org/bugs/show_bug.cgi?id=4668#c5): "The
> fundamental problem is we can't spill a register once it's fixed to a
> physical register."
> 
>> From discussion on IRC:
> 
> [17:14]	<_sabre_>	efriedma: sounds like a RA bug in linscan
> [17:14]	<_sabre_>	unfortunately, it is a fairly well known bug that is
> difficult to fix
> [17:14]	<_sabre_>	it would be worth bringing up on llvmdev, there may
> be a good approach since I stopped looking at the regalloc
> [17:14]	<_sabre_>	in any case, its worth surfacing the issue so that
> Jakob can keep it in mind as he's building the new RA infrastructure


I am not sure I completely understand the problem here since the PRs don't reproduce on TOT, but it sounds like a physical register is live across an inline asm that needs all the registers.

This can happen if the register coalescer decides to coalesce a physreg with a virtreg that is live across the inline asm. This is not easy to detect without a bad compile time regression.

I suppose something similar could happen for calls. For instance, some calls clobber all XMM registers, so a physical XMM register coalesced to be live across a call would be unspillable.

What can I say? Physreg coalescing is evil ;-)

I want to remove physreg coalescing entirely, but it requires the register allocator to be really good at taking hints. We are not quite there yet.

A quick fix would be to disable physreg coalescing for functions containing inline asm.

/jakob