[PATCH] Don't ever call materializeAllPermanently during LTO

[Duncan P. N. Exon Smith]

> On 2014-Oct-24, at 12:56, Rafael Espíndola <rafael.espindola at gmail.com> wrote:
> 
>>> There is still a *lot* of work to make it possible to lazily read anything else.
>> 
>> Sure is.
> 
> BTW, are you going to be in the dev meeting? I would love to discuss
> some ideas.

Yup, see you next week.

> Just so I don't forget, I will write down what I have in
> mind. I haven't put a lot of though on it recently, so some of the
> ideas might be wrong, but in any case:
> 
> Given
> 
> struct foo {};
> void f(foo *x) {}
> 
> We currently produce
> 
> !llvm.dbg.cu = !{!0}
> !0 = metadata !{....metadata !3....} ; [ DW_TAG_compile_unit ]
> !3 = metadata !{metadata !4} ; TempRetainTypes
> !4 = metadata !{.... metadata !"_ZTS3foo"} ; [ DW_TAG_structure_type ]
> 
> It is my understanding that debug info for types is a significant part
> of the total size, which is why breaking the cycles and enabling
> merging was a good memory saving.
> 
> Given that and your proposal for having dedicated types for debug
> info, we could maybe drop the explicit type list and instead have
> 
> $_ZTS3foo = DebugStructureType {....}
> 
> These can now be merged by name and we wouldn't even read a second
> copy once we have the first one.

Yup, something like this isn't hard.

> Another possible improvement would be to make the Module own the
> metadata instead of it being owned by the context. With this the
> lib/Linker would be the one responsible for explicitly copying it and
> could avoid copying unused bits. The blocking issue I know about this
> is that ld64 opens every IR file upfront, depending on the implicit
> merging that is currently done. This could be hacked (support both
> ownership modes), but hopefully it is not too hard to change ld64 to
> merge IR files incrementally.

I like this direction.  I was halfway through writing an email to you
when you sent this my way (you've mentioned this before).  It's easy to
throw an `LLVMContext` into `LTOModule` and make this happen without
changing the C API.

Is ld64 really the only blocker?

> And once Module owns the debug info, DebugStructureType (and other
> types if profitable), could get an isMaterializable method just like
> Function has right now. This would put us in a situation where we only
> ready a DebugStructureType if it is needed in the merged module.
> 
> And finally, the pie in the sky idea comes from the realization that
> we don't use some of the debug info until we get to codegen. Given
> that, we could support lazy loading from multiple .bc files. When
> merging the modules we would just keep a note saying that $_ZTS3foo
> can be found in foo.bc and only during codegen would we actually read
> it.
> 
> Cheers,
> Rafael