[llvm] [ThinLTO] Properly support targets that require importing all external functions (PR #133588)

[Shilei Tian via llvm-commits]

shiltian wrote:

> Looks like this flag was added several years ago for AMDGPU (https://reviews.llvm.org/D99683). Do you know why it was sufficient then but not now?

I'm not sure if ThinLTO actually worked on AMDGPU back then. At least, it doesn't seem to be working at the moment. I'd assume @yxsamliu might have more insight on this.

> While this was originally added for AMDGPU I'm concerned about changing the linkage type as other use cases of this option may have appeared, and this is inconsistent with the linkage type normally used for ThinLTO importing, and would result in linker multiply defined errors in the usual case.

Hmm, that's a valid concern, but as far as I can tell, unless someone explicitly sets this flag, it's only enabled for AMDGPU. I'm not sure if there are any other targets that support global variable object linking but not function object linking.

> If you aren't able to link objects, why not just use full LTO, which is what this essentially becomes?

We have full LTO enabled by default in GPU RDC mode, but it's extremely expensive. If the entire linked module ends up as a single SCC, then there's really no benefit to using ThinLTO. However, if there are multiple SCCs, ThinLTO can be quite beneficial. There are also other related efforts, like module splitting, but some benchmark results contributed by the community suggest that ThinLTO can still outperform full LTO by up to 9X.

https://github.com/llvm/llvm-project/pull/133588