[PATCH] D23966: ThinLTO: sort inputs and schedule by decreasing size

[Mehdi AMINI via llvm-commits]

mehdi_amini added a comment.

In https://reviews.llvm.org/D23966#527396, @tejohnson wrote:

> Code changes look fine. But what is the impact on memory? I wonder if this will bloat the peak memory because more of the large modules will get run in parallel.


The peak memory when linking clang with 4 threads goes up from 1.65GB to 2.22GB with this patch.

This makes me wonder if we could have a smarter scheduler that would balance the inputs to interleave the large ones with as many small one as needed.
Maybe another concern could be a smart scheduler for "locality" (if A imports from B and B imports from A, schedule them back-to-back as the files are more likely to be mapped in memory). But this seems quite secondary.

This patch was motivated by a use case (llvm-tblgen maybe?) where a single large file was taking almost as long as *all* the others to process and was scheduled late. The link time improvement was really important in this case.


https://reviews.llvm.org/D23966