[PATCH] D116367: [ELF][LTO] --thinlto-index-only: call madvise(MADV_DONTNEED)

Wed Dec 29 10:44:55 PST 2021

MaskRay added a comment.

In D116367#3212805 <https://reviews.llvm.org/D116367#3212805>, @tejohnson wrote:

> In D116367#3212369 <https://reviews.llvm.org/D116367#3212369>, @MaskRay wrote:
>
>> This can be used by in-process LTO. I haven't tested it yet.
>
> Would it be better to try unconditionally marking all/only the bitcode file memory buffers as MADV_DONTNEED?

(Uploaded the previous diff before seeing your comment.)

Agree. Bitcode files can be completely MADV_DONTNEED even in non --thinlto-index-only mode.
For `ELFFileBase` files, their section and symbol names are used in non --thinlto-index-only mode and nearly unused in --thinlto-index-only mode.

> For the large thin link this change is very close to the same peak RSS reduction as with my change to destroy the buffer objects completely:
> Head: 10.7G
> Destroy: 8.9G
> Don'tNeed: 9.1G
>
> However, when I tried my earlier version which unconditionally freed all the memory buffers, I found that the memory buffer name is used when creating the output indices, which I fixed by saving the name in the InputFile object. Would not doing so with this change cause a time degradation because we will need to pull all these pages back in to get the name from the buffers?

The MemoryBuffer identifier is stored separated from the buffer content. In my coarse estimate yesterday, I haven't found any time degradation.

I will benchmark this by running more times.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116367/new/

https://reviews.llvm.org/D116367