[all-commits] [llvm/llvm-project] b10dc2: [lld] Add caching for `--lto-partitions`

[Pierre van Houtryve via All-commits]

  Branch: refs/heads/users/pierre-vh/add-splitmodule-caching
  Home:   https://github.com/llvm/llvm-project
  Commit: b10dc27086cd23b8c935d18682456eaf6b17d895
      https://github.com/llvm/llvm-project/commit/b10dc27086cd23b8c935d18682456eaf6b17d895
  Author: pvanhout <pierre.vanhoutryve at amd.com>
  Date:   2026-09-08 (Tue, 08 Sep 2026)

  Changed paths:
    M lld/ELF/Config.h
    M lld/ELF/Driver.cpp
    M lld/ELF/LTO.cpp
    M lld/ELF/Options.td
    R lld/test/ELF/lto/cache-warnings.ll
    R lld/test/ELF/lto/cache.ll
    A lld/test/ELF/lto/lto-partitions-cache-warnings.ll
    A lld/test/ELF/lto/lto-partitions-cache.ll
    A lld/test/ELF/lto/thinlto-cache-warnings.ll
    A lld/test/ELF/lto/thinlto-cache.ll
    M llvm/include/llvm/DTLTO/DTLTO.h
    M llvm/include/llvm/LTO/LTO.h
    M llvm/include/llvm/LTO/LTOBackend.h
    M llvm/include/llvm/LTO/legacy/LTOCodeGenerator.h
    M llvm/lib/DTLTO/DTLTO.cpp
    M llvm/lib/LTO/LTO.cpp
    M llvm/lib/LTO/LTOBackend.cpp
    M llvm/lib/LTO/LTOCodeGenerator.cpp

  Log Message:
  -----------
  [lld] Add caching for `--lto-partitions`

Add an opt-in cache for the (full) LTO partitions generated via `--lto-partitions=N` when `N > 1`.
This is primarily for AMDGPU but implemented in a target-agnostic way. The goal is to avoid recompiling entire very large modules (can be hundreds of megabytes of bitcode) if someone just changed a single function or two, which don't affect most partitions.

This uses the LTO Config hash + a hash of the bitcode module itself. This is a very conservative approach, we may be able to fine-tune it to improve cache hits if it turns out the hit rate is poor.

Solves LCOMPILER-59


  Commit: 305a71d13fff0e694246f18ca4aca8919e9db1a5
      https://github.com/llvm/llvm-project/commit/305a71d13fff0e694246f18ca4aca8919e9db1a5
  Author: pvanhout <pierre.vanhoutryve at amd.com>
  Date:   2026-09-08 (Tue, 08 Sep 2026)

  Changed paths:
    M lld/ELF/Config.h
    M lld/ELF/Driver.cpp
    M lld/ELF/LTO.cpp
    M lld/ELF/Options.td
    R lld/test/ELF/lto/lto-partitions-cache-warnings.ll
    M lld/test/ELF/lto/lto-partitions-cache.ll
    M llvm/include/llvm/DTLTO/DTLTO.h
    M llvm/include/llvm/LTO/LTO.h
    M llvm/lib/DTLTO/DTLTO.cpp
    M llvm/lib/LTO/LTO.cpp

  Log Message:
  -----------
  Address comments


  Commit: 93a9429460e63ccbbd887af062539207b418e5e7
      https://github.com/llvm/llvm-project/commit/93a9429460e63ccbbd887af062539207b418e5e7
  Author: pvanhout <pierre.vanhoutryve at amd.com>
  Date:   2026-09-08 (Tue, 08 Sep 2026)

  Changed paths:
    M llvm/include/llvm/DTLTO/DTLTO.h
    M llvm/include/llvm/LTO/LTO.h
    M llvm/lib/LTO/LTO.cpp
    M llvm/lib/LTO/LTOBackend.cpp

  Log Message:
  -----------
  Comments


Compare: https://github.com/llvm/llvm-project/compare/1427c524240a...93a9429460e6

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications