[all-commits] [llvm/llvm-project] 9ecb67: [DTLTO] Overlap temporary file removal (#209423)

[Slava Zakharin via All-commits]

  Branch: refs/heads/release/23.x
  Home:   https://github.com/llvm/llvm-project
  Commit: 9ecb67b7bcc70d7060915c8778580806c90d3eb4
      https://github.com/llvm/llvm-project/commit/9ecb67b7bcc70d7060915c8778580806c90d3eb4
  Author: Ben Dunbobbin <Ben.Dunbobbin at sony.com>
  Date:   2026-07-31 (Fri, 31 Jul 2026)

  Changed paths:
    M lld/COFF/Driver.cpp
    M lld/COFF/LTO.cpp
    M lld/COFF/LTO.h
    M lld/COFF/SymbolTable.cpp
    M lld/COFF/SymbolTable.h
    M lld/ELF/Config.h
    M lld/ELF/Driver.cpp
    M lld/ELF/LTO.cpp
    M lld/ELF/LTO.h
    M llvm/include/llvm/DTLTO/DTLTO.h
    M llvm/include/llvm/LTO/LTO.h
    M llvm/lib/DTLTO/DTLTO.cpp
    M llvm/tools/llvm-lto2/llvm-lto2.cpp

  Log Message:
  -----------
  [DTLTO] Overlap temporary file removal (#209423)

Deleting the temporary files produced by the DTLTO pipeline can be
expensive on Windows hosts. For a Clang link (Debug build with
sanitizers and instrumentation) using an optimized toolchain (PGO
non-LTO, llvmorg-22.1.0) on a Windows 11 Pro (Build 26200), AMD Family
25 @ ~4.5 GHz, 16 cores/32 threads, 64 GB RAM machine, the mean duration
of the "Remove DTLTO temporary files" time trace scope was 1267.789 ms
(measured over 10 runs).

This patch performs the deletions on a background thread, allowing them
to overlap with the tail of the link to hide this cost.

This is a re-implementation of the asynchronous cleanup idea from
https://github.com/llvm/llvm-project/pull/186988, which had to be
reverted in https://github.com/llvm/llvm-project/pull/189043 because
cleanup was not guaranteed to complete before LLD invoked
timeTraceProfilerCleanup(). In certain cases timeTraceProfilerCleanup()
was called before temporary file deletion had completed in LLD, which
caused memory leaks that were flagged by sanitizer builds.

Note that the DTLTO implementation has been refactored heavily since
https://github.com/llvm/llvm-project/pull/186988, so this is a
re-implementation along the same lines.

To solve the ordering issue, LLD now calls a hook that defaults to a
no-op, and DTLTO overrides it to drain the background deletion work.
LLD calls this hook before time-trace write/cleanup.

(cherry picked from commit 914d5a1841dedf2148e21bd6de3b92409fe79285)


  Commit: d62f721d6eb069b9444c1534fc371f677da35994
      https://github.com/llvm/llvm-project/commit/d62f721d6eb069b9444c1534fc371f677da35994
  Author: Slava Zakharin <szakharin at nvidia.com>
  Date:   2026-07-31 (Fri, 31 Jul 2026)

  Changed paths:
    M llvm/lib/DTLTO/CMakeLists.txt

  Log Message:
  -----------
  [CMake] Added missing LLVM_PTHREAD_LIB dependency. (#211041)

Required after #209423.

(cherry picked from commit 4f5675a0500f9ccc60dcbabb57e1c4dc88c40a84)


Compare: https://github.com/llvm/llvm-project/compare/1e1225eb1f77...d62f721d6eb0

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications