[llvm] [LTO] Add a hook to customize the optimization pipeline (PR #71268)

[Igor Kudrin via llvm-commits]

https://github.com/igorkudrin created https://github.com/llvm/llvm-project/pull/71268

The hook allows the calling code to register instrumentation callbacks for the LTO optimization pipeline. In particular, a custom pass filter can be added to skip certain passes from the default pipeline.

>From 86ce9021316a02634bd14dd97b696202e497fe1a Mon Sep 17 00:00:00 2001
From: Igor Kudrin <ikudrin at accesssoftek.com>
Date: Fri, 3 Nov 2023 20:26:13 -0700
Subject: [PATCH] [LTO] Add a hook to customize the optimization pipeline

The hook allows the calling code to register instrumentation callbacks
for the LTO optimization pipeline. In particular, a custom pass filter
can be added to skip certain passes from the default pipeline.
---
 llvm/include/llvm/LTO/Config.h | 5 +++++
 llvm/lib/LTO/LTOBackend.cpp    | 2 ++
 2 files changed, 7 insertions(+)

diff --git a/llvm/include/llvm/LTO/Config.h b/llvm/include/llvm/LTO/Config.h
index 6fb55f1cf1686a5..fd22c17bc12a38c 100644
--- a/llvm/include/llvm/LTO/Config.h
+++ b/llvm/include/llvm/LTO/Config.h
@@ -257,6 +257,11 @@ struct Config {
       const DenseSet<GlobalValue::GUID> &GUIDPreservedSymbols)>;
   CombinedIndexHookFn CombinedIndexHook;
 
+  /// This hook is called when the optimization pipeline is being built.
+  using CustomizeOptPipelineHookFn =
+      std::function<void(PassInstrumentationCallbacks &)>;
+  CustomizeOptPipelineHookFn CustomizeOptPipeline;
+
   /// This is a convenience function that configures this Config object to write
   /// temporary files named after the given OutputFileName for each of the LTO
   /// phases to disk. A client can use this function to implement -save-temps.
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index ccc4276e36dacf0..7e1b35319a71d59 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -265,6 +265,8 @@ static void runNewPMPasses(const Config &Conf, Module &Mod, TargetMachine *TM,
   ModuleAnalysisManager MAM;
 
   PassInstrumentationCallbacks PIC;
+  if (Conf.CustomizeOptPipeline)
+    Conf.CustomizeOptPipeline(PIC);
   StandardInstrumentations SI(Mod.getContext(), Conf.DebugPassManager,
                               Conf.VerifyEach);
   SI.registerCallbacks(PIC, &MAM);