[llvm] [Passes] Always run LowerMatrixIntrinsics, request EarlyCSE on demand. (PR #118507)

via llvm-commits llvm-commits at lists.llvm.org
Tue Dec 3 08:09:02 PST 2024 

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-llvm-transforms

Author: Florian Hahn (fhahn)

<details>
<summary>Changes</summary>

This patch enables lowering matrix intrinsics by default. Without this, trying to code-gen modules with matrix intrinsics will crash during instruction selection. Enabling this by default is more convenient to users and also makes things nicely work with LTO and other frontend using matrix intrinsics.

In the current stage, it is a bit of a request for comment on how to proceed.

When there are no matrix intrinsics, the extra cost of running the pass is a single iteration over all instructions in a function. Except for the O0 case, the compile-time impact is negligible. Ideally we would just check if there are any matrix intrinsics defined in the module, but the problem is they are all overloaded. Maybe there's a cheap way to check if any matrix intrinsics are present I missed?

As for the O0 compile-time impact, it would probably be possible to naively expand the intrinsics in SelectionDAGBuilder with -O0.
 
Overall compile-time impact: stage1-O3 +0.01%, stage1-ReleaseThinLTO +0.02%, stage1-ReleaseLTO-g: +0.02%, stage1-ReleaseLTO-g +0.16%, stage2-O3 -0.01%, stage2-O0-g +0.10%.
https://llvm-compile-time-tracker.com/compare.php?from=8df7be7032dd0818c0bf3265949039c13864cfb4&to=260aaa43308e27549353229afa9792f157cdc929&stat=instructions:u

Depends on https://github.com/llvm/llvm-project/pull/118323


---
Full diff: https://github.com/llvm/llvm-project/pull/118507.diff


14 Files Affected:

- (modified) llvm/include/llvm/Transforms/Scalar/LowerMatrixIntrinsics.h (+10) 
- (added) llvm/include/llvm/Transforms/Utils/ExtraPassManager.h (+56) 
- (modified) llvm/include/llvm/Transforms/Vectorize/LoopVectorize.h (+8-32) 
- (modified) llvm/lib/Passes/PassBuilder.cpp (+1) 
- (modified) llvm/lib/Passes/PassBuilderPipelines.cpp (+8-12) 
- (modified) llvm/lib/Passes/PassRegistry.def (+2) 
- (modified) llvm/lib/Transforms/Scalar/LowerMatrixIntrinsics.cpp (+4) 
- (modified) llvm/lib/Transforms/Vectorize/LoopVectorize.cpp (+1) 
- (added) llvm/test/Other/cse-after-matrix-lowering.ll (+17) 
- (modified) llvm/test/Other/new-pm-O0-defaults.ll (+4-5) 
- (modified) llvm/test/Other/new-pm-defaults.ll (+1-6) 
- (modified) llvm/test/Other/new-pm-thinlto-postlink-defaults.ll (+1) 
- (modified) llvm/test/Other/new-pm-thinlto-postlink-pgo-defaults.ll (+1) 
- (modified) llvm/test/Other/new-pm-thinlto-postlink-samplepgo-defaults.ll (+1) 


``````````diff
diff --git a/llvm/include/llvm/Transforms/Scalar/LowerMatrixIntrinsics.h b/llvm/include/llvm/Transforms/Scalar/LowerMatrixIntrinsics.h
index d44d297dd4ffc7..e95fbc1ffcdef2 100644
--- a/llvm/include/llvm/Transforms/Scalar/LowerMatrixIntrinsics.h
+++ b/llvm/include/llvm/Transforms/Scalar/LowerMatrixIntrinsics.h
@@ -14,6 +14,7 @@
 #define LLVM_TRANSFORMS_SCALAR_LOWERMATRIXINTRINSICS_H
 
 #include "llvm/IR/PassManager.h"
+#include "llvm/Transforms/Utils/ExtraPassManager.h"
 
 namespace llvm {
 class LowerMatrixIntrinsicsPass
@@ -27,6 +28,15 @@ class LowerMatrixIntrinsicsPass
                      function_ref<StringRef(StringRef)> MapClassName2PassName);
   static bool isRequired() { return true; }
 };
+
+/// A marker analyss to determine if extra passes should be run after lowering
+/// matrix intrinsics.
+struct ShouldRunExtraMatrixPasses
+    : public ShouldRunExtraPasses<ShouldRunExtraMatrixPasses>,
+      public AnalysisInfoMixin<ShouldRunExtraMatrixPasses> {
+  static AnalysisKey Key;
+};
+
 } // namespace llvm
 
 #endif
diff --git a/llvm/include/llvm/Transforms/Utils/ExtraPassManager.h b/llvm/include/llvm/Transforms/Utils/ExtraPassManager.h
new file mode 100644
index 00000000000000..c09a0dfd7753ab
--- /dev/null
+++ b/llvm/include/llvm/Transforms/Utils/ExtraPassManager.h
@@ -0,0 +1,56 @@
+//===- ExtraPassManager.h - Loop pass management -----------------*- C++
+//-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file
+///
+/// This file provides a pass manager that only runs its passes if the
+/// provided marker analysis has been preserved, together with a class to
+/// define such a marker analysis.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TRANSFORMS_UTILS_EXTRAPASSMANAGER_H
+#define LLVM_TRANSFORMS_UTILS_EXTRAPASSMANAGER_H
+
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+
+/// A marker analysis to determine if extra passes should be run on demand.
+/// Passes requesting extra transformations to run need to request and preserve
+/// this analysis.
+template <typename MarkerTy> struct ShouldRunExtraPasses {
+  struct Result {
+    bool invalidate(Function &F, const PreservedAnalyses &PA,
+                    FunctionAnalysisManager::Invalidator &) {
+      // Check whether the analysis has been explicitly invalidated. Otherwise,
+      // it remains preserved.
+      auto PAC = PA.getChecker<MarkerTy>();
+      return !PAC.preservedWhenStateless();
+    }
+  };
+
+  Result run(Function &F, FunctionAnalysisManager &FAM) { return Result(); }
+};
+
+/// A pass manager to run a set of extra function passes if the
+/// ShouldRunExtraPasses marker analysis is present. This allows passes to
+/// request additional transformations on demand. An example is extra
+/// simplifications after loop-vectorization, if runtime checks have been added.
+template <typename MarkerTy>
+struct ExtraPassManager : public FunctionPassManager {
+  PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM) {
+    auto PA = PreservedAnalyses::all();
+    if (AM.getCachedResult<MarkerTy>(F))
+      PA.intersect(FunctionPassManager::run(F, AM));
+    PA.abandon<MarkerTy>();
+    return PA;
+  }
+};
+} // namespace llvm
+
+#endif // LLVM_TRANSFORMS_UTILS_EXTRAPASSMANAGER_H
diff --git a/llvm/include/llvm/Transforms/Vectorize/LoopVectorize.h b/llvm/include/llvm/Transforms/Vectorize/LoopVectorize.h
index 24b6354662955e..e086ad2ad4abf9 100644
--- a/llvm/include/llvm/Transforms/Vectorize/LoopVectorize.h
+++ b/llvm/include/llvm/Transforms/Vectorize/LoopVectorize.h
@@ -58,6 +58,7 @@
 
 #include "llvm/IR/PassManager.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Transforms/Utils/ExtraPassManager.h"
 #include <functional>
 
 namespace llvm {
@@ -80,38 +81,6 @@ class TargetTransformInfo;
 extern cl::opt<bool> EnableLoopInterleaving;
 extern cl::opt<bool> EnableLoopVectorization;
 
-/// A marker to determine if extra passes after loop vectorization should be
-/// run.
-struct ShouldRunExtraVectorPasses
-    : public AnalysisInfoMixin<ShouldRunExtraVectorPasses> {
-  static AnalysisKey Key;
-  struct Result {
-    bool invalidate(Function &F, const PreservedAnalyses &PA,
-                    FunctionAnalysisManager::Invalidator &) {
-      // Check whether the analysis has been explicitly invalidated. Otherwise,
-      // it remains preserved.
-      auto PAC = PA.getChecker<ShouldRunExtraVectorPasses>();
-      return !PAC.preservedWhenStateless();
-    }
-  };
-
-  Result run(Function &F, FunctionAnalysisManager &FAM) { return Result(); }
-};
-
-/// A pass manager to run a set of extra function simplification passes after
-/// vectorization, if requested. LoopVectorize caches the
-/// ShouldRunExtraVectorPasses analysis to request extra simplifications, if
-/// they could be beneficial.
-struct ExtraVectorPassManager : public FunctionPassManager {
-  PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM) {
-    auto PA = PreservedAnalyses::all();
-    if (AM.getCachedResult<ShouldRunExtraVectorPasses>(F))
-      PA.intersect(FunctionPassManager::run(F, AM));
-    PA.abandon<ShouldRunExtraVectorPasses>();
-    return PA;
-  }
-};
-
 struct LoopVectorizeOptions {
   /// If false, consider all loops for interleaving.
   /// If true, only loops that explicitly request interleaving are considered.
@@ -201,6 +170,13 @@ void reportVectorizationFailure(const StringRef DebugMsg,
     const StringRef OREMsg, const StringRef ORETag,
     OptimizationRemarkEmitter *ORE, Loop *TheLoop, Instruction *I = nullptr);
 
+/// A marker analyss to determine if extra passes should be run after loop
+/// vectorization.
+struct ShouldRunExtraVectorPasses
+    : public ShouldRunExtraPasses<ShouldRunExtraVectorPasses>,
+      public AnalysisInfoMixin<ShouldRunExtraVectorPasses> {
+  static AnalysisKey Key;
+};
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_VECTORIZE_LOOPVECTORIZE_H
diff --git a/llvm/lib/Passes/PassBuilder.cpp b/llvm/lib/Passes/PassBuilder.cpp
index ba52a37df9c25d..6d914c1557c229 100644
--- a/llvm/lib/Passes/PassBuilder.cpp
+++ b/llvm/lib/Passes/PassBuilder.cpp
@@ -310,6 +310,7 @@
 #include "llvm/Transforms/Utils/DXILUpgrade.h"
 #include "llvm/Transforms/Utils/Debugify.h"
 #include "llvm/Transforms/Utils/EntryExitInstrumenter.h"
+#include "llvm/Transforms/Utils/ExtraPassManager.h"
 #include "llvm/Transforms/Utils/FixIrreducible.h"
 #include "llvm/Transforms/Utils/HelloWorld.h"
 #include "llvm/Transforms/Utils/IRNormalizer.h"
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp b/llvm/lib/Passes/PassBuilderPipelines.cpp
index 5a7c327de95870..3c6c7fe952e048 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -134,6 +134,7 @@
 #include "llvm/Transforms/Utils/CanonicalizeAliases.h"
 #include "llvm/Transforms/Utils/CountVisits.h"
 #include "llvm/Transforms/Utils/EntryExitInstrumenter.h"
+#include "llvm/Transforms/Utils/ExtraPassManager.h"
 #include "llvm/Transforms/Utils/InjectTLIMappings.h"
 #include "llvm/Transforms/Utils/LibCallsShrinkWrap.h"
 #include "llvm/Transforms/Utils/Mem2Reg.h"
@@ -269,10 +270,6 @@ static cl::opt<bool> EnableOrderFileInstrumentation(
     "enable-order-file-instrumentation", cl::init(false), cl::Hidden,
     cl::desc("Enable order file instrumentation (default = off)"));
 
-static cl::opt<bool>
-    EnableMatrix("enable-matrix", cl::init(false), cl::Hidden,
-                 cl::desc("Enable lowering of the matrix intrinsics"));
-
 static cl::opt<bool> EnableConstraintElimination(
     "enable-constraint-elimination", cl::init(true), cl::Hidden,
     cl::desc(
@@ -1307,7 +1304,7 @@ void PassBuilder::addVectorPasses(OptimizationLevel Level,
   FPM.addPass(InstCombinePass());
 
   if (Level.getSpeedupLevel() > 1 && ExtraVectorizerPasses) {
-    ExtraVectorPassManager ExtraPasses;
+    ExtraPassManager<ShouldRunExtraVectorPasses> ExtraPasses;
     // At higher optimization levels, try to clean up any runtime overlap and
     // alignment checks inserted by the vectorizer. We want to track correlated
     // runtime checks for two inner loops in the same outer loop, fold any
@@ -1488,10 +1485,10 @@ PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
   OptimizePM.addPass(Float2IntPass());
   OptimizePM.addPass(LowerConstantIntrinsicsPass());
 
-  if (EnableMatrix) {
-    OptimizePM.addPass(LowerMatrixIntrinsicsPass());
-    OptimizePM.addPass(EarlyCSEPass());
-  }
+  OptimizePM.addPass(LowerMatrixIntrinsicsPass());
+  ExtraPassManager<ShouldRunExtraMatrixPasses> ExtraPasses;
+  ExtraPasses.addPass(EarlyCSEPass());
+  OptimizePM.addPass(std::move(ExtraPasses));
 
   // CHR pass should only be applied with the profile information.
   // The check is to check the profile summary information in CHR.
@@ -2188,9 +2185,8 @@ PassBuilder::buildO0DefaultPipeline(OptimizationLevel Level,
   if (PTO.MergeFunctions)
     MPM.addPass(MergeFunctionsPass());
 
-  if (EnableMatrix)
-    MPM.addPass(
-        createModuleToFunctionPassAdaptor(LowerMatrixIntrinsicsPass(true)));
+  MPM.addPass(
+      createModuleToFunctionPassAdaptor(LowerMatrixIntrinsicsPass(true)));
 
   if (!CGSCCOptimizerLateEPCallbacks.empty()) {
     CGSCCPassManager CGPM;
diff --git a/llvm/lib/Passes/PassRegistry.def b/llvm/lib/Passes/PassRegistry.def
index 7c3798f6462a46..860e58b50e96e9 100644
--- a/llvm/lib/Passes/PassRegistry.def
+++ b/llvm/lib/Passes/PassRegistry.def
@@ -304,6 +304,8 @@ FUNCTION_ANALYSIS("scalar-evolution", ScalarEvolutionAnalysis())
 FUNCTION_ANALYSIS("should-not-run-function-passes",
                   ShouldNotRunFunctionPassesAnalysis())
 FUNCTION_ANALYSIS("should-run-extra-vector-passes",
+                  ShouldRunExtraMatrixPasses())
+FUNCTION_ANALYSIS("should-run-extra-matrix-passes",
                   ShouldRunExtraVectorPasses())
 FUNCTION_ANALYSIS("ssp-layout", SSPLayoutAnalysis())
 FUNCTION_ANALYSIS("stack-safety-local", StackSafetyAnalysis())
diff --git a/llvm/lib/Transforms/Scalar/LowerMatrixIntrinsics.cpp b/llvm/lib/Transforms/Scalar/LowerMatrixIntrinsics.cpp
index 6a9ec48864b2c5..e0f077eac92342 100644
--- a/llvm/lib/Transforms/Scalar/LowerMatrixIntrinsics.cpp
+++ b/llvm/lib/Transforms/Scalar/LowerMatrixIntrinsics.cpp
@@ -130,6 +130,8 @@ auto m_AnyAdd(const LTy &L, const RTy &R) {
   return m_CombineOr(m_Add(L, R), m_FAdd(L, R));
 }
 
+AnalysisKey ShouldRunExtraMatrixPasses::Key;
+
 namespace {
 
 // Given an element pointer \p BasePtr to the start of a (sub) matrix, compute
@@ -2631,6 +2633,8 @@ PreservedAnalyses LowerMatrixIntrinsicsPass::run(Function &F,
     if (!Minimal) {
       PA.preserve<LoopAnalysis>();
       PA.preserve<DominatorTreeAnalysis>();
+      AM.getResult<ShouldRunExtraMatrixPasses>(F);
+      PA.preserve<ShouldRunExtraMatrixPasses>();
     }
     return PA;
   }
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index 90312c1a28df3c..2b98665fc8b77b 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -134,6 +134,7 @@
 #include "llvm/Support/NativeFormatting.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Transforms/Utils/BasicBlockUtils.h"
+#include "llvm/Transforms/Utils/ExtraPassManager.h"
 #include "llvm/Transforms/Utils/InjectTLIMappings.h"
 #include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Utils/LoopSimplify.h"
diff --git a/llvm/test/Other/cse-after-matrix-lowering.ll b/llvm/test/Other/cse-after-matrix-lowering.ll
new file mode 100644
index 00000000000000..66ea3837d7c119
--- /dev/null
+++ b/llvm/test/Other/cse-after-matrix-lowering.ll
@@ -0,0 +1,17 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -disable-verify -verify-analysis-invalidation=0 -eagerly-invalidate-analyses=0 -debug-pass-manager \
+; RUN:     -passes='default<O3>' -S %s 2>&1 | FileCheck %s
+
+define <4 x float> @multiply_2x2(<4 x float> %a, <4 x float> %b) {
+; CHECK:      Running pass: LowerMatrixIntrinsicsPass
+; CHECK-NEXT: Running analysis: ShouldRunExtraPasses
+; CHECK-NEXT: Invalidating analysis: PostDominatorTreeAnalysis
+; CHECK-NEXT: Running pass: EarlyCSEPass
+; CHECK-NEXT: Invalidating analysis: ShouldRunExtraPasses
+;
+entry:
+  %c = call <4 x float> @llvm.matrix.multiply.v4f32.v4f32.v4f32(<4 x float> %a, <4 x float> %b, i32 2, i32 2, i32 2)
+  ret <4 x float> %c
+}
+
+declare <4 x float> @llvm.matrix.multiply.v4f32.v4f32.v4f32(<4 x float>, <4 x float>, i32, i32, i32)
diff --git a/llvm/test/Other/new-pm-O0-defaults.ll b/llvm/test/Other/new-pm-O0-defaults.ll
index e8131ac7fab45a..59b91dfdc308b4 100644
--- a/llvm/test/Other/new-pm-O0-defaults.ll
+++ b/llvm/test/Other/new-pm-O0-defaults.ll
@@ -10,9 +10,6 @@
 ; RUN: opt -disable-verify -verify-analysis-invalidation=0 -debug-pass-manager \
 ; RUN:     -passes='default<O0>' -S %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefixes=CHECK,CHECK-DEFAULT,CHECK-CORO
-; RUN: opt -disable-verify -verify-analysis-invalidation=0 -debug-pass-manager -enable-matrix \
-; RUN:     -passes='default<O0>' -S %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefixes=CHECK,CHECK-DEFAULT,CHECK-MATRIX,CHECK-CORO
 ; RUN: opt -disable-verify -verify-analysis-invalidation=0 -debug-pass-manager -debug-info-for-profiling \
 ; RUN:     -passes='default<O0>' -S %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefixes=CHECK,CHECK-DIS,CHECK-CORO
@@ -34,12 +31,14 @@
 ; CHECK-DIS-NEXT: Running pass: AddDiscriminatorsPass
 ; CHECK-DIS-NEXT: Running pass: AlwaysInlinerPass
 ; CHECK-DIS-NEXT: Running analysis: ProfileSummaryAnalysis
+; CHECK-DIS: Running pass: LowerMatrixIntrinsicsPass
+; CHECK-DIS-NEXT: Running analysis: TargetIRAnalysis
 ; CHECK-DEFAULT: Running analysis: InnerAnalysisManagerProxy
 ; CHECK-DEFAULT-NEXT: Running pass: EntryExitInstrumenterPass
 ; CHECK-DEFAULT-NEXT: Running pass: AlwaysInlinerPass
 ; CHECK-DEFAULT-NEXT: Running analysis: ProfileSummaryAnalysis
-; CHECK-MATRIX: Running pass: LowerMatrixIntrinsicsPass
-; CHECK-MATRIX-NEXT: Running analysis: TargetIRAnalysis
+; CHECK-DEFAULT: Running pass: LowerMatrixIntrinsicsPass
+; CHECK-DEFAULT-NEXT: Running analysis: TargetIRAnalysis
 ; CHECK-CORO-NEXT: Running pass: CoroConditionalWrapper
 ; CHECK-PRE-LINK: Running pass: CanonicalizeAliasesPass
 ; CHECK-PRE-LINK-NEXT: Running pass: NameAnonGlobalPass
diff --git a/llvm/test/Other/new-pm-defaults.ll b/llvm/test/Other/new-pm-defaults.ll
index 7cf035b0c6f376..8e6390dbbcab7a 100644
--- a/llvm/test/Other/new-pm-defaults.ll
+++ b/llvm/test/Other/new-pm-defaults.ll
@@ -75,10 +75,6 @@
 ; RUN:     -passes='default<O3>' -enable-jump-table-to-switch -S  %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefixes=CHECK-O,CHECK-DEFAULT,CHECK-O3,CHECK-JUMP-TABLE-TO-SWITCH,CHECK-O23SZ,%llvmcheckext
 
-; RUN: opt -disable-verify -verify-analysis-invalidation=0 -eagerly-invalidate-analyses=0 -debug-pass-manager \
-; RUN:     -passes='default<O3>' -enable-matrix -S  %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefixes=CHECK-O,CHECK-DEFAULT,CHECK-O3,CHECK-O23SZ,%llvmcheckext,CHECK-MATRIX
-
 ; RUN: opt -disable-verify -verify-analysis-invalidation=0 -eagerly-invalidate-analyses=0 -debug-pass-manager \
 ; RUN:     -passes='default<O3>' -enable-merge-functions -S  %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefixes=CHECK-O,CHECK-DEFAULT,CHECK-O3,CHECK-O23SZ,%llvmcheckext,CHECK-MERGE-FUNCS
@@ -242,8 +238,7 @@
 ; CHECK-EP-OPTIMIZER-EARLY: Running pass: NoOpModulePass
 ; CHECK-O-NEXT: Running pass: Float2IntPass
 ; CHECK-O-NEXT: Running pass: LowerConstantIntrinsicsPass on foo
-; CHECK-MATRIX: Running pass: LowerMatrixIntrinsicsPass on f
-; CHECK-MATRIX-NEXT: Running pass: EarlyCSEPass on f
+; CHECK-O-NEXT: Running pass: LowerMatrixIntrinsicsPass
 ; CHECK-O3-NEXT: Running pass: ControlHeightReductionPass
 ; CHECK-EP-VECTORIZER-START-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-EXT: Running pass: {{.*}}::Bye on foo
diff --git a/llvm/test/Other/new-pm-thinlto-postlink-defaults.ll b/llvm/test/Other/new-pm-thinlto-postlink-defaults.ll
index ed13402e1c4b15..01e5e73c9b1fe5 100644
--- a/llvm/test/Other/new-pm-thinlto-postlink-defaults.ll
+++ b/llvm/test/Other/new-pm-thinlto-postlink-defaults.ll
@@ -168,6 +168,7 @@
 ; CHECK-POST-EP-OPT-EARLY-NEXT: Running pass: NoOpModulePass
 ; CHECK-POSTLINK-O-NEXT: Running pass: Float2IntPass
 ; CHECK-POSTLINK-O-NEXT: Running pass: LowerConstantIntrinsicsPass
+; CHECK-POSTLINK-O-NEXT: Running pass: LowerMatrixIntrinsicsPass
 ; CHECK-POSTLINK-O3-NEXT: Running pass: ControlHeightReductionPass
 ; CHECK-EXT: Running pass: {{.*}}::Bye
 ; CHECK-POSTLINK-O-NEXT: Running pass: LoopSimplifyPass
diff --git a/llvm/test/Other/new-pm-thinlto-postlink-pgo-defaults.ll b/llvm/test/Other/new-pm-thinlto-postlink-pgo-defaults.ll
index c82c34f7ff01e7..b2ad9901bffaa7 100644
--- a/llvm/test/Other/new-pm-thinlto-postlink-pgo-defaults.ll
+++ b/llvm/test/Other/new-pm-thinlto-postlink-pgo-defaults.ll
@@ -151,6 +151,7 @@
 ; CHECK-O-NEXT: Running pass: RecomputeGlobalsAAPass
 ; CHECK-O-NEXT: Running pass: Float2IntPass
 ; CHECK-O-NEXT: Running pass: LowerConstantIntrinsicsPass
+; CHECK-O-NEXT: Running pass: LowerMatrixIntrinsicsPass
 ; CHECK-O3-NEXT: Running pass: ControlHeightReductionPass
 ; CHECK-O3-NEXT: Running analysis: RegionInfoAnalysis
 ; CHECK-O3-NEXT: Running analysis: DominanceFrontierAnalysis
diff --git a/llvm/test/Other/new-pm-thinlto-postlink-samplepgo-defaults.ll b/llvm/test/Other/new-pm-thinlto-postlink-samplepgo-defaults.ll
index d375747547d61f..44423e342c9f3b 100644
--- a/llvm/test/Other/new-pm-thinlto-postlink-samplepgo-defaults.ll
+++ b/llvm/test/Other/new-pm-thinlto-postlink-samplepgo-defaults.ll
@@ -160,6 +160,7 @@
 ; CHECK-O-NEXT: Running pass: RecomputeGlobalsAAPass
 ; CHECK-O-NEXT: Running pass: Float2IntPass
 ; CHECK-O-NEXT: Running pass: LowerConstantIntrinsicsPass
+; CHECK-O-NEXT: Running pass: LowerMatrixIntrinsicsPass
 ; CHECK-O3-NEXT: Running pass: ControlHeightReductionPass
 ; CHECK-O3-NEXT: Running analysis: RegionInfoAnalysis
 ; CHECK-O3-NEXT: Running analysis: DominanceFrontierAnalysis

``````````

</details>


https://github.com/llvm/llvm-project/pull/118507

More information about the llvm-commits mailing list