[llvm] [AArch64][SVE] Select non-temporal instructions for unpredicated loads/stores with the nontemporal flag (PR #171261)

Ricardo Jesus via llvm-commits llvm-commits at lists.llvm.org
Thu Dec 11 06:51:45 PST 2025


================
@@ -3061,29 +3061,22 @@ let Predicates = [HasSVE_or_SME] in {
 
   multiclass unpred_store<PatFrag Store, ValueType Ty, Instruction RegRegInst,
                           Instruction RegImmInst, Instruction PTrue,
-                          ComplexPattern AddrCP> {
-    let AddedComplexity = 1 in {
+                          ComplexPattern AddrCP, int AddedComplexity = 0>  {
+    let AddedComplexity = !add(1, AddedComplexity) in {
       def _reg : Pat<(Store Ty:$val, (AddrCP GPR64sp:$base, GPR64:$offset)),
                      (RegRegInst ZPR:$val, (PTrue 31), GPR64sp:$base, GPR64:$offset)>;
     }
-    let AddedComplexity = 2 in {
+    let AddedComplexity = !add(2, AddedComplexity) in {
       def _imm : Pat<(Store Ty:$val, (am_sve_indexed_s4 GPR64sp:$base, simm4s1:$offset)),
                      (RegImmInst ZPR:$val, (PTrue 31), GPR64sp:$base, simm4s1:$offset)>;
     }
----------------
rj-jesus wrote:

It's up to you, but I'd suggest only moving the reg+imm patterns above the reg+reg ones for now:
```diff
   multiclass unpred_store<PatFrag Store, ValueType Ty, Instruction RegRegInst,
                           Instruction RegImmInst, Instruction PTrue,
                           ComplexPattern AddrCP> {
-    let AddedComplexity = 1 in {
-      def _reg : Pat<(Store Ty:$val, (AddrCP GPR64sp:$base, GPR64:$offset)),
-                     (RegRegInst ZPR:$val, (PTrue 31), GPR64sp:$base, GPR64:$offset)>;
-    }
     let AddedComplexity = 2 in {
       def _imm : Pat<(Store Ty:$val, (am_sve_indexed_s4 GPR64sp:$base, simm4s1:$offset)),
                      (RegImmInst ZPR:$val, (PTrue 31), GPR64sp:$base, simm4s1:$offset)>;
     }
+    let AddedComplexity = 1 in {
+      def _reg : Pat<(Store Ty:$val, (AddrCP GPR64sp:$base, GPR64:$offset)),
+                     (RegRegInst ZPR:$val, (PTrue 31), GPR64sp:$base, GPR64:$offset)>;
+    }
 
     def : Pat<(Store Ty:$val, GPR64:$base),
               (RegImmInst ZPR:$val, (PTrue 31), GPR64:$base, (i64 0))>;
```
This should be strictly a non-functional change as far as the existing patterns are concerned, but should allow you to bump the complexity of the new nontemporal patterns directly as I'd previously suggested.

In the future, we can remove the `AddedComplexity` from `unpred_load`/`unpred_store`, but that could be done in its own separate change.

If you prefer, we can also refactor `unpred_load`/`unpred_store` first (in its own PR), and then build on top of those changes here.

https://github.com/llvm/llvm-project/pull/171261


More information about the llvm-commits mailing list