[llvm] [IR][AtomicExpand] Add elementwise modifier to atomicrmw; automatically expand for all targets (PR #189517)

[Matt Arsenault via llvm-commits]

================
@@ -2475,13 +2475,33 @@ class LLVM_ABI TargetLoweringBase {
   }
 
   /// Returns how the IR-level AtomicExpand pass should expand the given
-  /// AtomicRMW, if at all. Default is to never expand.
+  /// AtomicRMW, if at all. Default is to never expand scalar atomics and expand
+  /// FP atomics via CmpXChg.
+  ///
+  /// Precondition: \p RMW is not elementwise. Elementwise atomicrmw
+  /// instructions are routed through \c shouldExpandAtomicRMWElementwiseInIR.
   virtual AtomicExpansionKind
   shouldExpandAtomicRMWInIR(const AtomicRMWInst *RMW) const {
     return RMW->isFloatingPointOperation() ?
       AtomicExpansionKind::CmpXChg : AtomicExpansionKind::None;
   }
 
+  /// Returns whether the IR-level AtomicExpand pass should expand the given
+  /// elementwise AtomicRMW into per-lane scalar atomicrmw instructions.
+  ///
+  /// Returning \c true (the default) tells AtomicExpand to first try to
+  /// conservatively drop the elementwise modifier and reuse an existing
+  /// whole-value atomicrmw lowering. If that is not possible, it scalarizes
+  /// into per-lane scalar atomicrmw instructions that are each fed back
+  /// through the normal atomic expansion pipeline.
+  ///
+  /// Targets that support native vector atomic instructions should return
+  /// \c false to preserve the elementwise atomicrmw for the backend.
+  virtual bool
+  shouldExpandAtomicRMWElementwiseInIR(const AtomicRMWInst *RMW) const {
+    return true;
+  }
----------------
arsenm wrote:

I'd rather not add this. The existing shouldExpandAtomic* hooks per operation are annoying enough as it is. This one already exists for AtomicRMW, the implementation can just check the elementwise bit 

https://github.com/llvm/llvm-project/pull/189517