[flang-commits] [flang] [flang][Lower] Add alternative real expression lowering (PR #207371)
Tom Eccles via flang-commits
flang-commits at lists.llvm.org
Mon Jul 6 09:48:44 PDT 2026
https://github.com/tblah updated https://github.com/llvm/llvm-project/pull/207371
>From 44610a2c04f6e5f2b6aec9e7255df4c38143f843 Mon Sep 17 00:00:00 2001
From: Tom Eccles <tom.eccles at arm.com>
Date: Wed, 24 Jun 2026 17:17:08 +0100
Subject: [PATCH 1/4] [flang][Lower] Add alternative real expression lowering
This is opt-in by an engineering option and disabled by default.
In section 10.1.5.2.4 of the 2023 Fortran standard "Evaluation of
numerical intrinsic operations", the standard explicitly allows
alternate mathematically equivalent lowerings. For example the source
expression X + Y + Z could be evaluated (X + Y) + Z, X + (Y + Z) or even
(X + Z) + Y, etc.
The open source benchmark SNBone shows significantly better results with
classic flang because classic flang emits real arithmetic expressions in
a different order. In the case of this benchmark it reduces dependency
depth for instructions issued to the vector unit, allowing for more of
the arithmetic to be parallelised over multiple vector execution units
in the ALU.
The lowering added by this patch tries to mimic the way classic flang
orders instructions for these expressions. I did not read any classic
flang source when writing this patch. There is still a notable
difference in that classic flang uses FMA intrinsics whereas LLVM Flang
relies on the rest of the pipeline to introduce FMA when it is safe to
do so.
This is a much less aggressive optimisation than simply enabling reassoc
in the fast-math flags because it does not allow reassociation between
Fortran language statements. This is why I implemented it in lowering.
The new option enables an experimental lowering path for scalar real
top-level addition chains. When enabled with
-enable-split-sum-expression-tree-lowering, eligible sums are split after
the first two terms and rebuilt as a right-associated tail plus head. This
lets the independent tail terms be evaluated before the assignment-related
head, giving the backend a different expression tree while leaving the
default lowering unchanged.
The transform is deliberately narrow. It only applies to scalar real RHS
expressions in assignments and rejects cases with vector subscripts,
parentheses, subtraction, procedure references, or volatile/asynchronous
symbols on either side of the assignment. Subtraction is left out because
the split would need to carry signed terms; division stays within an
individual additive term and does not change the top-level chain.
In testing I have found some tests in the Fujitsu test suite miscompare
due to small changes in floating point rounding. There are no failures
or regressions in SPEC2017 or SPEC2026. I think it would be legal
according to the Fortran standard to enable this by default, but I am not
proposing that here, and will not consider it until after the LLVM
release branch point.
I will add a compiler driver option in a separate patch because I was not
sure if people would be happy to have a user-facing option for this or
not.
Assisted-by: Codex
---
flang/include/flang/Evaluate/tools.h | 12 +
.../include/flang/Lower/ConvertExprToHLFIR.h | 7 +
flang/lib/Evaluate/tools.cpp | 61 ++
flang/lib/Lower/Bridge.cpp | 4 +-
flang/lib/Lower/ConvertExprToHLFIR.cpp | 125 +++
.../split-sum-expression-tree-lowering.f90 | 748 ++++++++++++++++++
6 files changed, 955 insertions(+), 2 deletions(-)
create mode 100644 flang/test/Lower/split-sum-expression-tree-lowering.f90
diff --git a/flang/include/flang/Evaluate/tools.h b/flang/include/flang/Evaluate/tools.h
index 4d193f135e2ad..471b8c2e7f3c1 100644
--- a/flang/include/flang/Evaluate/tools.h
+++ b/flang/include/flang/Evaluate/tools.h
@@ -1120,6 +1120,18 @@ bool HasConstant(const Expr<SomeType> &);
// Predicate: Does an expression contain a component
bool HasStructureComponent(const Expr<SomeType> &expr);
+// Predicate: does an expression contain parentheses?
+bool HasParentheses(const Expr<SomeType> &expr);
+
+// Predicate: does an expression contain a procedure reference?
+bool HasProcedureRef(const Expr<SomeType> &expr);
+
+// Predicate: does an expression contain subtraction?
+bool HasSubtract(const Expr<SomeType> &expr);
+
+// Predicate: does an expression contain a VOLATILE or ASYNCHRONOUS symbol?
+bool HasVolatileOrAsynchronousSymbol(const Expr<SomeType> &expr);
+
// Utilities for attaching the location of the declaration of a symbol
// of interest to a message. Handles the case of USE association gracefully.
parser::Message *AttachDeclaration(parser::Message &, const Symbol &);
diff --git a/flang/include/flang/Lower/ConvertExprToHLFIR.h b/flang/include/flang/Lower/ConvertExprToHLFIR.h
index 60ecd643be436..9be880c4b090a 100644
--- a/flang/include/flang/Lower/ConvertExprToHLFIR.h
+++ b/flang/include/flang/Lower/ConvertExprToHLFIR.h
@@ -41,6 +41,13 @@ convertExprToHLFIR(mlir::Location loc, Fortran::lower::AbstractConverter &,
const Fortran::lower::SomeExpr &, Fortran::lower::SymMap &,
Fortran::lower::StatementContext &);
+/// Lower an assignment RHS to HLFIR, optionally using assignment context to
+/// choose an experimental expression tree for eligible scalar real sums.
+hlfir::EntityWithAttributes convertAssignmentRhsToHLFIR(
+ mlir::Location loc, Fortran::lower::AbstractConverter &,
+ const Fortran::lower::SomeExpr &lhs, const Fortran::lower::SomeExpr &rhs,
+ Fortran::lower::SymMap &, Fortran::lower::StatementContext &);
+
inline fir::ExtendedValue translateToExtendedValue(
mlir::Location loc, fir::FirOpBuilder &builder, hlfir::Entity entity,
Fortran::lower::StatementContext &context, bool contiguityHint = false) {
diff --git a/flang/lib/Evaluate/tools.cpp b/flang/lib/Evaluate/tools.cpp
index d0548ebdfd0ea..f22ccdb4389a0 100644
--- a/flang/lib/Evaluate/tools.cpp
+++ b/flang/lib/Evaluate/tools.cpp
@@ -1321,6 +1321,67 @@ bool HasVectorSubscript(const ActualArgument &actual) {
return expr && HasVectorSubscript(*expr);
}
+namespace {
+
+struct HasParenthesesHelper : public AnyTraverse<HasParenthesesHelper> {
+ using Base = AnyTraverse<HasParenthesesHelper>;
+ HasParenthesesHelper() : Base{*this} {}
+ using Base::operator();
+ template <typename T> bool operator()(const Parentheses<T> &) const {
+ return true;
+ }
+};
+
+struct HasProcedureRefHelper : public AnyTraverse<HasProcedureRefHelper> {
+ using Base = AnyTraverse<HasProcedureRefHelper>;
+ HasProcedureRefHelper() : Base{*this} {}
+ using Base::operator();
+ bool operator()(const ProcedureRef &) const { return true; }
+};
+
+struct HasSubtractHelper : public AnyTraverse<HasSubtractHelper> {
+ using Base = AnyTraverse<HasSubtractHelper>;
+ HasSubtractHelper() : Base{*this} {}
+ using Base::operator();
+ template <typename T> bool operator()(const Subtract<T> &) const {
+ return true;
+ }
+};
+
+struct HasVolatileOrAsynchronousSymbolHelper
+ : public AnyTraverse<HasVolatileOrAsynchronousSymbolHelper> {
+ using Base = AnyTraverse<HasVolatileOrAsynchronousSymbolHelper>;
+ HasVolatileOrAsynchronousSymbolHelper() : Base{*this} {}
+ using Base::operator();
+ bool operator()(const Symbol &symbol) const {
+ const Symbol &ultimate{symbol.GetUltimate()};
+ if (ultimate.attrs().HasAny(
+ {semantics::Attr::VOLATILE, semantics::Attr::ASYNCHRONOUS}))
+ return true;
+ if (const auto *assoc{ultimate.detailsIf<semantics::AssocEntityDetails>()})
+ return (*this)(assoc->expr());
+ return false;
+ }
+};
+
+} // namespace
+
+bool HasParentheses(const Expr<SomeType> &expr) {
+ return HasParenthesesHelper{}(expr);
+}
+
+bool HasProcedureRef(const Expr<SomeType> &expr) {
+ return HasProcedureRefHelper{}(expr);
+}
+
+bool HasSubtract(const Expr<SomeType> &expr) {
+ return HasSubtractHelper{}(expr);
+}
+
+bool HasVolatileOrAsynchronousSymbol(const Expr<SomeType> &expr) {
+ return HasVolatileOrAsynchronousSymbolHelper{}(expr);
+}
+
bool IsArraySection(const Expr<SomeType> &expr) {
return expr.Rank() > 0 && IsVariable(expr) && !UnwrapWholeSymbolDataRef(expr);
}
diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index 27fa06362a1a8..da32832cf058b 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -5569,8 +5569,8 @@ class FirConverter : public Fortran::lower::AbstractConverter {
// Helper to generate the code evaluating the right-hand side.
auto evaluateRhs = [&](Fortran::lower::StatementContext &stmtCtx) {
- hlfir::Entity rhs = Fortran::lower::convertExprToHLFIR(
- loc, *this, assign.rhs, localSymbols, stmtCtx);
+ hlfir::Entity rhs = Fortran::lower::convertAssignmentRhsToHLFIR(
+ loc, *this, assign.lhs, assign.rhs, localSymbols, stmtCtx);
// Load trivial scalar RHS to allow the loads to be hoisted outside of
// loops early if possible. This also dereferences pointer and
// allocatable RHS: the target is being assigned from.
diff --git a/flang/lib/Lower/ConvertExprToHLFIR.cpp b/flang/lib/Lower/ConvertExprToHLFIR.cpp
index c03129f74a889..86380c1a8ad7e 100644
--- a/flang/lib/Lower/ConvertExprToHLFIR.cpp
+++ b/flang/lib/Lower/ConvertExprToHLFIR.cpp
@@ -33,11 +33,17 @@
#include "flang/Optimizer/Dialect/FIRAttr.h"
#include "flang/Optimizer/HLFIR/HLFIROps.h"
#include "mlir/IR/IRMapping.h"
+#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/TypeSwitch.h"
+#include "llvm/Support/CommandLine.h"
#include <optional>
namespace {
+static llvm::cl::opt<bool> enableSplitSumExpressionTreeLowering(
+ "enable-split-sum-expression-tree-lowering", llvm::cl::Hidden,
+ llvm::cl::desc("Enable experimental split sum expression tree lowering"));
+
// This was modelled after isParenthesizedVariable()
template <typename T>
static bool isParenthesized(const Fortran::evaluate::Expr<T> &expr) {
@@ -51,6 +57,113 @@ static bool isParenthesized(const Fortran::evaluate::Expr<T> &expr) {
}
}
+template <int KIND>
+using Real = Fortran::evaluate::Type<Fortran::common::TypeCategory::Real, KIND>;
+
+template <int KIND>
+using RealExpr = Fortran::evaluate::Expr<Real<KIND>>;
+
+template <int KIND>
+static void flattenTopLevelAdds(const RealExpr<KIND> &expr,
+ llvm::SmallVectorImpl<RealExpr<KIND>> &terms) {
+ if (const auto *add =
+ std::get_if<Fortran::evaluate::Add<Real<KIND>>>(&expr.u)) {
+ flattenTopLevelAdds(add->left(), terms);
+ flattenTopLevelAdds(add->right(), terms);
+ return;
+ }
+ terms.push_back(expr);
+}
+
+template <int KIND>
+static RealExpr<KIND>
+buildRightAssociatedAddFold(llvm::ArrayRef<RealExpr<KIND>> terms) {
+ assert(!terms.empty() && "cannot build empty add fold");
+ if (terms.size() == 1)
+ return terms.front();
+ RealExpr<KIND> result{terms.back()};
+ for (const RealExpr<KIND> &term : llvm::reverse(terms.drop_back()))
+ result = RealExpr<KIND>{Fortran::evaluate::Add<Real<KIND>>{term, result}};
+ return result;
+}
+
+template <typename L, typename R>
+static std::optional<Fortran::lower::SomeExpr>
+tryBuildSplitSumExpressionTree(const L &, const R &) {
+ return std::nullopt;
+}
+
+template <int KIND>
+static std::optional<Fortran::lower::SomeExpr>
+tryBuildSplitSumExpressionTree(const RealExpr<KIND> &lhs,
+ const RealExpr<KIND> &rhs) {
+ if (!std::get_if<Fortran::evaluate::Add<Real<KIND>>>(&rhs.u))
+ return std::nullopt;
+
+ llvm::SmallVector<RealExpr<KIND>, 8> terms;
+ flattenTopLevelAdds(rhs, terms);
+ if (terms.size() <= 2)
+ return std::nullopt;
+
+ llvm::SmallVector<RealExpr<KIND>, 2> head{terms[0], terms[1]};
+ llvm::SmallVector<RealExpr<KIND>, 8> tail(terms.begin() + 2, terms.end());
+ RealExpr<KIND> headExpr = buildRightAssociatedAddFold<KIND>(head);
+ RealExpr<KIND> tailExpr = buildRightAssociatedAddFold<KIND>(tail);
+ return Fortran::lower::SomeExpr{RealExpr<KIND>{
+ Fortran::evaluate::Add<Real<KIND>>{std::move(tailExpr), headExpr}}};
+}
+
+template <Fortran::common::TypeCategory CAT>
+static std::optional<Fortran::lower::SomeExpr> tryBuildSplitSumExpressionTree(
+ const Fortran::evaluate::Expr<Fortran::evaluate::SomeKind<CAT>> &lhs,
+ const Fortran::evaluate::Expr<Fortran::evaluate::SomeKind<CAT>> &rhs) {
+ if constexpr (CAT == Fortran::common::TypeCategory::Real) {
+ return Fortran::common::visit(
+ [&](const auto &typedLhs) -> std::optional<Fortran::lower::SomeExpr> {
+ return Fortran::common::visit(
+ [&](const auto &typedRhs)
+ -> std::optional<Fortran::lower::SomeExpr> {
+ return tryBuildSplitSumExpressionTree(typedLhs, typedRhs);
+ },
+ rhs.u);
+ },
+ lhs.u);
+ }
+ return std::nullopt;
+}
+
+static bool
+canBuildSplitSumExpressionTree(const Fortran::lower::SomeExpr &lhs,
+ const Fortran::lower::SomeExpr &rhs) {
+ // The split rewrites a top-level addition chain. Subtraction would need to
+ // be carried as signed terms; division is safe here because it remains inside
+ // an individual term rather than changing the additive chain.
+ return rhs.Rank() == 0 && lhs.Rank() == 0 &&
+ !Fortran::evaluate::HasVectorSubscript(rhs) &&
+ !Fortran::evaluate::HasVectorSubscript(lhs) &&
+ !Fortran::evaluate::HasParentheses(rhs) &&
+ !Fortran::evaluate::HasSubtract(rhs) &&
+ !Fortran::evaluate::HasProcedureRef(rhs) &&
+ !Fortran::evaluate::HasProcedureRef(lhs) &&
+ !Fortran::evaluate::HasVolatileOrAsynchronousSymbol(rhs) &&
+ !Fortran::evaluate::HasVolatileOrAsynchronousSymbol(lhs);
+}
+
+static std::optional<Fortran::lower::SomeExpr>
+tryBuildSplitSumExpressionTree(const Fortran::lower::SomeExpr &lhs,
+ const Fortran::lower::SomeExpr &rhs) {
+ return Fortran::common::visit(
+ [&](const auto &typedLhs) -> std::optional<Fortran::lower::SomeExpr> {
+ return Fortran::common::visit(
+ [&](const auto &typedRhs)
+ -> std::optional<Fortran::lower::SomeExpr> {
+ return tryBuildSplitSumExpressionTree(typedLhs, typedRhs);
+ },
+ rhs.u);
+ },
+ lhs.u);
+}
+
/// Lower Designators to HLFIR.
class HlfirDesignatorBuilder {
private:
@@ -2313,6 +2426,18 @@ hlfir::EntityWithAttributes Fortran::lower::convertExprToHLFIR(
return HlfirBuilder(loc, converter, symMap, stmtCtx).gen(expr);
}
+hlfir::EntityWithAttributes Fortran::lower::convertAssignmentRhsToHLFIR(
+ mlir::Location loc, Fortran::lower::AbstractConverter &converter,
+ const Fortran::lower::SomeExpr &lhs, const Fortran::lower::SomeExpr &rhs,
+ Fortran::lower::SymMap &symMap, Fortran::lower::StatementContext &stmtCtx) {
+ if (enableSplitSumExpressionTreeLowering &&
+ canBuildSplitSumExpressionTree(lhs, rhs))
+ if (std::optional<Fortran::lower::SomeExpr> rewritten =
+ tryBuildSplitSumExpressionTree(lhs, rhs))
+ return HlfirBuilder(loc, converter, symMap, stmtCtx).gen(*rewritten);
+ return convertExprToHLFIR(loc, converter, rhs, symMap, stmtCtx);
+}
+
fir::ExtendedValue Fortran::lower::convertToBox(
mlir::Location loc, Fortran::lower::AbstractConverter &converter,
hlfir::Entity entity, Fortran::lower::StatementContext &stmtCtx,
diff --git a/flang/test/Lower/split-sum-expression-tree-lowering.f90 b/flang/test/Lower/split-sum-expression-tree-lowering.f90
new file mode 100644
index 0000000000000..e6287533ed75e
--- /dev/null
+++ b/flang/test/Lower/split-sum-expression-tree-lowering.f90
@@ -0,0 +1,748 @@
+! RUN: %flang_fc1 -emit-hlfir -mllvm -enable-split-sum-expression-tree-lowering -o - %s | FileCheck %s --check-prefixes=SPLIT,NO-REWRITE
+! RUN: %flang_fc1 -emit-hlfir -o - %s | FileCheck %s --check-prefixes=DEFAULT,NO-REWRITE
+
+! Default: (((x + a*b) + c*d) + e*f)
+! Rewritten: ((c*d + e*f) + (x + a*b))
+subroutine eligible_self_update3(x,a,b,c,d,e,f)
+ real(8) :: x,a,b,c,d,e,f
+ x = x + a*b + c*d + e*f
+end
+
+! SPLIT-LABEL: func.func @_QPeligible_self_update3
+! SPLIT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ea"}
+! SPLIT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Eb"}
+! SPLIT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ec"}
+! SPLIT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ed"}
+! SPLIT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ee"}
+! SPLIT-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ef"}
+! SPLIT-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ex"}
+! SPLIT: %[[CV:.*]] = fir.load %[[C]]#0
+! SPLIT: %[[DV:.*]] = fir.load %[[D]]#0
+! SPLIT: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! SPLIT: %[[EV:.*]] = fir.load %[[E]]#0
+! SPLIT: %[[FV:.*]] = fir.load %[[F]]#0
+! SPLIT: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! SPLIT: %[[TAIL:.*]] = arith.addf %[[CD]], %[[EF]]
+! SPLIT: %[[XV:.*]] = fir.load %[[X]]#0
+! SPLIT: %[[AV:.*]] = fir.load %[[A]]#0
+! SPLIT: %[[BV:.*]] = fir.load %[[B]]#0
+! SPLIT: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! SPLIT: %[[HEAD:.*]] = arith.addf %[[XV]], %[[AB]]
+! SPLIT-NOT: arith.addf %[[HEAD]], %[[CD]]
+! SPLIT: %[[RES:.*]] = arith.addf %[[TAIL]], %[[HEAD]]
+! SPLIT: hlfir.assign %[[RES]] to %[[X]]#0
+
+! DEFAULT-LABEL: func.func @_QPeligible_self_update3
+! DEFAULT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ea"}
+! DEFAULT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Eb"}
+! DEFAULT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ec"}
+! DEFAULT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ed"}
+! DEFAULT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ee"}
+! DEFAULT-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ef"}
+! DEFAULT-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update3Ex"}
+! DEFAULT: %[[XV:.*]] = fir.load %[[X]]#0
+! DEFAULT: %[[AV:.*]] = fir.load %[[A]]#0
+! DEFAULT: %[[BV:.*]] = fir.load %[[B]]#0
+! DEFAULT: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! DEFAULT: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! DEFAULT: %[[CV:.*]] = fir.load %[[C]]#0
+! DEFAULT: %[[DV:.*]] = fir.load %[[D]]#0
+! DEFAULT: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! DEFAULT: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! DEFAULT: %[[EV:.*]] = fir.load %[[E]]#0
+! DEFAULT: %[[FV:.*]] = fir.load %[[F]]#0
+! DEFAULT: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! DEFAULT: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! DEFAULT: hlfir.assign %[[RES]] to %[[X]]#0
+
+! Default: ((((x + a*b) + c*d) + e*f) + g*h)
+! Rewritten: ((c*d + (e*f + g*h)) + (x + a*b))
+subroutine eligible_self_update4(x,a,b,c,d,e,f,g,h)
+ real(8) :: x,a,b,c,d,e,f,g,h
+ x = x + a*b + c*d + e*f + g*h
+end
+
+! SPLIT-LABEL: func.func @_QPeligible_self_update4
+! SPLIT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ea"}
+! SPLIT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Eb"}
+! SPLIT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ec"}
+! SPLIT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ed"}
+! SPLIT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ee"}
+! SPLIT-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ef"}
+! SPLIT-DAG: %[[G:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Eg"}
+! SPLIT-DAG: %[[H:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Eh"}
+! SPLIT-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ex"}
+! SPLIT: %[[CV:.*]] = fir.load %[[C]]#0
+! SPLIT: %[[DV:.*]] = fir.load %[[D]]#0
+! SPLIT: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! SPLIT: %[[EV:.*]] = fir.load %[[E]]#0
+! SPLIT: %[[FV:.*]] = fir.load %[[F]]#0
+! SPLIT: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! SPLIT: %[[GV:.*]] = fir.load %[[G]]#0
+! SPLIT: %[[HV:.*]] = fir.load %[[H]]#0
+! SPLIT: %[[GH:.*]] = arith.mulf %[[GV]], %[[HV]]
+! SPLIT: %[[EFGH:.*]] = arith.addf %[[EF]], %[[GH]]
+! SPLIT: %[[TAIL:.*]] = arith.addf %[[CD]], %[[EFGH]]
+! SPLIT: %[[XV:.*]] = fir.load %[[X]]#0
+! SPLIT: %[[AV:.*]] = fir.load %[[A]]#0
+! SPLIT: %[[BV:.*]] = fir.load %[[B]]#0
+! SPLIT: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! SPLIT: %[[HEAD:.*]] = arith.addf %[[XV]], %[[AB]]
+! SPLIT-NOT: arith.addf %[[HEAD]], %[[CD]]
+! SPLIT: %[[RES:.*]] = arith.addf %[[TAIL]], %[[HEAD]]
+! SPLIT: hlfir.assign %[[RES]] to %[[X]]#0
+
+! DEFAULT-LABEL: func.func @_QPeligible_self_update4
+! DEFAULT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ea"}
+! DEFAULT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Eb"}
+! DEFAULT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ec"}
+! DEFAULT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ed"}
+! DEFAULT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ee"}
+! DEFAULT-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ef"}
+! DEFAULT-DAG: %[[G:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Eg"}
+! DEFAULT-DAG: %[[H:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Eh"}
+! DEFAULT-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_self_update4Ex"}
+! DEFAULT: %[[XV:.*]] = fir.load %[[X]]#0
+! DEFAULT: %[[AV:.*]] = fir.load %[[A]]#0
+! DEFAULT: %[[BV:.*]] = fir.load %[[B]]#0
+! DEFAULT: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! DEFAULT: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! DEFAULT: %[[CV:.*]] = fir.load %[[C]]#0
+! DEFAULT: %[[DV:.*]] = fir.load %[[D]]#0
+! DEFAULT: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! DEFAULT: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! DEFAULT: %[[EV:.*]] = fir.load %[[E]]#0
+! DEFAULT: %[[FV:.*]] = fir.load %[[F]]#0
+! DEFAULT: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! DEFAULT: %[[XABCDEF:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! DEFAULT: %[[GV:.*]] = fir.load %[[G]]#0
+! DEFAULT: %[[HV:.*]] = fir.load %[[H]]#0
+! DEFAULT: %[[GH:.*]] = arith.mulf %[[GV]], %[[HV]]
+! DEFAULT: %[[RES:.*]] = arith.addf %[[XABCDEF]], %[[GH]]
+! DEFAULT: hlfir.assign %[[RES]] to %[[X]]#0
+
+! Default: (((a*b + c*d) + e*f) + g*h)
+! Rewritten: ((e*f + g*h) + (a*b + c*d))
+subroutine eligible_out_of_place4(y,a,b,c,d,e,f,g,h)
+ real(8) :: y,a,b,c,d,e,f,g,h
+ y = a*b + c*d + e*f + g*h
+end
+
+! SPLIT-LABEL: func.func @_QPeligible_out_of_place4
+! SPLIT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ea"}
+! SPLIT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Eb"}
+! SPLIT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ec"}
+! SPLIT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ed"}
+! SPLIT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ee"}
+! SPLIT-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ef"}
+! SPLIT-DAG: %[[G:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Eg"}
+! SPLIT-DAG: %[[H:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Eh"}
+! SPLIT-DAG: %[[Y:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ey"}
+! SPLIT: %[[EV:.*]] = fir.load %[[E]]#0
+! SPLIT: %[[FV:.*]] = fir.load %[[F]]#0
+! SPLIT: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! SPLIT: %[[GV:.*]] = fir.load %[[G]]#0
+! SPLIT: %[[HV:.*]] = fir.load %[[H]]#0
+! SPLIT: %[[GH:.*]] = arith.mulf %[[GV]], %[[HV]]
+! SPLIT: %[[TAIL:.*]] = arith.addf %[[EF]], %[[GH]]
+! SPLIT: %[[AV:.*]] = fir.load %[[A]]#0
+! SPLIT: %[[BV:.*]] = fir.load %[[B]]#0
+! SPLIT: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! SPLIT: %[[CV:.*]] = fir.load %[[C]]#0
+! SPLIT: %[[DV:.*]] = fir.load %[[D]]#0
+! SPLIT: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! SPLIT: %[[HEAD:.*]] = arith.addf %[[AB]], %[[CD]]
+! SPLIT-NOT: arith.addf %[[HEAD]], %[[EF]]
+! SPLIT: %[[RES:.*]] = arith.addf %[[TAIL]], %[[HEAD]]
+! SPLIT: hlfir.assign %[[RES]] to %[[Y]]#0
+
+! DEFAULT-LABEL: func.func @_QPeligible_out_of_place4
+! DEFAULT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ea"}
+! DEFAULT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Eb"}
+! DEFAULT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ec"}
+! DEFAULT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ed"}
+! DEFAULT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ee"}
+! DEFAULT-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ef"}
+! DEFAULT-DAG: %[[G:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Eg"}
+! DEFAULT-DAG: %[[H:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Eh"}
+! DEFAULT-DAG: %[[Y:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_out_of_place4Ey"}
+! DEFAULT: %[[AV:.*]] = fir.load %[[A]]#0
+! DEFAULT: %[[BV:.*]] = fir.load %[[B]]#0
+! DEFAULT: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! DEFAULT: %[[CV:.*]] = fir.load %[[C]]#0
+! DEFAULT: %[[DV:.*]] = fir.load %[[D]]#0
+! DEFAULT: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! DEFAULT: %[[ABCD:.*]] = arith.addf %[[AB]], %[[CD]]
+! DEFAULT: %[[EV:.*]] = fir.load %[[E]]#0
+! DEFAULT: %[[FV:.*]] = fir.load %[[F]]#0
+! DEFAULT: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! DEFAULT: %[[ABCDEF:.*]] = arith.addf %[[ABCD]], %[[EF]]
+! DEFAULT: %[[GV:.*]] = fir.load %[[G]]#0
+! DEFAULT: %[[HV:.*]] = fir.load %[[H]]#0
+! DEFAULT: %[[GH:.*]] = arith.mulf %[[GV]], %[[HV]]
+! DEFAULT: %[[RES:.*]] = arith.addf %[[ABCDEF]], %[[GH]]
+! DEFAULT: hlfir.assign %[[RES]] to %[[Y]]#0
+
+! Default: (((x + a) + b*c) + d*e)
+! Rewritten: ((b*c + d*e) + (x + a))
+subroutine eligible_scalar_term(x,a,b,c,d,e)
+ real(8) :: x,a,b,c,d,e
+ x = x + a + b*c + d*e
+end
+
+! SPLIT-LABEL: func.func @_QPeligible_scalar_term
+! SPLIT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEa"}
+! SPLIT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEb"}
+! SPLIT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEc"}
+! SPLIT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEd"}
+! SPLIT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEe"}
+! SPLIT-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEx"}
+! SPLIT: %[[BV:.*]] = fir.load %[[B]]#0
+! SPLIT: %[[CV:.*]] = fir.load %[[C]]#0
+! SPLIT: %[[BC:.*]] = arith.mulf %[[BV]], %[[CV]]
+! SPLIT: %[[DV:.*]] = fir.load %[[D]]#0
+! SPLIT: %[[EV:.*]] = fir.load %[[E]]#0
+! SPLIT: %[[DE:.*]] = arith.mulf %[[DV]], %[[EV]]
+! SPLIT: %[[TAIL:.*]] = arith.addf %[[BC]], %[[DE]]
+! SPLIT: %[[XV:.*]] = fir.load %[[X]]#0
+! SPLIT: %[[AV:.*]] = fir.load %[[A]]#0
+! SPLIT: %[[HEAD:.*]] = arith.addf %[[XV]], %[[AV]]
+! SPLIT-NOT: arith.addf %[[HEAD]], %[[BC]]
+! SPLIT: %[[RES:.*]] = arith.addf %[[TAIL]], %[[HEAD]]
+! SPLIT: hlfir.assign %[[RES]] to %[[X]]#0
+
+! DEFAULT-LABEL: func.func @_QPeligible_scalar_term
+! DEFAULT-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEa"}
+! DEFAULT-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEb"}
+! DEFAULT-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEc"}
+! DEFAULT-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEd"}
+! DEFAULT-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEe"}
+! DEFAULT-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFeligible_scalar_termEx"}
+! DEFAULT: %[[XV:.*]] = fir.load %[[X]]#0
+! DEFAULT: %[[AV:.*]] = fir.load %[[A]]#0
+! DEFAULT: %[[XA:.*]] = arith.addf %[[XV]], %[[AV]]
+! DEFAULT: %[[BV:.*]] = fir.load %[[B]]#0
+! DEFAULT: %[[CV:.*]] = fir.load %[[C]]#0
+! DEFAULT: %[[BC:.*]] = arith.mulf %[[BV]], %[[CV]]
+! DEFAULT: %[[XABC:.*]] = arith.addf %[[XA]], %[[BC]]
+! DEFAULT: %[[DV:.*]] = fir.load %[[D]]#0
+! DEFAULT: %[[EV:.*]] = fir.load %[[E]]#0
+! DEFAULT: %[[DE:.*]] = arith.mulf %[[DV]], %[[EV]]
+! DEFAULT: %[[RES:.*]] = arith.addf %[[XABC]], %[[DE]]
+! DEFAULT: hlfir.assign %[[RES]] to %[[X]]#0
+
+subroutine guard_parentheses(x,a,b,c,d,e,f)
+ real(8) :: x,a,b,c,d,e,f
+ x = (x + a*b) + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_parentheses
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_parenthesesEx"}
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_parenthesesEe"}
+! NO-REWRITE: fir.load %[[X]]#0
+! NO-REWRITE: hlfir.no_reassoc
+! NO-REWRITE: fir.load %[[E]]#0
+
+subroutine guard_subtract(x,a,b,c,d,e,f)
+ real(8) :: x,a,b,c,d,e,f
+ x = x - a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_subtract
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_subtractEa"}
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_subtractEb"}
+! NO-REWRITE-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_subtractEc"}
+! NO-REWRITE-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_subtractEd"}
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_subtractEe"}
+! NO-REWRITE-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_subtractEf"}
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_subtractEx"}
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[X]]#0
+! NO-REWRITE: %[[AV:.*]] = fir.load %[[A]]#0
+! NO-REWRITE: %[[BV:.*]] = fir.load %[[B]]#0
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.subf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %[[C]]#0
+! NO-REWRITE: %[[DV:.*]] = fir.load %[[D]]#0
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %[[E]]#0
+! NO-REWRITE: %[[FV:.*]] = fir.load %[[F]]#0
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %[[X]]#0
+
+real(8) function foo(a)
+ real(8) :: a
+ foo = a
+end
+
+subroutine guard_call(x,a,b,c,d,e)
+ real(8) :: x,a,b,c,d,e,foo
+ x = x + foo(a) + b*c + d*e
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_call
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_callEa"}
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_callEb"}
+! NO-REWRITE-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_callEc"}
+! NO-REWRITE-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_callEd"}
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_callEe"}
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_callEx"}
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[X]]#0
+! NO-REWRITE: %[[FOO:.*]] = fir.call @_QPfoo(%[[A]]#0)
+! NO-REWRITE: %[[XFOO:.*]] = arith.addf %[[XV]], %[[FOO]]
+! NO-REWRITE: %[[BV:.*]] = fir.load %[[B]]#0
+! NO-REWRITE: %[[CV:.*]] = fir.load %[[C]]#0
+! NO-REWRITE: %[[BC:.*]] = arith.mulf %[[BV]], %[[CV]]
+! NO-REWRITE: %[[XFOOBC:.*]] = arith.addf %[[XFOO]], %[[BC]]
+! NO-REWRITE: %[[DV:.*]] = fir.load %[[D]]#0
+! NO-REWRITE: %[[EV:.*]] = fir.load %[[E]]#0
+! NO-REWRITE: %[[DE:.*]] = arith.mulf %[[DV]], %[[EV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XFOOBC]], %[[DE]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %[[X]]#0
+
+subroutine guard_array(n,x,a,b,c,d,e,f)
+ integer :: n
+ real(8) :: x(n),a(n),b(n),c(n),d(n),e(n),f(n)
+ x = x + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_array
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_arrayEa"}
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_arrayEb"}
+! NO-REWRITE-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_arrayEc"}
+! NO-REWRITE-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_arrayEd"}
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_arrayEe"}
+! NO-REWRITE-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_arrayEf"}
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_arrayEx"}
+! NO-REWRITE: %[[AB:.*]] = hlfir.elemental
+! NO-REWRITE: fir.load
+! NO-REWRITE: fir.load
+! NO-REWRITE: %[[ABV:.*]] = arith.mulf
+! NO-REWRITE: hlfir.yield_element %[[ABV]]
+! NO-REWRITE: %[[XAB:.*]] = hlfir.elemental
+! NO-REWRITE: hlfir.designate %[[X]]#0
+! NO-REWRITE: %[[ABAPPLY:.*]] = hlfir.apply %[[AB]]
+! NO-REWRITE: %[[XV:.*]] = fir.load
+! NO-REWRITE: %[[XABV:.*]] = arith.addf %[[XV]], %[[ABAPPLY]]
+! NO-REWRITE: hlfir.yield_element %[[XABV]]
+! NO-REWRITE: %[[CD:.*]] = hlfir.elemental
+! NO-REWRITE: fir.load
+! NO-REWRITE: fir.load
+! NO-REWRITE: %[[CDV:.*]] = arith.mulf
+! NO-REWRITE: hlfir.yield_element %[[CDV]]
+! NO-REWRITE: %[[XABCD:.*]] = hlfir.elemental
+! NO-REWRITE: %[[XABAPPLY:.*]] = hlfir.apply %[[XAB]]
+! NO-REWRITE: %[[CDAPPLY:.*]] = hlfir.apply %[[CD]]
+! NO-REWRITE: %[[XABCDV:.*]] = arith.addf %[[XABAPPLY]], %[[CDAPPLY]]
+! NO-REWRITE: hlfir.yield_element %[[XABCDV]]
+! NO-REWRITE: %[[EF:.*]] = hlfir.elemental
+! NO-REWRITE: fir.load
+! NO-REWRITE: fir.load
+! NO-REWRITE: %[[EFV:.*]] = arith.mulf
+! NO-REWRITE: hlfir.yield_element %[[EFV]]
+! NO-REWRITE: %[[XABCDEF:.*]] = hlfir.elemental
+! NO-REWRITE: %[[XABCDAPPLY:.*]] = hlfir.apply %[[XABCD]]
+! NO-REWRITE: %[[EFAPPLY:.*]] = hlfir.apply %[[EF]]
+! NO-REWRITE: %[[XABCDEFV:.*]] = arith.addf %[[XABCDAPPLY]], %[[EFAPPLY]]
+! NO-REWRITE: hlfir.yield_element %[[XABCDEFV]]
+! NO-REWRITE: hlfir.assign %[[XABCDEF]] to %[[X]]#0
+
+subroutine guard_short_sum(x,a,b)
+ real(8) :: x,a,b
+ x = x + a*b
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_short_sum
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_short_sumEa"}
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_short_sumEb"}
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_short_sumEx"}
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[X]]#0
+! NO-REWRITE: %[[AV:.*]] = fir.load %[[A]]#0
+! NO-REWRITE: %[[BV:.*]] = fir.load %[[B]]#0
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %[[X]]#0
+
+subroutine guard_mixed_kind(x,a,b,c,d,e,f)
+ real(8) :: x
+ real(4) :: a,b,c,d,e,f
+ x = a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_mixed_kind
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_mixed_kindEa"}
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_mixed_kindEb"}
+! NO-REWRITE-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_mixed_kindEc"}
+! NO-REWRITE-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_mixed_kindEd"}
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_mixed_kindEe"}
+! NO-REWRITE-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_mixed_kindEf"}
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}} {uniq_name = "_QFguard_mixed_kindEx"}
+! NO-REWRITE: %[[AV:.*]] = fir.load %[[A]]#0
+! NO-REWRITE: %[[BV:.*]] = fir.load %[[B]]#0
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %[[C]]#0
+! NO-REWRITE: %[[DV:.*]] = fir.load %[[D]]#0
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[ABCD:.*]] = arith.addf %[[AB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %[[E]]#0
+! NO-REWRITE: %[[FV:.*]] = fir.load %[[F]]#0
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[SUM:.*]] = arith.addf %[[ABCD]], %[[EF]]
+! NO-REWRITE: %[[RES:.*]] = fir.convert %[[SUM]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %[[X]]#0
+
+module split_sum_guard_mod
+ real(8), volatile :: use_volatile_x
+ real(8), asynchronous :: use_asynchronous_x
+end module
+
+subroutine guard_use_assoc_volatile(y,a,b,c,d,e,f)
+ use split_sum_guard_mod
+ real(8) :: y,a,b,c,d,e,f
+ y = use_volatile_x + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_use_assoc_volatile
+! NO-REWRITE: %[[XV:.*]] = fir.load %{{.*}} : !fir.ref<f64, volatile>
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_use_assoc_asynchronous(y,a,b,c,d,e,f)
+ use split_sum_guard_mod
+ real(8) :: y,a,b,c,d,e,f
+ y = use_asynchronous_x + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_use_assoc_asynchronous
+! NO-REWRITE: %[[XV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_volatile(x,a,b,c,d,e,f)
+ real(8), volatile :: x
+ real(8) :: a,b,c,d,e,f
+ x = x + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_volatile
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatileEa"
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatileEb"
+! NO-REWRITE-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatileEc"
+! NO-REWRITE-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatileEd"
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatileEe"
+! NO-REWRITE-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatileEf"
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatileEx"
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[X]]#0
+! NO-REWRITE: %[[AV:.*]] = fir.load %[[A]]#0
+! NO-REWRITE: %[[BV:.*]] = fir.load %[[B]]#0
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %[[C]]#0
+! NO-REWRITE: %[[DV:.*]] = fir.load %[[D]]#0
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %[[E]]#0
+! NO-REWRITE: %[[FV:.*]] = fir.load %[[F]]#0
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %[[X]]#0
+
+subroutine guard_volatile_lhs_only(x,a,b,c,d,e,f)
+ real(8), volatile :: x
+ real(8) :: a,b,c,d,e,f
+ x = a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_volatile_lhs_only
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatile_lhs_onlyEa"
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatile_lhs_onlyEb"
+! NO-REWRITE-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatile_lhs_onlyEc"
+! NO-REWRITE-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatile_lhs_onlyEd"
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatile_lhs_onlyEe"
+! NO-REWRITE-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatile_lhs_onlyEf"
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_volatile_lhs_onlyEx"
+! NO-REWRITE: %[[AV:.*]] = fir.load %[[A]]#0
+! NO-REWRITE: %[[BV:.*]] = fir.load %[[B]]#0
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %[[C]]#0
+! NO-REWRITE: %[[DV:.*]] = fir.load %[[D]]#0
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[ABCD:.*]] = arith.addf %[[AB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %[[E]]#0
+! NO-REWRITE: %[[FV:.*]] = fir.load %[[F]]#0
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[ABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %[[X]]#0
+
+subroutine guard_asynchronous(x,a,b,c,d,e,f)
+ real(8), asynchronous :: x
+ real(8) :: a,b,c,d,e,f
+ x = x + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_asynchronous
+! NO-REWRITE-DAG: %[[A:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_asynchronousEa"
+! NO-REWRITE-DAG: %[[B:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_asynchronousEb"
+! NO-REWRITE-DAG: %[[C:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_asynchronousEc"
+! NO-REWRITE-DAG: %[[D:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_asynchronousEd"
+! NO-REWRITE-DAG: %[[E:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_asynchronousEe"
+! NO-REWRITE-DAG: %[[F:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_asynchronousEf"
+! NO-REWRITE-DAG: %[[X:.*]]:2 = hlfir.declare {{.*}}uniq_name = "_QFguard_asynchronousEx"
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[X]]#0
+! NO-REWRITE: %[[AV:.*]] = fir.load %[[A]]#0
+! NO-REWRITE: %[[BV:.*]] = fir.load %[[B]]#0
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %[[C]]#0
+! NO-REWRITE: %[[DV:.*]] = fir.load %[[D]]#0
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %[[E]]#0
+! NO-REWRITE: %[[FV:.*]] = fir.load %[[F]]#0
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %[[X]]#0
+
+subroutine guard_volatile_array_element(i,x,a,b,c,d,e,f)
+ integer :: i
+ real(8), volatile :: x(10)
+ real(8) :: a,b,c,d,e,f
+ x(i) = x(i) + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_volatile_array_element
+! NO-REWRITE: %[[XELT:.*]] = hlfir.designate {{.*}} -> !fir.ref<f64, volatile>
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[XELT]]
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_volatile_subscript(i,x,a,b,c,d,e,f)
+ integer, volatile :: i
+ real(8) :: x(10),a,b,c,d,e,f
+ x(i) = x(i) + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_volatile_subscript
+! NO-REWRITE: %[[IV:.*]] = fir.load %{{.*}} : !fir.ref<i32, volatile>
+! NO-REWRITE: %[[SUB:.*]] = fir.convert %[[IV]]
+! NO-REWRITE: %[[XELT:.*]] = hlfir.designate {{.*}}(%[[SUB]])
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[XELT]]
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_associate_volatile_array_element(i,x,y,a,b,c,d,e,f)
+ integer :: i
+ real(8), volatile :: x(10)
+ real(8) :: y,a,b,c,d,e,f
+ associate(v => x(i))
+ y = v + a*b + c*d + e*f
+ end associate
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_associate_volatile_array_element
+! NO-REWRITE: %[[VELT:.*]] = hlfir.designate {{.*}} -> !fir.ref<f64, volatile>
+! NO-REWRITE: %[[VV:.*]] = fir.load %{{.*}} : !fir.ref<f64, volatile>
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[VAB:.*]] = arith.addf %[[VV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[VABCD:.*]] = arith.addf %[[VAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[VABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_associate_asynchronous_array_element(i,x,y,a,b,c,d,e,f)
+ integer :: i
+ real(8), asynchronous :: x(10)
+ real(8) :: y,a,b,c,d,e,f
+ associate(v => x(i))
+ y = v + a*b + c*d + e*f
+ end associate
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_associate_asynchronous_array_element
+! NO-REWRITE: %[[VELT:.*]] = hlfir.designate {{.*}} -> !fir.ref<f64>
+! NO-REWRITE: %[[VV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[VAB:.*]] = arith.addf %[[VV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[VABCD:.*]] = arith.addf %[[VAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[VABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_volatile_complex_part(x,z,a,b,c,d,e,f)
+ real(8) :: x,a,b,c,d,e,f
+ complex(8), volatile :: z
+ x = z%re + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_volatile_complex_part
+! NO-REWRITE: %[[ZRE_REF:.*]] = hlfir.designate {{.*}} real
+! NO-REWRITE: %[[ZRE:.*]] = fir.load %[[ZRE_REF]]
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[ZAB:.*]] = arith.addf %[[ZRE]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[ZABCD:.*]] = arith.addf %[[ZAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[ZABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_associate_volatile_complex_part(x,z,a,b,c,d,e,f)
+ real(8) :: x,a,b,c,d,e,f
+ complex(8), volatile :: z
+ associate(v => z%re)
+ x = v + a*b + c*d + e*f
+ end associate
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_associate_volatile_complex_part
+! NO-REWRITE: %[[ZRE_REF:.*]] = hlfir.designate {{.*}} real
+! NO-REWRITE: %[[ZRE:.*]] = fir.load %{{.*}} : !fir.ref<f64, volatile>
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[ZAB:.*]] = arith.addf %[[ZRE]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[ZABCD:.*]] = arith.addf %[[ZAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[ZABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_associate_asynchronous_complex_part(x,z,a,b,c,d,e,f)
+ real(8) :: x,a,b,c,d,e,f
+ complex(8), asynchronous :: z
+ associate(v => z%re)
+ x = v + a*b + c*d + e*f
+ end associate
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_associate_asynchronous_complex_part
+! NO-REWRITE: %[[ZRE_REF:.*]] = hlfir.designate {{.*}} real
+! NO-REWRITE: %[[ZRE:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[ZAB:.*]] = arith.addf %[[ZRE]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[ZABCD:.*]] = arith.addf %[[ZAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[ZABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_asynchronous_complex_part(x,z,a,b,c,d,e,f)
+ real(8) :: x,a,b,c,d,e,f
+ complex(8), asynchronous :: z
+ x = z%re + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_asynchronous_complex_part
+! NO-REWRITE: %[[ZRE_REF:.*]] = hlfir.designate {{.*}} real
+! NO-REWRITE: %[[ZRE:.*]] = fir.load %[[ZRE_REF]]
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[ZAB:.*]] = arith.addf %[[ZRE]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[ZABCD:.*]] = arith.addf %[[ZAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[ZABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
+
+subroutine guard_asynchronous_array_element(i,x,a,b,c,d,e,f)
+ integer :: i
+ real(8), asynchronous :: x(10)
+ real(8) :: a,b,c,d,e,f
+ x(i) = x(i) + a*b + c*d + e*f
+end
+
+! NO-REWRITE-LABEL: func.func @_QPguard_asynchronous_array_element
+! NO-REWRITE: %[[XELT:.*]] = hlfir.designate {{.*}} -> !fir.ref<f64>
+! NO-REWRITE: %[[XV:.*]] = fir.load %[[XELT]]
+! NO-REWRITE: %[[AV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[BV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[AB:.*]] = arith.mulf %[[AV]], %[[BV]]
+! NO-REWRITE: %[[XAB:.*]] = arith.addf %[[XV]], %[[AB]]
+! NO-REWRITE: %[[CV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[DV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[CD:.*]] = arith.mulf %[[CV]], %[[DV]]
+! NO-REWRITE: %[[XABCD:.*]] = arith.addf %[[XAB]], %[[CD]]
+! NO-REWRITE: %[[EV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[FV:.*]] = fir.load %{{.*}} : !fir.ref<f64>
+! NO-REWRITE: %[[EF:.*]] = arith.mulf %[[EV]], %[[FV]]
+! NO-REWRITE: %[[RES:.*]] = arith.addf %[[XABCD]], %[[EF]]
+! NO-REWRITE: hlfir.assign %[[RES]] to %{{.*}}
>From 34c726fb2ab8c52ea1ec8751bb643fe28bf6ff66 Mon Sep 17 00:00:00 2001
From: Tom Eccles <tom.eccles at arm.com>
Date: Mon, 6 Jul 2026 15:36:21 +0100
Subject: [PATCH 2/4] No need for an lhs parameter when doing the actual
transform
This was an artefact of an earlier code structure and I missed
refactoring this. Removing the lhs argument dramatically reduces the
number of template instantiations.
---
flang/lib/Lower/ConvertExprToHLFIR.cpp | 41 +++++++++-----------------
1 file changed, 14 insertions(+), 27 deletions(-)
diff --git a/flang/lib/Lower/ConvertExprToHLFIR.cpp b/flang/lib/Lower/ConvertExprToHLFIR.cpp
index 86380c1a8ad7e..e5855cd2fea53 100644
--- a/flang/lib/Lower/ConvertExprToHLFIR.cpp
+++ b/flang/lib/Lower/ConvertExprToHLFIR.cpp
@@ -87,21 +87,20 @@ buildRightAssociatedAddFold(llvm::ArrayRef<RealExpr<KIND>> terms) {
return result;
}
-template <typename L, typename R>
+template <typename T>
static std::optional<Fortran::lower::SomeExpr>
-tryBuildSplitSumExpressionTree(const L &, const R &) {
+tryBuildSplitSumExpressionTree(const T &) {
return std::nullopt;
}
template <int KIND>
static std::optional<Fortran::lower::SomeExpr>
-tryBuildSplitSumExpressionTree(const RealExpr<KIND> &lhs,
- const RealExpr<KIND> &rhs) {
- if (!std::get_if<Fortran::evaluate::Add<Real<KIND>>>(&rhs.u))
+tryBuildSplitSumExpressionTree(const RealExpr<KIND> &expr) {
+ if (!std::get_if<Fortran::evaluate::Add<Real<KIND>>>(&expr.u))
return std::nullopt;
llvm::SmallVector<RealExpr<KIND>, 8> terms;
- flattenTopLevelAdds(rhs, terms);
+ flattenTopLevelAdds(expr, terms);
if (terms.size() <= 2)
return std::nullopt;
@@ -115,19 +114,13 @@ tryBuildSplitSumExpressionTree(const RealExpr<KIND> &lhs,
template <Fortran::common::TypeCategory CAT>
static std::optional<Fortran::lower::SomeExpr> tryBuildSplitSumExpressionTree(
- const Fortran::evaluate::Expr<Fortran::evaluate::SomeKind<CAT>> &lhs,
- const Fortran::evaluate::Expr<Fortran::evaluate::SomeKind<CAT>> &rhs) {
+ const Fortran::evaluate::Expr<Fortran::evaluate::SomeKind<CAT>> &expr) {
if constexpr (CAT == Fortran::common::TypeCategory::Real) {
return Fortran::common::visit(
- [&](const auto &typedLhs) -> std::optional<Fortran::lower::SomeExpr> {
- return Fortran::common::visit(
- [&](const auto &typedRhs)
- -> std::optional<Fortran::lower::SomeExpr> {
- return tryBuildSplitSumExpressionTree(typedLhs, typedRhs);
- },
- rhs.u);
+ [&](const auto &typedExpr) -> std::optional<Fortran::lower::SomeExpr> {
+ return tryBuildSplitSumExpressionTree(typedExpr);
},
- lhs.u);
+ expr.u);
}
return std::nullopt;
}
@@ -150,18 +143,12 @@ canBuildSplitSumExpressionTree(const Fortran::lower::SomeExpr &lhs,
}
static std::optional<Fortran::lower::SomeExpr>
-tryBuildSplitSumExpressionTree(const Fortran::lower::SomeExpr &lhs,
- const Fortran::lower::SomeExpr &rhs) {
+tryBuildSplitSumExpressionTree(const Fortran::lower::SomeExpr &expr) {
return Fortran::common::visit(
- [&](const auto &typedLhs) -> std::optional<Fortran::lower::SomeExpr> {
- return Fortran::common::visit(
- [&](const auto &typedRhs)
- -> std::optional<Fortran::lower::SomeExpr> {
- return tryBuildSplitSumExpressionTree(typedLhs, typedRhs);
- },
- rhs.u);
+ [&](const auto &typedExpr) -> std::optional<Fortran::lower::SomeExpr> {
+ return tryBuildSplitSumExpressionTree(typedExpr);
},
- lhs.u);
+ expr.u);
}
/// Lower Designators to HLFIR.
@@ -2433,7 +2420,7 @@ hlfir::EntityWithAttributes Fortran::lower::convertAssignmentRhsToHLFIR(
if (enableSplitSumExpressionTreeLowering &&
canBuildSplitSumExpressionTree(lhs, rhs))
if (std::optional<Fortran::lower::SomeExpr> rewritten =
- tryBuildSplitSumExpressionTree(lhs, rhs))
+ tryBuildSplitSumExpressionTree(rhs))
return HlfirBuilder(loc, converter, symMap, stmtCtx).gen(*rewritten);
return convertExprToHLFIR(loc, converter, rhs, symMap, stmtCtx);
}
>From b8bab3e40eeac66dd636fb39b08188e9f44e0840 Mon Sep 17 00:00:00 2001
From: Tom Eccles <tom.eccles at arm.com>
Date: Mon, 6 Jul 2026 17:43:36 +0100
Subject: [PATCH 3/4] Move to Evaluate
---
flang/include/flang/Evaluate/tools.h | 9 ++
.../include/flang/Lower/ConvertExprToHLFIR.h | 7 --
flang/lib/Evaluate/tools.cpp | 87 ++++++++++++++
flang/lib/Lower/Bridge.cpp | 19 ++-
flang/lib/Lower/ConvertExprToHLFIR.cpp | 111 ------------------
5 files changed, 113 insertions(+), 120 deletions(-)
diff --git a/flang/include/flang/Evaluate/tools.h b/flang/include/flang/Evaluate/tools.h
index 471b8c2e7f3c1..5a286e73dc39e 100644
--- a/flang/include/flang/Evaluate/tools.h
+++ b/flang/include/flang/Evaluate/tools.h
@@ -1132,6 +1132,15 @@ bool HasSubtract(const Expr<SomeType> &expr);
// Predicate: does an expression contain a VOLATILE or ASYNCHRONOUS symbol?
bool HasVolatileOrAsynchronousSymbol(const Expr<SomeType> &expr);
+// Can a scalar real RHS expression in an assignment be rewritten as a split
+// sum expression tree?
+bool CanBuildSplitSumExpressionTree(
+ const Expr<SomeType> &lhs, const Expr<SomeType> &rhs);
+
+// Try to rewrite a scalar real sum as a split sum expression tree.
+std::optional<Expr<SomeType>> TryBuildSplitSumExpressionTree(
+ const Expr<SomeType> &expr);
+
// Utilities for attaching the location of the declaration of a symbol
// of interest to a message. Handles the case of USE association gracefully.
parser::Message *AttachDeclaration(parser::Message &, const Symbol &);
diff --git a/flang/include/flang/Lower/ConvertExprToHLFIR.h b/flang/include/flang/Lower/ConvertExprToHLFIR.h
index 9be880c4b090a..60ecd643be436 100644
--- a/flang/include/flang/Lower/ConvertExprToHLFIR.h
+++ b/flang/include/flang/Lower/ConvertExprToHLFIR.h
@@ -41,13 +41,6 @@ convertExprToHLFIR(mlir::Location loc, Fortran::lower::AbstractConverter &,
const Fortran::lower::SomeExpr &, Fortran::lower::SymMap &,
Fortran::lower::StatementContext &);
-/// Lower an assignment RHS to HLFIR, optionally using assignment context to
-/// choose an experimental expression tree for eligible scalar real sums.
-hlfir::EntityWithAttributes convertAssignmentRhsToHLFIR(
- mlir::Location loc, Fortran::lower::AbstractConverter &,
- const Fortran::lower::SomeExpr &lhs, const Fortran::lower::SomeExpr &rhs,
- Fortran::lower::SymMap &, Fortran::lower::StatementContext &);
-
inline fir::ExtendedValue translateToExtendedValue(
mlir::Location loc, fir::FirOpBuilder &builder, hlfir::Entity entity,
Fortran::lower::StatementContext &context, bool contiguityHint = false) {
diff --git a/flang/lib/Evaluate/tools.cpp b/flang/lib/Evaluate/tools.cpp
index f22ccdb4389a0..4cb9376ac4d44 100644
--- a/flang/lib/Evaluate/tools.cpp
+++ b/flang/lib/Evaluate/tools.cpp
@@ -13,6 +13,8 @@
#include "flang/Evaluate/traverse.h"
#include "flang/Parser/message.h"
#include "flang/Semantics/tools.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringSwitch.h"
#include <algorithm>
#include <variant>
@@ -1382,6 +1384,91 @@ bool HasVolatileOrAsynchronousSymbol(const Expr<SomeType> &expr) {
return HasVolatileOrAsynchronousSymbolHelper{}(expr);
}
+template <int KIND> using Real = Type<common::TypeCategory::Real, KIND>;
+
+template <int KIND> using RealExpr = Expr<Real<KIND>>;
+
+template <int KIND>
+static void flattenTopLevelAdds(
+ const RealExpr<KIND> &expr, llvm::SmallVectorImpl<RealExpr<KIND>> &terms) {
+ if (const auto *add = std::get_if<Add<Real<KIND>>>(&expr.u)) {
+ flattenTopLevelAdds(add->left(), terms);
+ flattenTopLevelAdds(add->right(), terms);
+ return;
+ }
+ terms.push_back(expr);
+}
+
+template <int KIND>
+static RealExpr<KIND> buildRightAssociatedAddFold(
+ llvm::ArrayRef<RealExpr<KIND>> terms) {
+ assert(!terms.empty() && "cannot build empty add fold");
+ if (terms.size() == 1)
+ return terms.front();
+ RealExpr<KIND> result{terms.back()};
+ for (const RealExpr<KIND> &term : llvm::reverse(terms.drop_back()))
+ result = RealExpr<KIND>{Add<Real<KIND>>{term, result}};
+ return result;
+}
+
+template <typename T>
+static std::optional<Expr<SomeType>> tryBuildSplitSumExpressionTree(const T &) {
+ return std::nullopt;
+}
+
+template <int KIND>
+static std::optional<Expr<SomeType>> tryBuildSplitSumExpressionTree(
+ const RealExpr<KIND> &expr) {
+ if (!std::get_if<Add<Real<KIND>>>(&expr.u))
+ return std::nullopt;
+
+ llvm::SmallVector<RealExpr<KIND>, 8> terms;
+ flattenTopLevelAdds(expr, terms);
+ if (terms.size() <= 2)
+ return std::nullopt;
+
+ llvm::SmallVector<RealExpr<KIND>, 2> head{terms[0], terms[1]};
+ llvm::SmallVector<RealExpr<KIND>, 8> tail(terms.begin() + 2, terms.end());
+ RealExpr<KIND> headExpr = buildRightAssociatedAddFold<KIND>(head);
+ RealExpr<KIND> tailExpr = buildRightAssociatedAddFold<KIND>(tail);
+ return Expr<SomeType>{
+ RealExpr<KIND>{Add<Real<KIND>>{std::move(tailExpr), headExpr}}};
+}
+
+template <common::TypeCategory CAT>
+static std::optional<Expr<SomeType>> tryBuildSplitSumExpressionTree(
+ const Expr<SomeKind<CAT>> &expr) {
+ if constexpr (CAT == common::TypeCategory::Real) {
+ return common::visit(
+ [&](const auto &typedExpr) -> std::optional<Expr<SomeType>> {
+ return tryBuildSplitSumExpressionTree(typedExpr);
+ },
+ expr.u);
+ }
+ return std::nullopt;
+}
+
+bool CanBuildSplitSumExpressionTree(
+ const Expr<SomeType> &lhs, const Expr<SomeType> &rhs) {
+ // The split rewrites a top-level addition chain. Subtraction would need to
+ // be carried as signed terms; division is safe here because it remains inside
+ // an individual term rather than changing the additive chain.
+ return rhs.Rank() == 0 && lhs.Rank() == 0 && !HasVectorSubscript(rhs) &&
+ !HasVectorSubscript(lhs) && !HasParentheses(rhs) && !HasSubtract(rhs) &&
+ !HasProcedureRef(rhs) && !HasProcedureRef(lhs) &&
+ !HasVolatileOrAsynchronousSymbol(rhs) &&
+ !HasVolatileOrAsynchronousSymbol(lhs);
+}
+
+std::optional<Expr<SomeType>> TryBuildSplitSumExpressionTree(
+ const Expr<SomeType> &expr) {
+ return common::visit(
+ [&](const auto &typedExpr) -> std::optional<Expr<SomeType>> {
+ return tryBuildSplitSumExpressionTree(typedExpr);
+ },
+ expr.u);
+}
+
bool IsArraySection(const Expr<SomeType> &expr) {
return expr.Rank() > 0 && IsVariable(expr) && !UnwrapWholeSymbolDataRef(expr);
}
diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index da32832cf058b..86a53dfca8631 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -12,6 +12,7 @@
#include "flang/Lower/Bridge.h"
+#include "flang/Evaluate/tools.h"
#include "flang/Lower/Allocatable.h"
#include "flang/Lower/CUDA.h"
#include "flang/Lower/CallInterface.h"
@@ -91,6 +92,10 @@ static llvm::cl::opt<bool> forceLoopToExecuteOnce(
"always-execute-loop-body", llvm::cl::init(false),
llvm::cl::desc("force the body of a loop to execute at least once"));
+static llvm::cl::opt<bool> enableSplitSumExpressionTreeLowering(
+ "enable-split-sum-expression-tree-lowering", llvm::cl::Hidden,
+ llvm::cl::desc("Enable experimental split sum expression tree lowering"));
+
namespace {
/// Information for generating a structured or unstructured increment loop.
struct IncrementLoopInfo {
@@ -5569,8 +5574,18 @@ class FirConverter : public Fortran::lower::AbstractConverter {
// Helper to generate the code evaluating the right-hand side.
auto evaluateRhs = [&](Fortran::lower::StatementContext &stmtCtx) {
- hlfir::Entity rhs = Fortran::lower::convertAssignmentRhsToHLFIR(
- loc, *this, assign.lhs, assign.rhs, localSymbols, stmtCtx);
+ const Fortran::lower::SomeExpr *rhsExpr = &assign.rhs;
+ std::optional<Fortran::lower::SomeExpr> rewritten;
+ if (enableSplitSumExpressionTreeLowering &&
+ Fortran::evaluate::CanBuildSplitSumExpressionTree(assign.lhs,
+ assign.rhs)) {
+ rewritten =
+ Fortran::evaluate::TryBuildSplitSumExpressionTree(assign.rhs);
+ if (rewritten)
+ rhsExpr = &*rewritten;
+ }
+ hlfir::Entity rhs = Fortran::lower::convertExprToHLFIR(
+ loc, *this, *rhsExpr, localSymbols, stmtCtx);
// Load trivial scalar RHS to allow the loads to be hoisted outside of
// loops early if possible. This also dereferences pointer and
// allocatable RHS: the target is being assigned from.
diff --git a/flang/lib/Lower/ConvertExprToHLFIR.cpp b/flang/lib/Lower/ConvertExprToHLFIR.cpp
index e5855cd2fea53..bac7b6cbd2a89 100644
--- a/flang/lib/Lower/ConvertExprToHLFIR.cpp
+++ b/flang/lib/Lower/ConvertExprToHLFIR.cpp
@@ -35,15 +35,10 @@
#include "mlir/IR/IRMapping.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/TypeSwitch.h"
-#include "llvm/Support/CommandLine.h"
#include <optional>
namespace {
-static llvm::cl::opt<bool> enableSplitSumExpressionTreeLowering(
- "enable-split-sum-expression-tree-lowering", llvm::cl::Hidden,
- llvm::cl::desc("Enable experimental split sum expression tree lowering"));
-
// This was modelled after isParenthesizedVariable()
template <typename T>
static bool isParenthesized(const Fortran::evaluate::Expr<T> &expr) {
@@ -57,100 +52,6 @@ static bool isParenthesized(const Fortran::evaluate::Expr<T> &expr) {
}
}
-template <int KIND>
-using Real = Fortran::evaluate::Type<Fortran::common::TypeCategory::Real, KIND>;
-
-template <int KIND>
-using RealExpr = Fortran::evaluate::Expr<Real<KIND>>;
-
-template <int KIND>
-static void flattenTopLevelAdds(const RealExpr<KIND> &expr,
- llvm::SmallVectorImpl<RealExpr<KIND>> &terms) {
- if (const auto *add =
- std::get_if<Fortran::evaluate::Add<Real<KIND>>>(&expr.u)) {
- flattenTopLevelAdds(add->left(), terms);
- flattenTopLevelAdds(add->right(), terms);
- return;
- }
- terms.push_back(expr);
-}
-
-template <int KIND>
-static RealExpr<KIND>
-buildRightAssociatedAddFold(llvm::ArrayRef<RealExpr<KIND>> terms) {
- assert(!terms.empty() && "cannot build empty add fold");
- if (terms.size() == 1)
- return terms.front();
- RealExpr<KIND> result{terms.back()};
- for (const RealExpr<KIND> &term : llvm::reverse(terms.drop_back()))
- result = RealExpr<KIND>{Fortran::evaluate::Add<Real<KIND>>{term, result}};
- return result;
-}
-
-template <typename T>
-static std::optional<Fortran::lower::SomeExpr>
-tryBuildSplitSumExpressionTree(const T &) {
- return std::nullopt;
-}
-
-template <int KIND>
-static std::optional<Fortran::lower::SomeExpr>
-tryBuildSplitSumExpressionTree(const RealExpr<KIND> &expr) {
- if (!std::get_if<Fortran::evaluate::Add<Real<KIND>>>(&expr.u))
- return std::nullopt;
-
- llvm::SmallVector<RealExpr<KIND>, 8> terms;
- flattenTopLevelAdds(expr, terms);
- if (terms.size() <= 2)
- return std::nullopt;
-
- llvm::SmallVector<RealExpr<KIND>, 2> head{terms[0], terms[1]};
- llvm::SmallVector<RealExpr<KIND>, 8> tail(terms.begin() + 2, terms.end());
- RealExpr<KIND> headExpr = buildRightAssociatedAddFold<KIND>(head);
- RealExpr<KIND> tailExpr = buildRightAssociatedAddFold<KIND>(tail);
- return Fortran::lower::SomeExpr{RealExpr<KIND>{
- Fortran::evaluate::Add<Real<KIND>>{std::move(tailExpr), headExpr}}};
-}
-
-template <Fortran::common::TypeCategory CAT>
-static std::optional<Fortran::lower::SomeExpr> tryBuildSplitSumExpressionTree(
- const Fortran::evaluate::Expr<Fortran::evaluate::SomeKind<CAT>> &expr) {
- if constexpr (CAT == Fortran::common::TypeCategory::Real) {
- return Fortran::common::visit(
- [&](const auto &typedExpr) -> std::optional<Fortran::lower::SomeExpr> {
- return tryBuildSplitSumExpressionTree(typedExpr);
- },
- expr.u);
- }
- return std::nullopt;
-}
-
-static bool
-canBuildSplitSumExpressionTree(const Fortran::lower::SomeExpr &lhs,
- const Fortran::lower::SomeExpr &rhs) {
- // The split rewrites a top-level addition chain. Subtraction would need to
- // be carried as signed terms; division is safe here because it remains inside
- // an individual term rather than changing the additive chain.
- return rhs.Rank() == 0 && lhs.Rank() == 0 &&
- !Fortran::evaluate::HasVectorSubscript(rhs) &&
- !Fortran::evaluate::HasVectorSubscript(lhs) &&
- !Fortran::evaluate::HasParentheses(rhs) &&
- !Fortran::evaluate::HasSubtract(rhs) &&
- !Fortran::evaluate::HasProcedureRef(rhs) &&
- !Fortran::evaluate::HasProcedureRef(lhs) &&
- !Fortran::evaluate::HasVolatileOrAsynchronousSymbol(rhs) &&
- !Fortran::evaluate::HasVolatileOrAsynchronousSymbol(lhs);
-}
-
-static std::optional<Fortran::lower::SomeExpr>
-tryBuildSplitSumExpressionTree(const Fortran::lower::SomeExpr &expr) {
- return Fortran::common::visit(
- [&](const auto &typedExpr) -> std::optional<Fortran::lower::SomeExpr> {
- return tryBuildSplitSumExpressionTree(typedExpr);
- },
- expr.u);
-}
-
/// Lower Designators to HLFIR.
class HlfirDesignatorBuilder {
private:
@@ -2413,18 +2314,6 @@ hlfir::EntityWithAttributes Fortran::lower::convertExprToHLFIR(
return HlfirBuilder(loc, converter, symMap, stmtCtx).gen(expr);
}
-hlfir::EntityWithAttributes Fortran::lower::convertAssignmentRhsToHLFIR(
- mlir::Location loc, Fortran::lower::AbstractConverter &converter,
- const Fortran::lower::SomeExpr &lhs, const Fortran::lower::SomeExpr &rhs,
- Fortran::lower::SymMap &symMap, Fortran::lower::StatementContext &stmtCtx) {
- if (enableSplitSumExpressionTreeLowering &&
- canBuildSplitSumExpressionTree(lhs, rhs))
- if (std::optional<Fortran::lower::SomeExpr> rewritten =
- tryBuildSplitSumExpressionTree(rhs))
- return HlfirBuilder(loc, converter, symMap, stmtCtx).gen(*rewritten);
- return convertExprToHLFIR(loc, converter, rhs, symMap, stmtCtx);
-}
-
fir::ExtendedValue Fortran::lower::convertToBox(
mlir::Location loc, Fortran::lower::AbstractConverter &converter,
hlfir::Entity entity, Fortran::lower::StatementContext &stmtCtx,
>From b95ac078bb345cd42c41e10b1d9fdb9fc629e16f Mon Sep 17 00:00:00 2001
From: Tom Eccles <tom.eccles at arm.com>
Date: Mon, 6 Jul 2026 17:48:20 +0100
Subject: [PATCH 4/4] Improve comment wording
---
flang/lib/Evaluate/tools.cpp | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/flang/lib/Evaluate/tools.cpp b/flang/lib/Evaluate/tools.cpp
index 4cb9376ac4d44..9167c697e6523 100644
--- a/flang/lib/Evaluate/tools.cpp
+++ b/flang/lib/Evaluate/tools.cpp
@@ -1450,9 +1450,9 @@ static std::optional<Expr<SomeType>> tryBuildSplitSumExpressionTree(
bool CanBuildSplitSumExpressionTree(
const Expr<SomeType> &lhs, const Expr<SomeType> &rhs) {
- // The split rewrites a top-level addition chain. Subtraction would need to
- // be carried as signed terms; division is safe here because it remains inside
- // an individual term rather than changing the additive chain.
+ // The split only understands top-level Add nodes. Reject Subtract
+ // conservatively for now rather than trying to model signed terms in
+ // additive chains; this also rejects subtraction in subexpressions.
return rhs.Rank() == 0 && lhs.Rank() == 0 && !HasVectorSubscript(rhs) &&
!HasVectorSubscript(lhs) && !HasParentheses(rhs) && !HasSubtract(rhs) &&
!HasProcedureRef(rhs) && !HasProcedureRef(lhs) &&
More information about the flang-commits
mailing list