[clang] [llvm] Minimal support of floating-point operand bundles (PR #135658)
Serge Pavlov via llvm-commits
llvm-commits at lists.llvm.org
Sat Mar 21 07:08:35 PDT 2026
https://github.com/spavloff updated https://github.com/llvm/llvm-project/pull/135658
>From 077776b7c396a6c60a894181ede174a8fe0baa6f Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Mon, 14 Apr 2025 12:51:43 +0700
Subject: [PATCH 01/24] Minimal support of floating-point operand bundles
This is a lite version of https://github.com/llvm/llvm-project/pull/109798,
where code changes are minimized to facilitate discussion about the
implementation. The motivations and ideas behind the new floating-point
operation support are described in that PR and in the discussion
https://discourse.llvm.org/t/rfc-change-of-strict-fp-operation-representation-in-ir/85021.
There are concerns that the proposed changes are too invasive and a new
approach is required to make the transition smoother.
This implementation is essentially a subset of PR109798, where
everything beyond the minimum is removed. It tries to build eventually
the same implementation as that PR but in different steps.
The patch does not attempt to modify the existing implementation based
on the constrained intrinsics. Instead it introduces a new one using
operand bundles. This new implementation initially has very limited
functionality, which latter will be extended and finally can replace the
existing one.
This PR introduces the notion of floating-point operation, this is an
intrinsic, that is listed in the file "FloatingPointOps.def". These have
two additional properties:
1. In the strict environment (a function with strictfp attribute) calls
to these operations acquire side effect, now it is read/write access
to inaccessible memory, just as constrained intrinsics do.
2. Calls to these operations may have floating-point operand bundles.
There are two kinds of such bundles, tagged with "fp.control" and
"fp.except", which are used to carry additional information about
control modes and exception handling. Initially the set of control
modes consists of rounding mode only.
The set of operations enlisted in "FloatingPointOps.def" and in
"ConstrainedOps.def" are completely independent, an intrinsic may be in
one list or in both. The set of floating-point operations is expected to
grow and finally all FP intrinsics will be available in the new
implementation. In this patch set of intrinsics in
"FloatingPointOps.def" is minimum necessary for tests.
---
llvm/docs/LangRef.rst | 48 ++++-
llvm/docs/ReleaseNotes.md | 1 +
llvm/include/llvm/IR/FPEnv.h | 18 ++
llvm/include/llvm/IR/FloatingPointOps.def | 24 +++
llvm/include/llvm/IR/IRBuilder.h | 45 ++--
llvm/include/llvm/IR/InstrTypes.h | 16 ++
llvm/include/llvm/IR/IntrinsicInst.h | 8 +
llvm/include/llvm/IR/LLVMContext.h | 4 +-
llvm/include/llvm/Support/ModRef.h | 5 +
llvm/lib/IR/FPEnv.cpp | 66 ++++++
llvm/lib/IR/IRBuilder.cpp | 75 +++++++
llvm/lib/IR/Instructions.cpp | 87 +++++++-
llvm/lib/IR/IntrinsicInst.cpp | 10 +
llvm/lib/IR/LLVMContext.cpp | 14 ++
llvm/lib/IR/Verifier.cpp | 31 ++-
.../Bitcode/operand-bundles-bc-analyzer.ll | 2 +
llvm/test/Verifier/fp-intrinsics.ll | 91 ++++++++
llvm/unittests/IR/IRBuilderTest.cpp | 197 ++++++++++++++++++
18 files changed, 720 insertions(+), 22 deletions(-)
create mode 100644 llvm/include/llvm/IR/FloatingPointOps.def
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index a1835436ae359..7c749c538bc03 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3283,6 +3283,51 @@ value of its first argument instead of calling the specified function
or intrinsic. This is achieved with ``PATCHINST`` relocations on the
target instructions (see the AArch64 psABI for details).
+.. _ob_fp:
+
+Floating-point Operand Bundles
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+These operand bundles are used for calls that involve floating-point
+operations and interact with :ref:`floating-point environment <floatenv>` or
+depend on floating-point options, such as rounding mode, denormal modes, etc.
+There are two kinds of such operand bundles, which represent the value of
+floating-point control modes and the treatment of status bits respectively.
+
+An operand bundle tagged with "fp.control" contains information about the
+control modes used for the operation execution. Operands specified in this
+bundle represent particular options. Currently, only rounding mode is supported.
+It is represented by a metadata string value, which specifies the rounding mode
+to be used for the operation evaluation. Possible values are:
+
+::
+
+ "rtz" - toward zero
+ "rte" - to nearest, ties to even
+ "rtp" - toward positive infinity
+ "rtn" - toward negative infinity
+ "rmm" - to nearest, ties away from zero
+ "dyn" - rounding mode is taken from control register
+
+If "fp.control" is absent, the default rounding rounding mode is taken from the
+control register (dynamic rounding). In the particular case of
+:ref:`default floating-point environment <floatenv>`, it must be rounding to
+nearest, ties to even.
+
+An operand bundle tagged with "fp.except" may be associated with operations
+that can read or write floating-point exception flags. It contains a single
+metadata string value, which can have one of the following values:
+
+::
+
+ "ignore"
+ "strict"
+ "maytrap"
+
+It has the same meaning as the corresponding argument in
+:ref:`constrained intrinsics <constrainedfp>`.
+
+
.. _moduleasm:
Module-Level Inline Assembly
@@ -4037,7 +4082,8 @@ round-to-nearest rounding mode, and subnormals are assumed to be preserved.
Running LLVM code in an environment where these assumptions are not met
typically leads to undefined behavior. The ``strictfp`` and
:ref:`denormal_fpenv <denormal_fpenv>` attributes as well as
-:ref:`Constrained Floating-Point Intrinsics <constrainedfp>` can be
+:ref:`Constrained Floating-Point Intrinsics <constrainedfp>` and
+:ref:`floating-point operand bundles<ob_fp> can be
used to weaken LLVM's assumptions and ensure defined behavior in
non-default floating-point environments; see their respective
documentation for details.
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 2e0c5c5cb9370..90e7ca5e98a5f 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -62,6 +62,7 @@ Changes to the LLVM IR
* Removed `llvm.convert.to.fp16` and `llvm.convert.from.fp16`
intrinsics. These are equivalent to `fptrunc` and `fpext` with half
with a bitcast.
+* Floating-point operand bundles have been added.
* "denormal-fp-math" and "denormal-fp-math-f32" string attributes were
migrated to first-class denormal_fpenv attribute.
diff --git a/llvm/include/llvm/IR/FPEnv.h b/llvm/include/llvm/IR/FPEnv.h
index 38395b15c8c09..78a5d6d5b75f5 100644
--- a/llvm/include/llvm/IR/FPEnv.h
+++ b/llvm/include/llvm/IR/FPEnv.h
@@ -49,20 +49,38 @@ enum ExceptionBehavior : uint8_t {
/// metadata.
LLVM_ABI std::optional<RoundingMode> convertStrToRoundingMode(StringRef);
+/// Returns a valid RoundingMode enumerator given a string that is used as
+/// rounding mode specifier in operand bundles.
+std::optional<RoundingMode> convertBundleToRoundingMode(StringRef);
+
/// For any RoundingMode enumerator, returns a string valid as input in
/// constrained intrinsic rounding mode metadata.
LLVM_ABI std::optional<StringRef> convertRoundingModeToStr(RoundingMode);
+/// For any RoundingMode enumerator, returns a string to be used in operand
+/// bundles.
+std::optional<StringRef> convertRoundingModeToBundle(RoundingMode);
+
/// Returns a valid ExceptionBehavior enumerator when given a string
/// valid as input in constrained intrinsic exception behavior metadata.
LLVM_ABI std::optional<fp::ExceptionBehavior>
convertStrToExceptionBehavior(StringRef);
+/// Returns a valid ExceptionBehavior enumerator given a string from the operand
+/// bundle argument.
+std::optional<fp::ExceptionBehavior>
+ convertBundleToExceptionBehavior(StringRef);
+
/// For any ExceptionBehavior enumerator, returns a string valid as
/// input in constrained intrinsic exception behavior metadata.
LLVM_ABI std::optional<StringRef>
convertExceptionBehaviorToStr(fp::ExceptionBehavior);
+/// Return string representing the given exception behavior for use in operand
+/// bundles
+std::optional<StringRef>
+ convertExceptionBehaviorToBundle(fp::ExceptionBehavior);
+
/// Returns true if the exception handling behavior and rounding mode
/// match what is used in the default floating point environment.
inline bool isDefaultFPEnvironment(fp::ExceptionBehavior EB, RoundingMode RM) {
diff --git a/llvm/include/llvm/IR/FloatingPointOps.def b/llvm/include/llvm/IR/FloatingPointOps.def
new file mode 100644
index 0000000000000..8567b5dbac302
--- /dev/null
+++ b/llvm/include/llvm/IR/FloatingPointOps.def
@@ -0,0 +1,24 @@
+//===- llvm/IR/FloatingPointOps.def - FP intrinsics -------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Defines set of intrinsics, which are classified as floating-point operations.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef FUNCTION
+#define FUNCTION(N,D)
+#endif
+
+// Arguments of the entries are:
+// - intrinsic function name,
+// - DAG node corresponding to the intrinsic.
+
+FUNCTION(nearbyint, FNEARBYINT)
+FUNCTION(trunc, FTRUNC)
+
+#undef FUNCTION
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index 94d5943522070..ab9f5b2c82885 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -1003,6 +1003,16 @@ class IRBuilderBase {
FMFSource FMFSource = {},
const Twine &Name = "");
+ /// Create a call to intrinsic \p ID with \p Args, mangled using \p Types and
+ /// with operand bundles.
+ /// If \p FMFSource is provided, copy fast-math-flags from that instruction to
+ /// the intrinsic.
+ CallInst *CreateIntrinsic(Intrinsic::ID ID, ArrayRef<Type *> Types,
+ ArrayRef<Value *> Args,
+ ArrayRef<OperandBundleDef> OpBundles,
+ Instruction *FMFSource = nullptr,
+ const Twine &Name = "");
+
/// Create a call to non-overloaded intrinsic \p ID with \p Args. If
/// \p FMFSource is provided, copy fast-math-flags from that instruction to
/// the intrinsic.
@@ -2487,24 +2497,13 @@ class IRBuilderBase {
CallInst *CreateCall(FunctionType *FTy, Value *Callee,
ArrayRef<Value *> Args = {}, const Twine &Name = "",
MDNode *FPMathTag = nullptr) {
- CallInst *CI = CallInst::Create(FTy, Callee, Args, DefaultOperandBundles);
- if (IsFPConstrained)
- setConstrainedFPCallAttr(CI);
- if (isa<FPMathOperator>(CI))
- setFPAttrs(CI, FPMathTag, FMF);
- return Insert(CI, Name);
+ return CreateCall(FTy, Callee, Args, DefaultOperandBundles, Name,
+ FPMathTag);
}
CallInst *CreateCall(FunctionType *FTy, Value *Callee, ArrayRef<Value *> Args,
ArrayRef<OperandBundleDef> OpBundles,
- const Twine &Name = "", MDNode *FPMathTag = nullptr) {
- CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
- if (IsFPConstrained)
- setConstrainedFPCallAttr(CI);
- if (isa<FPMathOperator>(CI))
- setFPAttrs(CI, FPMathTag, FMF);
- return Insert(CI, Name);
- }
+ const Twine &Name = "", MDNode *FPMathTag = nullptr);
CallInst *CreateCall(FunctionCallee Callee, ArrayRef<Value *> Args = {},
const Twine &Name = "", MDNode *FPMathTag = nullptr) {
@@ -2766,6 +2765,24 @@ class IRBuilderBase {
/// assumption on the provided pointer.
LLVM_ABI CallInst *CreateDereferenceableAssumption(Value *PtrValue,
Value *SizeValue);
+
+ /// Create an operand bundle in the provided bundle set to represent given FP
+ /// rounding mode.
+ ///
+ /// If the rounding mode is not defined, adds the default rounding mode,
+ /// stored in this builder object.
+ void
+ createFPRoundingBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<RoundingMode> Rounding = std::nullopt);
+
+ /// Create an operand bundle in the provided bundle set to represent FP
+ /// exception behavior.
+ ///
+ /// If the exception behavior is not defined, adds the default behavior,
+ /// stored in this builder object.
+ void createFPExceptionBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<fp::ExceptionBehavior> Except = std::nullopt);
};
/// This provides a uniform API for creating instructions and inserting
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index 7aea805e0b86b..d438a4c53a7c1 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -25,6 +25,7 @@
#include "llvm/IR/CallingConv.h"
#include "llvm/IR/DerivedTypes.h"
#include "llvm/IR/FMF.h"
+#include "llvm/IR/FPEnv.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/Instruction.h"
#include "llvm/IR/LLVMContext.h"
@@ -1093,6 +1094,13 @@ template <typename InputTy> class OperandBundleDefT {
using OperandBundleDef = OperandBundleDefT<Value *>;
using ConstOperandBundleDef = OperandBundleDefT<const Value *>;
+void addFPRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding);
+void addFPExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except);
+
//===----------------------------------------------------------------------===//
// CallBase Class
//===----------------------------------------------------------------------===//
@@ -1152,6 +1160,8 @@ class CallBase : public Instruction {
/// number of extra operands.
LLVM_ABI unsigned getNumSubclassExtraOperandsDynamic() const;
+ MemoryEffects getFloatingPointMemoryEffects() const;
+
public:
using Instruction::getContext;
@@ -2162,6 +2172,12 @@ class CallBase : public Instruction {
return false;
}
+ /// Return rounding mode specified by operand bundles.
+ RoundingMode getRoundingMode() const;
+
+ /// Return exception behavior specified by operand bundles.
+ std::optional<fp::ExceptionBehavior> getExceptionBehavior() const;
+
/// Used to keep track of an operand bundle. See the main comment on
/// OperandBundleUser above.
struct BundleOpInfo {
diff --git a/llvm/include/llvm/IR/IntrinsicInst.h b/llvm/include/llvm/IR/IntrinsicInst.h
index 784e599c53f44..98a2987afbba0 100644
--- a/llvm/include/llvm/IR/IntrinsicInst.h
+++ b/llvm/include/llvm/IR/IntrinsicInst.h
@@ -135,6 +135,14 @@ class IntrinsicInst : public CallInst {
/// course of IR transformations
LLVM_ABI static bool mayLowerToFunctionCall(Intrinsic::ID IID);
+ /// Check if \p ID represents a function that may access FP environment and
+ /// may have FP operand bundles.
+ ///
+ /// Access to FP environment means that in the strict FP environment the
+ /// function has read/write memory effect, which is used to maintain proper
+ /// instructions ordering.
+ static bool isFloatingPointOperation(Intrinsic::ID IID);
+
/// Methods for support type inquiry through isa, cast, and dyn_cast:
static bool classof(const CallInst *I) {
auto *F = dyn_cast_or_null<Function>(I->getCalledOperand());
diff --git a/llvm/include/llvm/IR/LLVMContext.h b/llvm/include/llvm/IR/LLVMContext.h
index 646a04673dbd0..20a2807fe9613 100644
--- a/llvm/include/llvm/IR/LLVMContext.h
+++ b/llvm/include/llvm/IR/LLVMContext.h
@@ -99,7 +99,9 @@ class LLVMContext {
OB_convergencectrl = 9, // "convergencectrl"
OB_align = 10, // "align"
OB_deactivation_symbol = 11, // "deactivation-symbol"
- OB_LastBundleID = OB_deactivation_symbol
+ OB_fp_control = 12, // "fp.control"
+ OB_fp_except = 13, // "fp.except"
+ OB_LastBundleID = OB_fp_except
};
/// getMDKindID - Return a unique non-zero ID for the specified metadata kind.
diff --git a/llvm/include/llvm/Support/ModRef.h b/llvm/include/llvm/Support/ModRef.h
index b28d878df314a..bede5e88de55f 100644
--- a/llvm/include/llvm/Support/ModRef.h
+++ b/llvm/include/llvm/Support/ModRef.h
@@ -252,6 +252,11 @@ template <typename LocationEnum> class MemoryEffectsBase {
return ME.getWithoutLoc(Location::InaccessibleMem).doesNotAccessMemory();
}
+ /// Whether this function accesses inaccessible memory.
+ bool doesAccessInaccessibleMem() const {
+ return isModOrRefSet(getModRef(Location::InaccessibleMem));
+ }
+
/// Whether this function only (at most) accesses errno memory.
bool onlyAccessesErrnoMem() const {
return getWithoutLoc(Location::ErrnoMem).doesNotAccessMemory();
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index c41d7b3181a37..aa862cc2bef61 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -35,6 +35,18 @@ llvm::convertStrToRoundingMode(StringRef RoundingArg) {
.Default(std::nullopt);
}
+std::optional<RoundingMode>
+llvm::convertBundleToRoundingMode(StringRef RoundingArg) {
+ return StringSwitch<std::optional<RoundingMode>>(RoundingArg)
+ .Case("dyn", RoundingMode::Dynamic)
+ .Case("rte", RoundingMode::NearestTiesToEven)
+ .Case("rmm", RoundingMode::NearestTiesToAway)
+ .Case("rtn", RoundingMode::TowardNegative)
+ .Case("rtp", RoundingMode::TowardPositive)
+ .Case("rtz", RoundingMode::TowardZero)
+ .Default(std::nullopt);
+}
+
std::optional<StringRef>
llvm::convertRoundingModeToStr(RoundingMode UseRounding) {
std::optional<StringRef> RoundingStr;
@@ -63,6 +75,34 @@ llvm::convertRoundingModeToStr(RoundingMode UseRounding) {
return RoundingStr;
}
+std::optional<StringRef>
+llvm::convertRoundingModeToBundle(RoundingMode UseRounding) {
+ std::optional<StringRef> RoundingStr;
+ switch (UseRounding) {
+ case RoundingMode::Dynamic:
+ RoundingStr = "dyn";
+ break;
+ case RoundingMode::NearestTiesToEven:
+ RoundingStr = "rte";
+ break;
+ case RoundingMode::NearestTiesToAway:
+ RoundingStr = "rmm";
+ break;
+ case RoundingMode::TowardNegative:
+ RoundingStr = "rtn";
+ break;
+ case RoundingMode::TowardPositive:
+ RoundingStr = "rtp";
+ break;
+ case RoundingMode::TowardZero:
+ RoundingStr = "rtz";
+ break;
+ default:
+ break;
+ }
+ return RoundingStr;
+}
+
std::optional<fp::ExceptionBehavior>
llvm::convertStrToExceptionBehavior(StringRef ExceptionArg) {
return StringSwitch<std::optional<fp::ExceptionBehavior>>(ExceptionArg)
@@ -72,6 +112,15 @@ llvm::convertStrToExceptionBehavior(StringRef ExceptionArg) {
.Default(std::nullopt);
}
+std::optional<fp::ExceptionBehavior>
+llvm::convertBundleToExceptionBehavior(StringRef ExceptionArg) {
+ return StringSwitch<std::optional<fp::ExceptionBehavior>>(ExceptionArg)
+ .Case("ignore", fp::ebIgnore)
+ .Case("maytrap", fp::ebMayTrap)
+ .Case("strict", fp::ebStrict)
+ .Default(std::nullopt);
+}
+
std::optional<StringRef>
llvm::convertExceptionBehaviorToStr(fp::ExceptionBehavior UseExcept) {
std::optional<StringRef> ExceptStr;
@@ -89,6 +138,23 @@ llvm::convertExceptionBehaviorToStr(fp::ExceptionBehavior UseExcept) {
return ExceptStr;
}
+std::optional<StringRef>
+llvm::convertExceptionBehaviorToBundle(fp::ExceptionBehavior UseExcept) {
+ std::optional<StringRef> ExceptStr;
+ switch (UseExcept) {
+ case fp::ebStrict:
+ ExceptStr = "strict";
+ break;
+ case fp::ebIgnore:
+ ExceptStr = "ignore";
+ break;
+ case fp::ebMayTrap:
+ ExceptStr = "maytrap";
+ break;
+ }
+ return ExceptStr;
+}
+
Intrinsic::ID llvm::getConstrainedIntrinsicID(const Instruction &Instr) {
Intrinsic::ID IID = Intrinsic::not_intrinsic;
switch (Instr.getOpcode()) {
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 13e467b05f691..3a10ea7ad6c61 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -186,6 +186,67 @@ IRBuilderBase::createCallHelper(Function *Callee, ArrayRef<Value *> Ops,
return CI;
}
+CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
+ ArrayRef<Value *> Args,
+ ArrayRef<OperandBundleDef> OpBundles,
+ const Twine &Name, MDNode *FPMathTag) {
+ assert(std::count_if(OpBundles.begin(), OpBundles.end(),
+ [](const OperandBundleDef &Item) {
+ return Item.getTag() == "fp.control";
+ }) <= 1);
+ assert(std::count_if(OpBundles.begin(), OpBundles.end(),
+ [](const OperandBundleDef &Item) {
+ return Item.getTag() == "fp.except";
+ }) <= 1);
+
+ ArrayRef<OperandBundleDef> ActualBundlesRef = OpBundles;
+ SmallVector<OperandBundleDef, 2> ActualBundles;
+
+ // If the builder is in strictfp mode and has non-default options (like
+ // non-dynamic rounding), add corresponding operand bundle. If such bundle is
+ // already present, assume it overwrites defaults.
+ bool doesTheCallAccessFPEnv = false;
+ if (IsFPConstrained) {
+ if (const auto *Func = dyn_cast<Function>(Callee)) {
+ if (Intrinsic::ID ID = Func->getIntrinsicID()) {
+ if (IntrinsicInst::isFloatingPointOperation(ID)) {
+ doesTheCallAccessFPEnv = true;
+ bool NeedRound = DefaultConstrainedRounding != RoundingMode::Dynamic;
+ bool NeedExcept = DefaultConstrainedExcept != fp::ebStrict;
+ for (const auto &Item : OpBundles) {
+ if (NeedRound && Item.getTag() == "fp.control")
+ NeedRound = false;
+ else if (NeedExcept && Item.getTag() == "fp.except")
+ NeedExcept = false;
+ ActualBundles.push_back(Item);
+ }
+ if (NeedRound)
+ createFPRoundingBundle(ActualBundles, DefaultConstrainedRounding);
+ if (NeedExcept)
+ createFPExceptionBundle(ActualBundles, DefaultConstrainedExcept);
+ ActualBundlesRef = ActualBundles;
+ }
+ }
+ }
+ }
+
+ // If the call accesses FPE, update memory effects accordingly.
+ CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
+ if (doesTheCallAccessFPEnv) {
+ MemoryEffects ME = MemoryEffects::inaccessibleMemOnly();
+ if (CI->getAttributes().hasFnAttr(Attribute::Memory))
+ ME |= CI->getAttributes().getMemoryEffects();
+ auto A = Attribute::getWithMemoryEffects(getContext(), ME);
+ CI->addFnAttr(A);
+ }
+
+ if (IsFPConstrained)
+ setConstrainedFPCallAttr(CI);
+ if (isa<FPMathOperator>(CI))
+ setFPAttrs(CI, FPMathTag, FMF);
+ return Insert(CI, Name);
+}
+
static Value *CreateVScaleMultiple(IRBuilderBase &B, Type *Ty, uint64_t Scale) {
Value *VScale = B.CreateVScale(Ty);
if (Scale == 1)
@@ -1396,6 +1457,20 @@ CallInst *IRBuilderBase::CreateDereferenceableAssumption(Value *PtrValue,
{DereferenceableOpB});
}
+void IRBuilderBase::createFPRoundingBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<RoundingMode> Rounding) {
+ addFPRoundingBundle(Context, Bundles,
+ Rounding.value_or(DefaultConstrainedRounding));
+}
+
+void IRBuilderBase::createFPExceptionBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<fp::ExceptionBehavior> Except) {
+ addFPExceptionBundle(Context, Bundles,
+ Except.value_or(DefaultConstrainedExcept));
+}
+
IRBuilderDefaultInserter::~IRBuilderDefaultInserter() = default;
IRBuilderCallbackInserter::~IRBuilderCallbackInserter() = default;
IRBuilderFolder::~IRBuilderFolder() = default;
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index 52b6c35e18d95..241dd6e508a43 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -613,10 +613,11 @@ bool CallBase::hasReadingOperandBundles() const {
// Implementation note: this is a conservative implementation of operand
// bundle semantics, where *any* non-assume operand bundle (other than
// ptrauth) forces a callsite to be at least readonly.
- return hasOperandBundlesOtherThan({LLVMContext::OB_ptrauth,
- LLVMContext::OB_kcfi,
- LLVMContext::OB_convergencectrl,
- LLVMContext::OB_deactivation_symbol}) &&
+ return hasOperandBundlesOtherThan(
+ {LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
+ LLVMContext::OB_convergencectrl,
+ LLVMContext::OB_deactivation_symbol, LLVMContext::OB_fp_control,
+ LLVMContext::OB_fp_except}) &&
getIntrinsicID() != Intrinsic::assume;
}
@@ -625,14 +626,70 @@ bool CallBase::hasClobberingOperandBundles() const {
{LLVMContext::OB_deopt, LLVMContext::OB_funclet,
LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
LLVMContext::OB_convergencectrl,
- LLVMContext::OB_deactivation_symbol}) &&
+ LLVMContext::OB_deactivation_symbol, LLVMContext::OB_fp_control,
+ LLVMContext::OB_fp_except}) &&
getIntrinsicID() != Intrinsic::assume;
}
+RoundingMode CallBase::getRoundingMode() const {
+ // Try reading rounding mode from FP bundle.
+ std::optional<RoundingMode> RM;
+ if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_control)) {
+ Value *V = RoundingBundle->Inputs.front();
+ Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
+ RM = convertBundleToRoundingMode(cast<MDString>(MD)->getString());
+ }
+ if (RM)
+ return *RM;
+
+ // No FP bundle, try to guess from the current mode.
+ if (getParent())
+ if (auto *F = getFunction(); F)
+ return F->getAttributes().hasFnAttr(Attribute::StrictFP)
+ ? RoundingMode::Dynamic
+ : RoundingMode::NearestTiesToEven;
+
+ // Isolated call. Assume default environment.
+ return RoundingMode::NearestTiesToEven;
+}
+
+std::optional<fp::ExceptionBehavior> CallBase::getExceptionBehavior() const {
+ // Try determining exception behavior from FP bundle.
+ std::optional<fp::ExceptionBehavior> EB;
+ if (auto ExceptionBundle = getOperandBundle(LLVMContext::OB_fp_except)) {
+ Value *V = ExceptionBundle->Inputs.front();
+ Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
+ EB = convertBundleToExceptionBehavior(cast<MDString>(MD)->getString());
+ }
+ if (EB)
+ return *EB;
+
+ // No FP bundle, try to guess from the current mode.
+ if (getParent())
+ if (auto *F = getFunction(); F)
+ return F->getAttributes().hasFnAttr(Attribute::StrictFP) ? fp::ebStrict
+ : fp::ebIgnore;
+
+ // Isolated call. Assume default environment.
+ return fp::ebIgnore;
+}
+
+MemoryEffects CallBase::getFloatingPointMemoryEffects() const {
+ if (Intrinsic::ID IntrID = getIntrinsicID())
+ if (const BasicBlock *BB = getParent())
+ if (const Function *F = BB->getParent())
+ if (F->hasFnAttribute(Attribute::StrictFP))
+ if (IntrinsicInst::isFloatingPointOperation(IntrID)) {
+ return MemoryEffects::inaccessibleMemOnly();
+ }
+ return MemoryEffects::none();
+}
+
MemoryEffects CallBase::getMemoryEffects() const {
MemoryEffects ME = getAttributes().getMemoryEffects();
if (auto *Fn = dyn_cast<Function>(getCalledOperand())) {
MemoryEffects FnME = Fn->getMemoryEffects();
+ FnME |= getFloatingPointMemoryEffects();
if (hasOperandBundles()) {
// TODO: Add a method to get memory effects for operand bundles instead.
if (hasReadingOperandBundles())
@@ -741,6 +798,26 @@ bool CallBase::hasArgumentWithAdditionalReturnCaptureComponents() const {
return false;
}
+void llvm::addFPRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding) {
+ std::optional<StringRef> RndStr = convertRoundingModeToBundle(Rounding);
+ assert(RndStr && "Garbage rounding mode!");
+ auto *RoundingMDS = MDString::get(Ctx, *RndStr);
+ auto *RM = MetadataAsValue::get(Ctx, RoundingMDS);
+ Bundles.emplace_back("fp.control", RM);
+}
+
+void llvm::addFPExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except) {
+ std::optional<StringRef> ExcStr = convertExceptionBehaviorToBundle(Except);
+ assert(ExcStr && "Garbage exception behavior!");
+ auto *ExceptMDS = MDString::get(Ctx, *ExcStr);
+ auto *EB = MetadataAsValue::get(Ctx, ExceptMDS);
+ Bundles.emplace_back("fp.except", EB);
+}
+
//===----------------------------------------------------------------------===//
// CallInst Implementation
//===----------------------------------------------------------------------===//
diff --git a/llvm/lib/IR/IntrinsicInst.cpp b/llvm/lib/IR/IntrinsicInst.cpp
index 281cbd4388b58..8d7c781e9f47a 100644
--- a/llvm/lib/IR/IntrinsicInst.cpp
+++ b/llvm/lib/IR/IntrinsicInst.cpp
@@ -66,6 +66,16 @@ bool IntrinsicInst::mayLowerToFunctionCall(Intrinsic::ID IID) {
}
}
+bool IntrinsicInst::isFloatingPointOperation(Intrinsic::ID IID) {
+ switch (IID) {
+#define FUNCTION(NAME, D) case Intrinsic::NAME:
+#include "llvm/IR/FloatingPointOps.def"
+ return true;
+ default:
+ return false;
+ }
+}
+
//===----------------------------------------------------------------------===//
/// DbgVariableIntrinsic - This is the common base class for debug info
/// intrinsics for variables.
diff --git a/llvm/lib/IR/LLVMContext.cpp b/llvm/lib/IR/LLVMContext.cpp
index 10aba759185a7..e074c81c44eb9 100644
--- a/llvm/lib/IR/LLVMContext.cpp
+++ b/llvm/lib/IR/LLVMContext.cpp
@@ -57,6 +57,10 @@ static StringRef knownBundleName(unsigned BundleTagID) {
return "align";
case LLVMContext::OB_deactivation_symbol:
return "deactivation-symbol";
+ case LLVMContext::OB_fp_control:
+ return "fp.control";
+ case LLVMContext::OB_fp_except:
+ return "fp.except";
default:
llvm_unreachable("unknown bundle id");
}
@@ -86,6 +90,16 @@ LLVMContext::LLVMContext() : pImpl(new LLVMContextImpl(*this)) {
assert(Entry->second == BundleTagID && "operand bundle id drifted!");
}
+ auto *RoundingEntry = pImpl->getOrInsertBundleTag("fp.control");
+ assert(RoundingEntry->second == LLVMContext::OB_fp_control &&
+ "fp.control operand bundle id drifted!");
+ (void)RoundingEntry;
+
+ auto *ExceptionEntry = pImpl->getOrInsertBundleTag("fp.except");
+ assert(ExceptionEntry->second == LLVMContext::OB_fp_except &&
+ "fp.except operand bundle id drifted!");
+ (void)ExceptionEntry;
+
SyncScope::ID SingleThreadSSID =
pImpl->getOrInsertSyncScopeID("singlethread");
assert(SingleThreadSSID == SyncScope::SingleThread &&
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index f986f5406b2b3..eb5e9b255e025 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -4055,7 +4055,8 @@ void Verifier::visitCallBase(CallBase &Call) {
FoundGCTransitionBundle = false, FoundCFGuardTargetBundle = false,
FoundPreallocatedBundle = false, FoundGCLiveBundle = false,
FoundPtrauthBundle = false, FoundKCFIBundle = false,
- FoundAttachedCallBundle = false;
+ FoundAttachedCallBundle = false, FoundFpeControlBundle = false,
+ FoundFpeExceptBundle = false;
for (unsigned i = 0, e = Call.getNumOperandBundles(); i < e; ++i) {
OperandBundleUse BU = Call.getOperandBundleAt(i);
uint32_t Tag = BU.getTagID();
@@ -4118,6 +4119,34 @@ void Verifier::visitCallBase(CallBase &Call) {
"Multiple \"clang.arc.attachedcall\" operand bundles", Call);
FoundAttachedCallBundle = true;
verifyAttachedCallBundle(Call, BU);
+ } else if (Tag == LLVMContext::OB_fp_control) {
+ Check(!FoundFpeControlBundle, "Multiple fp.control operand bundles",
+ Call);
+ Check(BU.Inputs.size() == 1,
+ "Expected exactly one fp.control bundle operand", Call);
+ auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
+ Check(V, "Value of fp.control bundle operand must be a metadata", Call);
+ auto *MDS = dyn_cast<MDString>(V->getMetadata());
+ Check(MDS, "Value of fp.control bundle operand must be a string", Call);
+ auto RM = convertBundleToRoundingMode(MDS->getString());
+ Check(RM.has_value(),
+ "Value of fp.control bundle operand is not a correct rounding mode",
+ Call);
+ FoundFpeControlBundle = true;
+ } else if (Tag == LLVMContext::OB_fp_except) {
+ Check(!FoundFpeExceptBundle, "Multiple fp.except operand bundles", Call);
+ Check(BU.Inputs.size() == 1,
+ "Expected exactly one fp.except bundle operand", Call);
+ auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
+ Check(V, "Value of fp.except bundle operand must be a metadata", Call);
+ auto *MDS = dyn_cast<MDString>(V->getMetadata());
+ Check(MDS, "Value of fp.except bundle operand must be a string", Call);
+ auto EB = convertBundleToExceptionBehavior(MDS->getString());
+ Check(EB.has_value(),
+ "Value of fp.except bundle operand is not a correct exception "
+ "behavior",
+ Call);
+ FoundFpeExceptBundle = true;
}
}
diff --git a/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll b/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll
index 01e5b3f6673ae..df008e20f9f36 100644
--- a/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll
+++ b/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll
@@ -15,6 +15,8 @@
; CHECK-NEXT: <OPERAND_BUNDLE_TAG
; CHECK-NEXT: <OPERAND_BUNDLE_TAG
; CHECK-NEXT: <OPERAND_BUNDLE_TAG
+; CHECK-NEXT: <OPERAND_BUNDLE_TAG
+; CHECK-NEXT: <OPERAND_BUNDLE_TAG
; CHECK-NEXT: </OPERAND_BUNDLE_TAGS_BLOCK
; CHECK: <FUNCTION_BLOCK
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index 4934843d5a2ed..12b0fca97e5af 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -51,4 +51,95 @@ entry:
ret double %fadd
}
+; Test multiple fp.control bundles.
+; CHECK-NEXT: Multiple fp.control operand bundles
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+define double @f6(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that has more than one operands.
+; CHECK-NEXT: Expected exactly one fp.control bundle operand
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+define double @f7(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that has non-metadata operand.
+; CHECK-NEXT: Value of fp.control bundle operand must be a metadata
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(i32 0) ]
+define double @f8(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(i32 0) ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that has non-string operand.
+; CHECK-NEXT: Value of fp.control bundle operand must be a string
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata i64 3) ]
+define double @f9(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !{i64 3}) ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that specifies incorrect value.
+; CHECK-NEXT: Value of fp.control bundle operand is not a correct rounding mode
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"qqq") ]
+define double @f10(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"qqq") ]
+ ret double %ftrunc
+}
+
+; Test multiple fp.except bundles.
+; CHECK-NEXT: Multiple fp.except operand bundles
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict"), "fp.except"(metadata !"strict") ]
+define double @f11(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !"strict"), "fp.except"(metadata !"strict") ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that has more than one operands.
+; CHECK-NEXT: Expected exactly one fp.except bundle operand
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict", metadata !"strict") ]
+define double @f12(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !"strict", metadata !"strict") ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that has non-metadata operand.
+; CHECK-NEXT: Value of fp.except bundle operand must be a metadata
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(i32 0) ]
+define double @f13(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(i32 0) ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that has non-string operand.
+; CHECK-NEXT: Value of fp.except bundle operand must be a string
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata i64 3) ]
+define double @f14(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !{i64 3}) ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that specifies incorrect value.
+; CHECK-NEXT: Value of fp.except bundle operand is not a correct exception behavior
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"qqq") ]
+define double @f15(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !"qqq") ]
+ ret double %ftrunc
+}
+
+
attributes #0 = { strictfp }
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index 4361594050668..84f44eb0a6499 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -515,6 +515,203 @@ TEST_F(IRBuilderTest, ConstrainedFPFunctionCall) {
EXPECT_FALSE(verifyModule(*M));
}
+TEST_F(IRBuilderTest, FPBundlesDefault) {
+ IRBuilder<> Builder(BB);
+ GlobalVariable *GVDouble = new GlobalVariable(
+ *M, Type::getDoubleTy(Ctx), true, GlobalValue::ExternalLinkage, nullptr);
+ Value *FnArg = Builder.CreateLoad(GVDouble->getValueType(), GVDouble);
+ Function *Fn = Intrinsic::getOrInsertDeclaration(
+ M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
+
+ // A floating-point operation does not have side effects in default
+ // environment even.
+ {
+ Value *V = Builder.CreateCall(Fn, {FnArg});
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // Check call with FP bundles, rounding is set to default value.
+ // nearbyint(%x) [ "fp.control" (metadata !"rte") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // Check call with FP bundles, exception behavior is set to default value.
+ // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // Check call with FP bundles, both rounding mode and exception behavior are
+ // set.
+ // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+}
+
+TEST_F(IRBuilderTest, FPBundlesStrict) {
+ F->addFnAttr(Attribute::StrictFP);
+
+ IRBuilder<> Builder(BB);
+ Builder.setDefaultConstrainedExcept(fp::ebStrict);
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardZero);
+ Builder.setIsFPConstrained(true);
+
+ GlobalVariable *GVDouble = new GlobalVariable(
+ *M, Type::getDoubleTy(Ctx), true, GlobalValue::ExternalLinkage, nullptr);
+ Value *FnArg = Builder.CreateLoad(GVDouble->getValueType(), GVDouble);
+ Function *Fn = Intrinsic::getOrInsertDeclaration(
+ M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
+
+ // A floating-point operation has side effects in strictfp environment even
+ // if it has no FP bundles.
+ {
+ Value *V = Builder.CreateCall(Fn, {FnArg});
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, with default (dynamic) rounding mode
+ // nearbyint(%x) [ "fp.control" (metadata !"dyn") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, with specific rounding mode
+ // nearbyint(%x) [ "fp.control" (metadata !"rtz") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, exception behavior is set to default value.
+ // nearbyint(%x) [ "fp.except" (metadata !"strict") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebStrict);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, exception behavior is set to specific value.
+ // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with both FP bundles.
+ // nearbyint(%x) [ "fp.control" (metadata !"rtz"),
+ // "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Function calls, that do not depend on FP options, does not have
+ // memory effects.
+ {
+ Function *Fn = Intrinsic::getOrInsertDeclaration(M.get(), Intrinsic::fabs,
+ {Type::getDoubleTy(Ctx)});
+ Value *V = Builder.CreateCall(Fn, {FnArg});
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+}
+
TEST_F(IRBuilderTest, Lifetime) {
IRBuilder<> Builder(BB);
AllocaInst *Var1 = Builder.CreateAlloca(Builder.getInt8Ty());
>From f3cefd3a9951740d20ce8449fbd016ca8bcadf68 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 17 Apr 2025 14:00:55 +0700
Subject: [PATCH 02/24] Set MemoryEffects of call only if called function is
nomem
It make call close to constrained calls, where memory effects of the
calls do not define memory effects. It helps migration to operand
bundles.
---
llvm/lib/IR/IRBuilder.cpp | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 3a10ea7ad6c61..a93b652447a17 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -205,12 +205,13 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
// If the builder is in strictfp mode and has non-default options (like
// non-dynamic rounding), add corresponding operand bundle. If such bundle is
// already present, assume it overwrites defaults.
- bool doesTheCallAccessFPEnv = false;
+ bool NeedUpdateMemoryEffects = false;
if (IsFPConstrained) {
if (const auto *Func = dyn_cast<Function>(Callee)) {
if (Intrinsic::ID ID = Func->getIntrinsicID()) {
if (IntrinsicInst::isFloatingPointOperation(ID)) {
- doesTheCallAccessFPEnv = true;
+ MemoryEffects FME = Func->getMemoryEffects();
+ NeedUpdateMemoryEffects = !FME.doesAccessInaccessibleMem();
bool NeedRound = DefaultConstrainedRounding != RoundingMode::Dynamic;
bool NeedExcept = DefaultConstrainedExcept != fp::ebStrict;
for (const auto &Item : OpBundles) {
@@ -232,10 +233,8 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
// If the call accesses FPE, update memory effects accordingly.
CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
- if (doesTheCallAccessFPEnv) {
+ if (NeedUpdateMemoryEffects) {
MemoryEffects ME = MemoryEffects::inaccessibleMemOnly();
- if (CI->getAttributes().hasFnAttr(Attribute::Memory))
- ME |= CI->getAttributes().getMemoryEffects();
auto A = Attribute::getWithMemoryEffects(getContext(), ME);
CI->addFnAttr(A);
}
>From 6510dfa0c2ac9abb4bba155acdb58092447d2ec2 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 17 Apr 2025 15:30:17 +0700
Subject: [PATCH 03/24] Fix method signature
---
llvm/include/llvm/IR/InstrTypes.h | 6 +++---
llvm/lib/IR/Instructions.cpp | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index d438a4c53a7c1..97ef5fcd35769 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -2172,11 +2172,11 @@ class CallBase : public Instruction {
return false;
}
- /// Return rounding mode specified by operand bundles.
+ /// Return rounding mode specified for this call.
RoundingMode getRoundingMode() const;
- /// Return exception behavior specified by operand bundles.
- std::optional<fp::ExceptionBehavior> getExceptionBehavior() const;
+ /// Return exception behavior specified for this call.
+ fp::ExceptionBehavior getExceptionBehavior() const;
/// Used to keep track of an operand bundle. See the main comment on
/// OperandBundleUser above.
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index 241dd6e508a43..a7b463fd496b6 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -653,7 +653,7 @@ RoundingMode CallBase::getRoundingMode() const {
return RoundingMode::NearestTiesToEven;
}
-std::optional<fp::ExceptionBehavior> CallBase::getExceptionBehavior() const {
+fp::ExceptionBehavior CallBase::getExceptionBehavior() const {
// Try determining exception behavior from FP bundle.
std::optional<fp::ExceptionBehavior> EB;
if (auto ExceptionBundle = getOperandBundle(LLVMContext::OB_fp_except)) {
>From 1c06b0118b421ce1a11938c9b1c5068ec3f38364 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Fri, 18 Apr 2025 01:55:35 +0700
Subject: [PATCH 04/24] Organize checks in Verifier
---
llvm/lib/IR/Verifier.cpp | 52 ++++++++++++++++++-----------
llvm/test/Verifier/fp-intrinsics.ll | 22 ++++++------
2 files changed, 44 insertions(+), 30 deletions(-)
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index eb5e9b255e025..088cfabe3339e 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -4120,32 +4120,46 @@ void Verifier::visitCallBase(CallBase &Call) {
FoundAttachedCallBundle = true;
verifyAttachedCallBundle(Call, BU);
} else if (Tag == LLVMContext::OB_fp_control) {
- Check(!FoundFpeControlBundle, "Multiple fp.control operand bundles",
- Call);
- Check(BU.Inputs.size() == 1,
- "Expected exactly one fp.control bundle operand", Call);
- auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
- Check(V, "Value of fp.control bundle operand must be a metadata", Call);
- auto *MDS = dyn_cast<MDString>(V->getMetadata());
- Check(MDS, "Value of fp.control bundle operand must be a string", Call);
- auto RM = convertBundleToRoundingMode(MDS->getString());
- Check(RM.has_value(),
- "Value of fp.control bundle operand is not a correct rounding mode",
+ Check(!FoundFpeControlBundle, "Multiple \"fp.control\" operand bundles",
Call);
+ bool FoundRoundingMode = false;
+ for (auto &U : BU.Inputs) {
+ Value *V = U.get();
+ Check(isa<MetadataAsValue>(V),
+ "Value of a \"fp.control\" bundle operand must be a metadata",
+ Call);
+ Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
+ Check(isa<MDString>(MD),
+ "Value of a \"fp.control\" bundle operand must be a string",
+ Call);
+ StringRef Item = cast<MDString>(MD)->getString();
+ if (convertBundleToRoundingMode(Item)) {
+ Check(!FoundRoundingMode, "Rounding mode is specified more that once",
+ Call);
+ FoundRoundingMode = true;
+ } else {
+ CheckFailed("Unrecognized value in \"fp.control\" bundle operand",
+ Call);
+ }
+ }
FoundFpeControlBundle = true;
} else if (Tag == LLVMContext::OB_fp_except) {
- Check(!FoundFpeExceptBundle, "Multiple fp.except operand bundles", Call);
+ Check(!FoundFpeExceptBundle, "Multiple \"fp.except\" operand bundles",
+ Call);
Check(BU.Inputs.size() == 1,
- "Expected exactly one fp.except bundle operand", Call);
+ "Expected exactly one \"fp.except\" bundle operand", Call);
auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
- Check(V, "Value of fp.except bundle operand must be a metadata", Call);
+ Check(V, "Value of a \"fp.except\" bundle operand must be a metadata",
+ Call);
auto *MDS = dyn_cast<MDString>(V->getMetadata());
- Check(MDS, "Value of fp.except bundle operand must be a string", Call);
- auto EB = convertBundleToExceptionBehavior(MDS->getString());
- Check(EB.has_value(),
- "Value of fp.except bundle operand is not a correct exception "
- "behavior",
+ Check(MDS, "Value of a \"fp.except\" bundle operand must be a string",
Call);
+ auto EB = convertBundleToExceptionBehavior(MDS->getString());
+ Check(
+ EB.has_value(),
+ "Value of a \"fp.except\" bundle operand is not a correct exception "
+ "behavior",
+ Call);
FoundFpeExceptBundle = true;
}
}
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index 12b0fca97e5af..1e1df1b0da96a 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -52,7 +52,7 @@ entry:
}
; Test multiple fp.control bundles.
-; CHECK-NEXT: Multiple fp.control operand bundles
+; CHECK-NEXT: Multiple "fp.control" operand bundles
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
define double @f6(double %a) #0 {
entry:
@@ -60,8 +60,8 @@ entry:
ret double %ftrunc
}
-; Test fp.control bundle that has more than one operands.
-; CHECK-NEXT: Expected exactly one fp.control bundle operand
+; Test fp.control bundle that has more than one rounding mode specification.
+; CHECK-NEXT: Rounding mode is specified more that once
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz", metadata !"rte") ]
define double @f7(double %a) #0 {
entry:
@@ -70,7 +70,7 @@ entry:
}
; Test fp.control bundle that has non-metadata operand.
-; CHECK-NEXT: Value of fp.control bundle operand must be a metadata
+; CHECK-NEXT: Value of a "fp.control" bundle operand must be a metadata
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(i32 0) ]
define double @f8(double %a) #0 {
entry:
@@ -79,7 +79,7 @@ entry:
}
; Test fp.control bundle that has non-string operand.
-; CHECK-NEXT: Value of fp.control bundle operand must be a string
+; CHECK-NEXT: Value of a "fp.control" bundle operand must be a string
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata i64 3) ]
define double @f9(double %a) #0 {
entry:
@@ -88,7 +88,7 @@ entry:
}
; Test fp.control bundle that specifies incorrect value.
-; CHECK-NEXT: Value of fp.control bundle operand is not a correct rounding mode
+; CHECK-NEXT: Unrecognized value in "fp.control" bundle operand
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"qqq") ]
define double @f10(double %a) #0 {
entry:
@@ -97,7 +97,7 @@ entry:
}
; Test multiple fp.except bundles.
-; CHECK-NEXT: Multiple fp.except operand bundles
+; CHECK-NEXT: Multiple "fp.except" operand bundles
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict"), "fp.except"(metadata !"strict") ]
define double @f11(double %a) #0 {
entry:
@@ -106,7 +106,7 @@ entry:
}
; Test fp.except bundle that has more than one operands.
-; CHECK-NEXT: Expected exactly one fp.except bundle operand
+; CHECK-NEXT: Expected exactly one "fp.except" bundle operand
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict", metadata !"strict") ]
define double @f12(double %a) #0 {
entry:
@@ -115,7 +115,7 @@ entry:
}
; Test fp.except bundle that has non-metadata operand.
-; CHECK-NEXT: Value of fp.except bundle operand must be a metadata
+; CHECK-NEXT: Value of a "fp.except" bundle operand must be a metadata
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(i32 0) ]
define double @f13(double %a) #0 {
entry:
@@ -124,7 +124,7 @@ entry:
}
; Test fp.except bundle that has non-string operand.
-; CHECK-NEXT: Value of fp.except bundle operand must be a string
+; CHECK-NEXT: Value of a "fp.except" bundle operand must be a string
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata i64 3) ]
define double @f14(double %a) #0 {
entry:
@@ -133,7 +133,7 @@ entry:
}
; Test fp.except bundle that specifies incorrect value.
-; CHECK-NEXT: Value of fp.except bundle operand is not a correct exception behavior
+; CHECK-NEXT: Value of a "fp.except" bundle operand is not a correct exception behavior
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"qqq") ]
define double @f15(double %a) #0 {
entry:
>From 6a4b5a045b6e52321b85cb1b920ee781b46dbe8d Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Sat, 19 Apr 2025 02:05:28 +0700
Subject: [PATCH 05/24] Small changes in documentation
---
llvm/docs/LangRef.rst | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 7c749c538bc03..59a4f575daf82 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3297,8 +3297,9 @@ floating-point control modes and the treatment of status bits respectively.
An operand bundle tagged with "fp.control" contains information about the
control modes used for the operation execution. Operands specified in this
bundle represent particular options. Currently, only rounding mode is supported.
-It is represented by a metadata string value, which specifies the rounding mode
-to be used for the operation evaluation. Possible values are:
+
+Rounding mode is represented by a metadata string value, which specifies the
+the mode used for the operation evaluation. Possible values are:
::
@@ -3309,10 +3310,10 @@ to be used for the operation evaluation. Possible values are:
"rmm" - to nearest, ties away from zero
"dyn" - rounding mode is taken from control register
-If "fp.control" is absent, the default rounding rounding mode is taken from the
-control register (dynamic rounding). In the particular case of
-:ref:`default floating-point environment <floatenv>`, it must be rounding to
-nearest, ties to even.
+Only one such value may be specified. If "fp.control" is absent, the default
+rounding rounding mode is taken from the control register (dynamic rounding).
+In the particular case of :ref:`default floating-point environment <floatenv>`,
+the operation uses rounding to nearest, ties to even.
An operand bundle tagged with "fp.except" may be associated with operations
that can read or write floating-point exception flags. It contains a single
>From ccf676483d578eeeabdc13f3e7a8f690031343c6 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Wed, 23 Jul 2025 13:16:45 +0700
Subject: [PATCH 06/24] Update according to the obtained feedback
- Bundles are renamed: "fp.control" -> "fp.round",
- Corrected documentation,
- DAG nodes STRICT_* are used to lower the supported intrinsics,
- Constrained properties in IRBuilder are reused for handling FP bundles,
- DAG gets methods for lowering FP operations,
- IR dump prints FP memory effects,
- Fixed some problems in IRBuilderBase::CreateCall,
- Added a new check to Verifier,
- Added tests for checking code generation.
---
clang/lib/CodeGen/CodeGenFunction.cpp | 2 +-
llvm/docs/LangRef.rst | 53 ++---
llvm/include/llvm/IR/FloatingPointOps.def | 16 +-
llvm/include/llvm/IR/IRBuilder.h | 35 ++--
llvm/include/llvm/IR/InstrTypes.h | 22 ++-
llvm/include/llvm/IR/LLVMContext.h | 2 +-
.../SelectionDAG/SelectionDAGBuilder.cpp | 86 +++++++-
.../SelectionDAG/SelectionDAGBuilder.h | 3 +
llvm/lib/IR/AsmWriter.cpp | 25 +++
llvm/lib/IR/FPEnv.cpp | 24 +--
llvm/lib/IR/IRBuilder.cpp | 87 +++++----
llvm/lib/IR/Instructions.cpp | 67 ++++---
llvm/lib/IR/LLVMContext.cpp | 10 +-
llvm/lib/IR/Verifier.cpp | 24 ++-
.../CodeGen/X86/fp-strict-scalar-round.ll | 184 ++++++++++++++++++
llvm/test/CodeGen/X86/vec-strict-512.ll | 36 ++++
llvm/test/Verifier/fp-intrinsics.ll | 47 +++--
llvm/unittests/IR/IRBuilderTest.cpp | 91 ++++++---
18 files changed, 615 insertions(+), 199 deletions(-)
diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp
index b14d9d7e8d060..2112bb6729714 100644
--- a/clang/lib/CodeGen/CodeGenFunction.cpp
+++ b/clang/lib/CodeGen/CodeGenFunction.cpp
@@ -1089,7 +1089,7 @@ void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
if ((FD && (FD->UsesFPIntrin() || FD->hasAttr<StrictFPAttr>())) ||
(!FD && (FPExceptionBehavior != llvm::fp::ebIgnore ||
RM != llvm::RoundingMode::NearestTiesToEven))) {
- Builder.setIsFPConstrained(true);
+ Builder.setIsFPConstrained(true, false);
Fn->addFnAttr(llvm::Attribute::StrictFP);
}
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 59a4f575daf82..c5869daf2a05c 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3291,43 +3291,48 @@ Floating-point Operand Bundles
These operand bundles are used for calls that involve floating-point
operations and interact with :ref:`floating-point environment <floatenv>` or
depend on floating-point options, such as rounding mode, denormal modes, etc.
-There are two kinds of such operand bundles, which represent the value of
-floating-point control modes and the treatment of status bits respectively.
-An operand bundle tagged with "fp.control" contains information about the
-control modes used for the operation execution. Operands specified in this
-bundle represent particular options. Currently, only rounding mode is supported.
-
-Rounding mode is represented by a metadata string value, which specifies the
-the mode used for the operation evaluation. Possible values are:
+An operand bundle tagged with "fp.round" contains information about the
+rounding mode used for the operation execution (the effective rounding mode).
+This mode is represented by a metadata string value and specifies the mode used
+for the operation evaluation. Possible values are:
::
- "rtz" - toward zero
- "rte" - to nearest, ties to even
- "rtp" - toward positive infinity
- "rtn" - toward negative infinity
- "rmm" - to nearest, ties away from zero
- "dyn" - rounding mode is taken from control register
+ "towardzero"
+ "tonearest"
+ "upward"
+ "downward"
+ "tonearestaway"
+ "dynamic"
-Only one such value may be specified. If "fp.control" is absent, the default
-rounding rounding mode is taken from the control register (dynamic rounding).
-In the particular case of :ref:`default floating-point environment <floatenv>`,
-the operation uses rounding to nearest, ties to even.
+Only one value may be specified. If the "fp.round" bundle is absent, and
+the operation depends on rounding mode, the default behavior is to use the value
+from a control register (dynamic rounding). In the specific case of
+:ref:`default floating-point environment <floatenv>`, the register is assumed to
+set rounding to nearest, ties to even.
An operand bundle tagged with "fp.except" may be associated with operations
-that can read or write floating-point exception flags. It contains a single
-metadata string value, which can have one of the following values:
+that can raise floating-point exceptions. It contains a single metadata string
+value, which can have one of the following:
::
"ignore"
"strict"
- "maytrap"
-
-It has the same meaning as the corresponding argument in
-:ref:`constrained intrinsics <constrainedfp>`.
+If the argument is "ignore", floating-point exceptions raised by the call are not
+intended to be observed. Optimization transformations may reorder such operations
+or omit them in some cases. For example, if the call result is unused, the call
+may be removed, even if it could raise exceptions. This is the only permitted value
+in the :ref:`default floating-point environment <floatenv>`.
+
+If the argument of "fp.except" is "strict", all transformations must preserve the
+floating-point exception semantics of the original code. Any exception that would
+have been raised by the original code must also be raised by the transformed code
+unless it can be proven unobservable. No new observable floating-point exceptions
+may be introduced. This value may only be used only in functions with
+``strictfp`` attribute.
.. _moduleasm:
diff --git a/llvm/include/llvm/IR/FloatingPointOps.def b/llvm/include/llvm/IR/FloatingPointOps.def
index 8567b5dbac302..465be751a0087 100644
--- a/llvm/include/llvm/IR/FloatingPointOps.def
+++ b/llvm/include/llvm/IR/FloatingPointOps.def
@@ -10,15 +10,21 @@
//
//===----------------------------------------------------------------------===//
+// Describes floating-point operation.
+// Arguments of the entries are:
+// N - intrinsic function name,
+// D - DAG node corresponding to the intrinsic.
#ifndef FUNCTION
#define FUNCTION(N,D)
#endif
-// Arguments of the entries are:
-// - intrinsic function name,
-// - DAG node corresponding to the intrinsic.
+// Describes floating-point operation, which lowers to STRICT_* nodes in DAG.
+#ifndef LEGACY_DAG
+#define LEGACY_DAG(N,D) FUNCTION(N,D)
+#endif
-FUNCTION(nearbyint, FNEARBYINT)
-FUNCTION(trunc, FTRUNC)
+LEGACY_DAG(nearbyint, FNEARBYINT)
+LEGACY_DAG(trunc, FTRUNC)
#undef FUNCTION
+#undef LEGACY_DAG
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index ab9f5b2c82885..ef386930cab53 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -153,8 +153,8 @@ class IRBuilderBase {
FastMathFlags FMF;
bool IsFPConstrained = false;
- fp::ExceptionBehavior DefaultConstrainedExcept = fp::ebStrict;
- RoundingMode DefaultConstrainedRounding = RoundingMode::Dynamic;
+ fp::ExceptionBehavior DefaultConstrainedExcept = fp::ebIgnore;
+ RoundingMode DefaultConstrainedRounding = RoundingMode::NearestTiesToEven;
ArrayRef<OperandBundleDef> DefaultOperandBundles;
@@ -348,7 +348,18 @@ class IRBuilderBase {
/// enabled the CreateF<op>() calls instead create constrained
/// floating point intrinsic calls. Fast math flags are unaffected
/// by this setting.
- void setIsFPConstrained(bool IsCon) { IsFPConstrained = IsCon; }
+ void setIsFPConstrained(bool IsCon, bool AndReset = true) {
+ if (AndReset) {
+ if (IsCon) {
+ setDefaultConstrainedRounding(RoundingMode::Dynamic);
+ setDefaultConstrainedExcept(fp::ebStrict);
+ } else {
+ setDefaultConstrainedRounding(RoundingMode::NearestTiesToEven);
+ setDefaultConstrainedExcept(fp::ebIgnore);
+ }
+ }
+ IsFPConstrained = IsCon;
+ }
/// Query for the use of constrained floating point math
bool getIsFPConstrained() { return IsFPConstrained; }
@@ -2766,23 +2777,21 @@ class IRBuilderBase {
LLVM_ABI CallInst *CreateDereferenceableAssumption(Value *PtrValue,
Value *SizeValue);
- /// Create an operand bundle in the provided bundle set to represent given FP
- /// rounding mode.
+ /// Create an operand bundle in the provided bundle set to represent the given
+ /// floating-point rounding mode.
///
/// If the rounding mode is not defined, adds the default rounding mode,
/// stored in this builder object.
- void
- createFPRoundingBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<RoundingMode> Rounding = std::nullopt);
+ void createRoundingBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode RM);
- /// Create an operand bundle in the provided bundle set to represent FP
- /// exception behavior.
+ /// Create an operand bundle in the provided bundle set to represent the given
+ /// floating-point exception behavior.
///
/// If the exception behavior is not defined, adds the default behavior,
/// stored in this builder object.
- void createFPExceptionBundle(
- SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<fp::ExceptionBehavior> Except = std::nullopt);
+ void createExceptionBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except);
};
/// This provides a uniform API for creating instructions and inserting
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index 97ef5fcd35769..a0d4176bc646b 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -1094,12 +1094,17 @@ template <typename InputTy> class OperandBundleDefT {
using OperandBundleDef = OperandBundleDefT<Value *>;
using ConstOperandBundleDef = OperandBundleDefT<const Value *>;
-void addFPRoundingBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- RoundingMode Rounding);
-void addFPExceptionBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- fp::ExceptionBehavior Except);
+/// Add a bundle with tag "fp.round" and the specified rounding to the given
+/// bundle set.
+void addRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding);
+
+/// Add a bundle with tag "fp.except" and the specified exception behavior to
+/// the given bundle set.
+void addExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except);
//===----------------------------------------------------------------------===//
// CallBase Class
@@ -1160,6 +1165,7 @@ class CallBase : public Instruction {
/// number of extra operands.
LLVM_ABI unsigned getNumSubclassExtraOperandsDynamic() const;
+ /// Get memory effects specific to floating-point operations.
MemoryEffects getFloatingPointMemoryEffects() const;
public:
@@ -2172,10 +2178,10 @@ class CallBase : public Instruction {
return false;
}
- /// Return rounding mode specified for this call.
+ /// Return the effective rounding mode for this call.
RoundingMode getRoundingMode() const;
- /// Return exception behavior specified for this call.
+ /// Return the effective exception behavior for this call.
fp::ExceptionBehavior getExceptionBehavior() const;
/// Used to keep track of an operand bundle. See the main comment on
diff --git a/llvm/include/llvm/IR/LLVMContext.h b/llvm/include/llvm/IR/LLVMContext.h
index 20a2807fe9613..7ef8fddeb27bc 100644
--- a/llvm/include/llvm/IR/LLVMContext.h
+++ b/llvm/include/llvm/IR/LLVMContext.h
@@ -99,7 +99,7 @@ class LLVMContext {
OB_convergencectrl = 9, // "convergencectrl"
OB_align = 10, // "align"
OB_deactivation_symbol = 11, // "deactivation-symbol"
- OB_fp_control = 12, // "fp.control"
+ OB_fp_round = 12, // "fp.control"
OB_fp_except = 13, // "fp.except"
OB_LastBundleID = OB_fp_except
};
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 392e53b99c64e..2d2363b87d55a 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -6923,9 +6923,7 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
case Intrinsic::exp10:
case Intrinsic::floor:
case Intrinsic::ceil:
- case Intrinsic::trunc:
case Intrinsic::rint:
- case Intrinsic::nearbyint:
case Intrinsic::round:
case Intrinsic::roundeven:
case Intrinsic::canonicalize: {
@@ -6947,9 +6945,7 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
case Intrinsic::exp10: Opcode = ISD::FEXP10; break;
case Intrinsic::floor: Opcode = ISD::FFLOOR; break;
case Intrinsic::ceil: Opcode = ISD::FCEIL; break;
- case Intrinsic::trunc: Opcode = ISD::FTRUNC; break;
case Intrinsic::rint: Opcode = ISD::FRINT; break;
- case Intrinsic::nearbyint: Opcode = ISD::FNEARBYINT; break;
case Intrinsic::round: Opcode = ISD::FROUND; break;
case Intrinsic::roundeven: Opcode = ISD::FROUNDEVEN; break;
case Intrinsic::canonicalize: Opcode = ISD::FCANONICALIZE; break;
@@ -7084,6 +7080,11 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
#include "llvm/IR/VPIntrinsics.def"
visitVectorPredicationIntrinsic(cast<VPIntrinsic>(I));
return;
+#define FUNCTION(NAME, DAGN) \
+ case Intrinsic::NAME: \
+ visitFPOperation(I, ISD::DAGN); \
+ break;
+#include "llvm/IR/FloatingPointOps.def"
case Intrinsic::fptrunc_round: {
// Get the last argument, the metadata and convert it to an integer in the
// call
@@ -8501,6 +8502,33 @@ void SelectionDAGBuilder::pushFPOpOutChain(SDValue Result,
}
}
+void SelectionDAGBuilder::pushOutChain(SDValue Result,
+ fp::ExceptionBehavior EB) {
+ assert(Result.getNode()->getNumValues() == 2);
+
+ // Push node to the appropriate list so that future instructions can be
+ // chained up correctly.
+ SDValue OutChain = Result.getValue(1);
+ switch (EB) {
+ case fp::ExceptionBehavior::ebIgnore:
+ // The only reason why ebIgnore nodes still need to be chained is that
+ // they might depend on the current rounding mode, and therefore must
+ // not be moved across instruction that may change that mode.
+ [[fallthrough]];
+ case fp::ExceptionBehavior::ebMayTrap:
+ // These must not be moved across calls or instructions that may change
+ // floating-point exception masks.
+ PendingConstrainedFP.push_back(OutChain);
+ break;
+ case fp::ExceptionBehavior::ebStrict:
+ // These must not be moved across calls or instructions that may change
+ // floating-point exception masks or read floating-point exception flags.
+ // In addition, they cannot be optimized out even if unused.
+ PendingConstrainedFPStrict.push_back(OutChain);
+ break;
+ }
+}
+
void SelectionDAGBuilder::visitConstrainedFPIntrinsic(
const ConstrainedFPIntrinsic &FPI) {
SDLoc sdl = getCurSDLoc();
@@ -9604,6 +9632,56 @@ bool SelectionDAGBuilder::visitBinaryFloatCall(const CallInst &I,
return true;
}
+bool SelectionDAGBuilder::visitFPOperation(const CallInst &I, unsigned Opcode) {
+ // We already checked this call's prototype; verify it doesn't modify errno.
+ MemoryEffects ME = I.getMemoryEffects();
+ if (!ME.onlyAccessesInaccessibleMem())
+ return false;
+
+ SmallVector<SDValue, 4> Operands;
+ bool HasChain = ME.doesAccessInaccessibleMem();
+ if (HasChain)
+ Operands.push_back(getRoot());
+ for (auto &Arg : I.args())
+ Operands.push_back(getValue(Arg));
+
+ const TargetLowering &TLI = DAG.getTargetLoweringInfo();
+ EVT VT = TLI.getValueType(DAG.getDataLayout(), I.getType(), true);
+ SDVTList NodeVT;
+ if (HasChain)
+ NodeVT = DAG.getVTList(VT, MVT::Other);
+ else
+ NodeVT = DAG.getVTList(VT);
+
+ SDNodeFlags Flags;
+ fp::ExceptionBehavior EB = I.getExceptionBehavior();
+ if (EB == fp::ExceptionBehavior::ebIgnore)
+ Flags.setNoFPExcept(true);
+ if (auto *FPOp = dyn_cast<FPMathOperator>(&I))
+ Flags.copyFMF(*FPOp);
+
+ // Temporary solution: use STRICT_* nodes.
+ if (HasChain)
+ switch (Opcode) {
+ default:
+ break;
+#define LEGACY_DAG(NAME, DAGN) \
+ case ISD::DAGN: \
+ Opcode = ISD::STRICT_##DAGN; \
+ break;
+#include "llvm/IR/FloatingPointOps.def"
+ }
+
+ SDLoc sdl = getCurSDLoc();
+ SDValue Result = DAG.getNode(Opcode, sdl, NodeVT, Operands, Flags);
+ if (HasChain)
+ pushOutChain(Result, EB);
+
+ SDValue FPResult = Result.getValue(0);
+ setValue(&I, FPResult);
+ return true;
+}
+
void SelectionDAGBuilder::visitCall(const CallInst &I) {
// Handle inline assembly differently.
if (I.isInlineAsm()) {
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
index a81a240b057d5..61347438fa719 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
@@ -380,6 +380,8 @@ class SelectionDAGBuilder {
/// Evict any dangling debug information, attempting to salvage it first.
void resolveOrClearDbgInfo();
+ void pushOutChain(SDValue Result, fp::ExceptionBehavior EB);
+
SDValue getValue(const Value *V);
SDValue getNonRegisterValue(const Value *V);
@@ -637,6 +639,7 @@ class SelectionDAGBuilder {
bool visitStrstrCall(const CallInst &I);
bool visitUnaryFloatCall(const CallInst &I, unsigned Opcode);
bool visitBinaryFloatCall(const CallInst &I, unsigned Opcode);
+ bool visitFPOperation(const CallInst &I, unsigned Opcode);
void visitAtomicLoad(const LoadInst &I);
void visitAtomicStore(const StoreInst &I);
void visitLoadFromSwiftError(const LoadInst &I);
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index dd99b14d45d6e..db584029178e7 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -4370,6 +4370,31 @@ void AssemblyWriter::printInfoComment(const Value &V, bool isMaterializable) {
if (PrintInstAddrs)
Out << " ; " << &V;
+
+ if (auto *CI = dyn_cast<CallInst>(&V))
+ if (Intrinsic::ID IID = CI->getIntrinsicID())
+ if (IntrinsicInst::isFloatingPointOperation(IID))
+ if (const BasicBlock *BB = CI->getParent())
+ if (const Function *F = BB->getParent())
+ if (F->hasFnAttribute(Attribute::StrictFP)) {
+ MemoryEffects ME = CI->getMemoryEffects();
+ ModRefInfo MR = ME.getModRef(IRMemLocation::InaccessibleMem);
+ Out << " ; fpe=[";
+ switch (MR) {
+ case ModRefInfo::NoModRef:
+ break;
+ case ModRefInfo::Ref:
+ Out << "r";
+ break;
+ case ModRefInfo::Mod:
+ Out << "w";
+ break;
+ case ModRefInfo::ModRef:
+ Out << "rw";
+ break;
+ }
+ Out << "]";
+ }
}
static void maybePrintCallAddrSpace(const Value *Operand, const Instruction *I,
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index aa862cc2bef61..dd6668e48cbb7 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -38,12 +38,12 @@ llvm::convertStrToRoundingMode(StringRef RoundingArg) {
std::optional<RoundingMode>
llvm::convertBundleToRoundingMode(StringRef RoundingArg) {
return StringSwitch<std::optional<RoundingMode>>(RoundingArg)
- .Case("dyn", RoundingMode::Dynamic)
- .Case("rte", RoundingMode::NearestTiesToEven)
- .Case("rmm", RoundingMode::NearestTiesToAway)
- .Case("rtn", RoundingMode::TowardNegative)
- .Case("rtp", RoundingMode::TowardPositive)
- .Case("rtz", RoundingMode::TowardZero)
+ .Case("dynamic", RoundingMode::Dynamic)
+ .Case("tonearest", RoundingMode::NearestTiesToEven)
+ .Case("tonearestaway", RoundingMode::NearestTiesToAway)
+ .Case("downward", RoundingMode::TowardNegative)
+ .Case("upward", RoundingMode::TowardPositive)
+ .Case("towardzero", RoundingMode::TowardZero)
.Default(std::nullopt);
}
@@ -80,22 +80,22 @@ llvm::convertRoundingModeToBundle(RoundingMode UseRounding) {
std::optional<StringRef> RoundingStr;
switch (UseRounding) {
case RoundingMode::Dynamic:
- RoundingStr = "dyn";
+ RoundingStr = "dynamic";
break;
case RoundingMode::NearestTiesToEven:
- RoundingStr = "rte";
+ RoundingStr = "tonearest";
break;
case RoundingMode::NearestTiesToAway:
- RoundingStr = "rmm";
+ RoundingStr = "tonearestaway";
break;
case RoundingMode::TowardNegative:
- RoundingStr = "rtn";
+ RoundingStr = "downward";
break;
case RoundingMode::TowardPositive:
- RoundingStr = "rtp";
+ RoundingStr = "upward";
break;
case RoundingMode::TowardZero:
- RoundingStr = "rtz";
+ RoundingStr = "towardzero";
break;
default:
break;
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index a93b652447a17..a4f05b32f16cf 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -190,49 +190,54 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
ArrayRef<Value *> Args,
ArrayRef<OperandBundleDef> OpBundles,
const Twine &Name, MDNode *FPMathTag) {
- assert(std::count_if(OpBundles.begin(), OpBundles.end(),
- [](const OperandBundleDef &Item) {
- return Item.getTag() == "fp.control";
- }) <= 1);
- assert(std::count_if(OpBundles.begin(), OpBundles.end(),
- [](const OperandBundleDef &Item) {
- return Item.getTag() == "fp.except";
- }) <= 1);
-
+ // Operand bundles can be extended by floating-point bundles. In this case
+ // we need a copy of OpBundles, because ArrayRef is immutable.
ArrayRef<OperandBundleDef> ActualBundlesRef = OpBundles;
SmallVector<OperandBundleDef, 2> ActualBundles;
- // If the builder is in strictfp mode and has non-default options (like
- // non-dynamic rounding), add corresponding operand bundle. If such bundle is
- // already present, assume it overwrites defaults.
bool NeedUpdateMemoryEffects = false;
- if (IsFPConstrained) {
- if (const auto *Func = dyn_cast<Function>(Callee)) {
- if (Intrinsic::ID ID = Func->getIntrinsicID()) {
- if (IntrinsicInst::isFloatingPointOperation(ID)) {
+ if (const auto *Func = dyn_cast<Function>(Callee))
+ if (Intrinsic::ID ID = Func->getIntrinsicID())
+ if (IntrinsicInst::isFloatingPointOperation(ID)) {
+ // If the builder has non-default floating-point options, add
+ // corresponding operand bundle unless a bundle with such tag is already
+ // present.
+ bool NeedRounding;
+ bool NeedExceptions;
+ if (IsFPConstrained) {
+ NeedRounding = DefaultConstrainedRounding != RoundingMode::Dynamic;
+ NeedExceptions = DefaultConstrainedExcept != fp::ebStrict;
+ } else {
+ NeedRounding =
+ DefaultConstrainedRounding != RoundingMode::NearestTiesToEven;
+ NeedExceptions = false;
+ assert(DefaultConstrainedExcept == fp::ebIgnore &&
+ "FP exception in default mode must be ignored");
+ }
+ if (NeedRounding || NeedExceptions) {
+ for (const auto &Bundle : OpBundles) {
+ if (NeedRounding && Bundle.getTag() == "fp.round")
+ NeedRounding = false;
+ if (NeedExceptions && Bundle.getTag() == "fp.except")
+ NeedExceptions = false;
+ }
+ if (NeedRounding || NeedExceptions) {
+ ActualBundles.append(OpBundles.begin(), OpBundles.end());
+ if (NeedRounding)
+ createRoundingBundle(ActualBundles, DefaultConstrainedRounding);
+ if (NeedExceptions)
+ createExceptionBundle(ActualBundles, DefaultConstrainedExcept);
+ ActualBundlesRef = ActualBundles;
+ }
+ }
+ if (IsFPConstrained) {
MemoryEffects FME = Func->getMemoryEffects();
NeedUpdateMemoryEffects = !FME.doesAccessInaccessibleMem();
- bool NeedRound = DefaultConstrainedRounding != RoundingMode::Dynamic;
- bool NeedExcept = DefaultConstrainedExcept != fp::ebStrict;
- for (const auto &Item : OpBundles) {
- if (NeedRound && Item.getTag() == "fp.control")
- NeedRound = false;
- else if (NeedExcept && Item.getTag() == "fp.except")
- NeedExcept = false;
- ActualBundles.push_back(Item);
- }
- if (NeedRound)
- createFPRoundingBundle(ActualBundles, DefaultConstrainedRounding);
- if (NeedExcept)
- createFPExceptionBundle(ActualBundles, DefaultConstrainedExcept);
- ActualBundlesRef = ActualBundles;
}
}
- }
- }
// If the call accesses FPE, update memory effects accordingly.
- CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
+ CallInst *CI = CallInst::Create(FTy, Callee, Args, ActualBundlesRef);
if (NeedUpdateMemoryEffects) {
MemoryEffects ME = MemoryEffects::inaccessibleMemOnly();
auto A = Attribute::getWithMemoryEffects(getContext(), ME);
@@ -1456,18 +1461,14 @@ CallInst *IRBuilderBase::CreateDereferenceableAssumption(Value *PtrValue,
{DereferenceableOpB});
}
-void IRBuilderBase::createFPRoundingBundle(
- SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<RoundingMode> Rounding) {
- addFPRoundingBundle(Context, Bundles,
- Rounding.value_or(DefaultConstrainedRounding));
+void IRBuilderBase::createRoundingBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles, RoundingMode RM) {
+ addRoundingBundle(Context, Bundles, RM);
}
-void IRBuilderBase::createFPExceptionBundle(
- SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<fp::ExceptionBehavior> Except) {
- addFPExceptionBundle(Context, Bundles,
- Except.value_or(DefaultConstrainedExcept));
+void IRBuilderBase::createExceptionBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles, fp::ExceptionBehavior Except) {
+ addExceptionBundle(Context, Bundles, Except);
}
IRBuilderDefaultInserter::~IRBuilderDefaultInserter() = default;
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index a7b463fd496b6..c8713821333d5 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -616,7 +616,7 @@ bool CallBase::hasReadingOperandBundles() const {
return hasOperandBundlesOtherThan(
{LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
LLVMContext::OB_convergencectrl,
- LLVMContext::OB_deactivation_symbol, LLVMContext::OB_fp_control,
+ LLVMContext::OB_deactivation_symbol, LLVMContext::OB_fp_round,
LLVMContext::OB_fp_except}) &&
getIntrinsicID() != Intrinsic::assume;
}
@@ -626,25 +626,23 @@ bool CallBase::hasClobberingOperandBundles() const {
{LLVMContext::OB_deopt, LLVMContext::OB_funclet,
LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
LLVMContext::OB_convergencectrl,
- LLVMContext::OB_deactivation_symbol, LLVMContext::OB_fp_control,
+ LLVMContext::OB_deactivation_symbol, LLVMContext::OB_fp_round,
LLVMContext::OB_fp_except}) &&
getIntrinsicID() != Intrinsic::assume;
}
RoundingMode CallBase::getRoundingMode() const {
- // Try reading rounding mode from FP bundle.
- std::optional<RoundingMode> RM;
- if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_control)) {
+ // Try reading rounding mode from operand bundle.
+ if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_round)) {
Value *V = RoundingBundle->Inputs.front();
Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
- RM = convertBundleToRoundingMode(cast<MDString>(MD)->getString());
+ if (auto RM = convertBundleToRoundingMode(cast<MDString>(MD)->getString()))
+ return *RM;
}
- if (RM)
- return *RM;
- // No FP bundle, try to guess from the current mode.
- if (getParent())
- if (auto *F = getFunction(); F)
+ // No FP bundle, try to guess from attributes of the current function.
+ if (const BasicBlock *BB = getParent())
+ if (const Function *F = BB->getParent())
return F->getAttributes().hasFnAttr(Attribute::StrictFP)
? RoundingMode::Dynamic
: RoundingMode::NearestTiesToEven;
@@ -655,18 +653,17 @@ RoundingMode CallBase::getRoundingMode() const {
fp::ExceptionBehavior CallBase::getExceptionBehavior() const {
// Try determining exception behavior from FP bundle.
- std::optional<fp::ExceptionBehavior> EB;
if (auto ExceptionBundle = getOperandBundle(LLVMContext::OB_fp_except)) {
Value *V = ExceptionBundle->Inputs.front();
Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
- EB = convertBundleToExceptionBehavior(cast<MDString>(MD)->getString());
+ if (auto EB =
+ convertBundleToExceptionBehavior(cast<MDString>(MD)->getString()))
+ return *EB;
}
- if (EB)
- return *EB;
- // No FP bundle, try to guess from the current mode.
- if (getParent())
- if (auto *F = getFunction(); F)
+ // No FP bundle, try to guess from attributes of the current function.
+ if (const BasicBlock *BB = getParent())
+ if (const Function *F = BB->getParent())
return F->getAttributes().hasFnAttr(Attribute::StrictFP) ? fp::ebStrict
: fp::ebIgnore;
@@ -678,10 +675,12 @@ MemoryEffects CallBase::getFloatingPointMemoryEffects() const {
if (Intrinsic::ID IntrID = getIntrinsicID())
if (const BasicBlock *BB = getParent())
if (const Function *F = BB->getParent())
- if (F->hasFnAttribute(Attribute::StrictFP))
- if (IntrinsicInst::isFloatingPointOperation(IntrID)) {
- return MemoryEffects::inaccessibleMemOnly();
- }
+ if (F->hasFnAttribute(Attribute::StrictFP) &&
+ IntrinsicInst::isFloatingPointOperation(IntrID)) {
+ // Floating-point operations in strictfp function always have side
+ // effect at least because they can raise exceptions.
+ return MemoryEffects::inaccessibleMemOnly();
+ }
return MemoryEffects::none();
}
@@ -798,19 +797,29 @@ bool CallBase::hasArgumentWithAdditionalReturnCaptureComponents() const {
return false;
}
-void llvm::addFPRoundingBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- RoundingMode Rounding) {
+void llvm::addRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding) {
+ assert(std::find_if(Bundles.begin(), Bundles.end(),
+ [](const OperandBundleDef &B) {
+ return B.getTag() == "fp.round";
+ }) == Bundles.end() &&
+ "Bundle 'fp.round' already exists");
std::optional<StringRef> RndStr = convertRoundingModeToBundle(Rounding);
assert(RndStr && "Garbage rounding mode!");
auto *RoundingMDS = MDString::get(Ctx, *RndStr);
auto *RM = MetadataAsValue::get(Ctx, RoundingMDS);
- Bundles.emplace_back("fp.control", RM);
+ Bundles.emplace_back("fp.round", RM);
}
-void llvm::addFPExceptionBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- fp::ExceptionBehavior Except) {
+void llvm::addExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except) {
+ assert(std::find_if(Bundles.begin(), Bundles.end(),
+ [](const OperandBundleDef &B) {
+ return B.getTag() == "fp.except";
+ }) == Bundles.end() &&
+ "Bundle 'fp.except' already exists");
std::optional<StringRef> ExcStr = convertExceptionBehaviorToBundle(Except);
assert(ExcStr && "Garbage exception behavior!");
auto *ExceptMDS = MDString::get(Ctx, *ExcStr);
diff --git a/llvm/lib/IR/LLVMContext.cpp b/llvm/lib/IR/LLVMContext.cpp
index e074c81c44eb9..b061643f5ebd5 100644
--- a/llvm/lib/IR/LLVMContext.cpp
+++ b/llvm/lib/IR/LLVMContext.cpp
@@ -57,8 +57,8 @@ static StringRef knownBundleName(unsigned BundleTagID) {
return "align";
case LLVMContext::OB_deactivation_symbol:
return "deactivation-symbol";
- case LLVMContext::OB_fp_control:
- return "fp.control";
+ case LLVMContext::OB_fp_round:
+ return "fp.round";
case LLVMContext::OB_fp_except:
return "fp.except";
default:
@@ -90,9 +90,9 @@ LLVMContext::LLVMContext() : pImpl(new LLVMContextImpl(*this)) {
assert(Entry->second == BundleTagID && "operand bundle id drifted!");
}
- auto *RoundingEntry = pImpl->getOrInsertBundleTag("fp.control");
- assert(RoundingEntry->second == LLVMContext::OB_fp_control &&
- "fp.control operand bundle id drifted!");
+ auto *RoundingEntry = pImpl->getOrInsertBundleTag("fp.round");
+ assert(RoundingEntry->second == LLVMContext::OB_fp_round &&
+ "fp.round operand bundle id drifted!");
(void)RoundingEntry;
auto *ExceptionEntry = pImpl->getOrInsertBundleTag("fp.except");
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 088cfabe3339e..e9201936b2310 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -4055,7 +4055,7 @@ void Verifier::visitCallBase(CallBase &Call) {
FoundGCTransitionBundle = false, FoundCFGuardTargetBundle = false,
FoundPreallocatedBundle = false, FoundGCLiveBundle = false,
FoundPtrauthBundle = false, FoundKCFIBundle = false,
- FoundAttachedCallBundle = false, FoundFpeControlBundle = false,
+ FoundAttachedCallBundle = false, FoundFpeRoundBundle = false,
FoundFpeExceptBundle = false;
for (unsigned i = 0, e = Call.getNumOperandBundles(); i < e; ++i) {
OperandBundleUse BU = Call.getOperandBundleAt(i);
@@ -4119,30 +4119,29 @@ void Verifier::visitCallBase(CallBase &Call) {
"Multiple \"clang.arc.attachedcall\" operand bundles", Call);
FoundAttachedCallBundle = true;
verifyAttachedCallBundle(Call, BU);
- } else if (Tag == LLVMContext::OB_fp_control) {
- Check(!FoundFpeControlBundle, "Multiple \"fp.control\" operand bundles",
+ } else if (Tag == LLVMContext::OB_fp_round) {
+ Check(!FoundFpeRoundBundle, "Multiple \"fp.round\" operand bundles",
Call);
bool FoundRoundingMode = false;
for (auto &U : BU.Inputs) {
Value *V = U.get();
Check(isa<MetadataAsValue>(V),
- "Value of a \"fp.control\" bundle operand must be a metadata",
+ "Value of a \"fp.round\" bundle operand must be a metadata",
Call);
Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
Check(isa<MDString>(MD),
- "Value of a \"fp.control\" bundle operand must be a string",
- Call);
+ "Value of a \"fp.round\" bundle operand must be a string", Call);
StringRef Item = cast<MDString>(MD)->getString();
if (convertBundleToRoundingMode(Item)) {
Check(!FoundRoundingMode, "Rounding mode is specified more that once",
Call);
FoundRoundingMode = true;
} else {
- CheckFailed("Unrecognized value in \"fp.control\" bundle operand",
+ CheckFailed("Unrecognized value in \"fp.round\" bundle operand",
Call);
}
}
- FoundFpeControlBundle = true;
+ FoundFpeRoundBundle = true;
} else if (Tag == LLVMContext::OB_fp_except) {
Check(!FoundFpeExceptBundle, "Multiple \"fp.except\" operand bundles",
Call);
@@ -4160,6 +4159,15 @@ void Verifier::visitCallBase(CallBase &Call) {
"Value of a \"fp.except\" bundle operand is not a correct exception "
"behavior",
Call);
+ if (EB && *EB != fp::ebIgnore)
+ if (const BasicBlock *BB = Call.getParent())
+ if (const Function *F = BB->getParent()) {
+ bool StrictFP = F->hasFnAttribute(Attribute::StrictFP);
+ Check(StrictFP,
+ "Value of a \"fp.except\" bundle operand in "
+ "default mode must be \"ignore\"",
+ Call);
+ }
FoundFpeExceptBundle = true;
}
}
diff --git a/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll b/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll
index 13f890ae6e191..35c066afb9c13 100644
--- a/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll
+++ b/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll
@@ -250,6 +250,46 @@ define float @ftrunc32(float %f) #0 {
ret float %res
}
+define float @ftrunc32ob(float %f) #0 {
+; SSE41-X86-LABEL: ftrunc32ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; SSE41-X86-NEXT: roundss $11, %xmm0, %xmm0
+; SSE41-X86-NEXT: movss %xmm0, (%esp)
+; SSE41-X86-NEXT: flds (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: popl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: ftrunc32ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundss $11, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: ftrunc32ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX-X86-NEXT: vroundss $11, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovss %xmm0, (%esp)
+; AVX-X86-NEXT: flds (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: popl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: ftrunc32ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundss $11, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call float @llvm.trunc.f32(float %f) [ "fp.except"(metadata !"strict") ]
+ ret float %res
+}
+
define double @ftruncf64(double %f) #0 {
; SSE41-X86-LABEL: ftruncf64:
; SSE41-X86: # %bb.0:
@@ -303,6 +343,58 @@ define double @ftruncf64(double %f) #0 {
ret double %res
}
+define double @ftruncf64ob(double %f) #0 {
+; SSE41-X86-LABEL: ftruncf64ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: .cfi_offset %ebp, -8
+; SSE41-X86-NEXT: movl %esp, %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_register %ebp
+; SSE41-X86-NEXT: andl $-8, %esp
+; SSE41-X86-NEXT: subl $8, %esp
+; SSE41-X86-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
+; SSE41-X86-NEXT: roundsd $11, %xmm0, %xmm0
+; SSE41-X86-NEXT: movsd %xmm0, (%esp)
+; SSE41-X86-NEXT: fldl (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: movl %ebp, %esp
+; SSE41-X86-NEXT: popl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa %esp, 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: ftruncf64ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundsd $11, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: ftruncf64ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: .cfi_offset %ebp, -8
+; AVX-X86-NEXT: movl %esp, %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_register %ebp
+; AVX-X86-NEXT: andl $-8, %esp
+; AVX-X86-NEXT: subl $8, %esp
+; AVX-X86-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX-X86-NEXT: vroundsd $11, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovsd %xmm0, (%esp)
+; AVX-X86-NEXT: fldl (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: movl %ebp, %esp
+; AVX-X86-NEXT: popl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa %esp, 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: ftruncf64ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundsd $11, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call double @llvm.trunc.f64(double %f) [ "fp.except"(metadata !"strict") ]
+ ret double %res
+}
+
define float @frint32(float %f) #0 {
; SSE41-X86-LABEL: frint32:
; SSE41-X86: # %bb.0:
@@ -441,6 +533,46 @@ define float @fnearbyint32(float %f) #0 {
ret float %res
}
+define float @fnearbyint32ob(float %f) #0 {
+; SSE41-X86-LABEL: fnearbyint32ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; SSE41-X86-NEXT: roundss $12, %xmm0, %xmm0
+; SSE41-X86-NEXT: movss %xmm0, (%esp)
+; SSE41-X86-NEXT: flds (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: popl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: fnearbyint32ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundss $12, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: fnearbyint32ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX-X86-NEXT: vroundss $12, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovss %xmm0, (%esp)
+; AVX-X86-NEXT: flds (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: popl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: fnearbyint32ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundss $12, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call float @llvm.nearbyint.f32(float %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret float %res
+}
+
define double @fnearbyintf64(double %f) #0 {
; SSE41-X86-LABEL: fnearbyintf64:
; SSE41-X86: # %bb.0:
@@ -495,6 +627,58 @@ define double @fnearbyintf64(double %f) #0 {
ret double %res
}
+define double @fnearbyint64ob(double %f) #0 {
+; SSE41-X86-LABEL: fnearbyint64ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: .cfi_offset %ebp, -8
+; SSE41-X86-NEXT: movl %esp, %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_register %ebp
+; SSE41-X86-NEXT: andl $-8, %esp
+; SSE41-X86-NEXT: subl $8, %esp
+; SSE41-X86-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
+; SSE41-X86-NEXT: roundsd $12, %xmm0, %xmm0
+; SSE41-X86-NEXT: movsd %xmm0, (%esp)
+; SSE41-X86-NEXT: fldl (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: movl %ebp, %esp
+; SSE41-X86-NEXT: popl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa %esp, 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: fnearbyint64ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundsd $12, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: fnearbyint64ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: .cfi_offset %ebp, -8
+; AVX-X86-NEXT: movl %esp, %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_register %ebp
+; AVX-X86-NEXT: andl $-8, %esp
+; AVX-X86-NEXT: subl $8, %esp
+; AVX-X86-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX-X86-NEXT: vroundsd $12, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovsd %xmm0, (%esp)
+; AVX-X86-NEXT: fldl (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: movl %ebp, %esp
+; AVX-X86-NEXT: popl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa %esp, 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: fnearbyint64ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundsd $12, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call double @llvm.nearbyint.f64(double %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret double %res
+}
+
define float @fround32(float %f) #0 {
; SSE41-X86-LABEL: fround32:
; SSE41-X86: # %bb.0:
diff --git a/llvm/test/CodeGen/X86/vec-strict-512.ll b/llvm/test/CodeGen/X86/vec-strict-512.ll
index 2cafd74af4953..5bd78d99c078b 100644
--- a/llvm/test/CodeGen/X86/vec-strict-512.ll
+++ b/llvm/test/CodeGen/X86/vec-strict-512.ll
@@ -231,6 +231,15 @@ define <16 x float> @strict_vector_ftrunc_v16f32(<16 x float> %f) #0 {
ret <16 x float> %res
}
+define <16 x float> @strict_vector_ftrunc_v16f32ob(<16 x float> %f) #0 {
+; CHECK-LABEL: strict_vector_ftrunc_v16f32ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscaleps $11, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <16 x float> @llvm.trunc.v16f32(<16 x float> %f) [ "fp.except"(metadata !"strict") ]
+ ret <16 x float> %res
+}
+
define <8 x double> @strict_vector_ftrunc_v8f64(<8 x double> %f) #0 {
; CHECK-LABEL: strict_vector_ftrunc_v8f64:
; CHECK: # %bb.0:
@@ -240,6 +249,15 @@ define <8 x double> @strict_vector_ftrunc_v8f64(<8 x double> %f) #0 {
ret <8 x double> %res
}
+define <8 x double> @strict_vector_ftrunc_v8f64ob(<8 x double> %f) #0 {
+; CHECK-LABEL: strict_vector_ftrunc_v8f64ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscalepd $11, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <8 x double> @llvm.trunc.v8f64(<8 x double> %f) [ "fp.except"(metadata !"strict") ]
+ ret <8 x double> %res
+}
+
define <16 x float> @strict_vector_frint_v16f32(<16 x float> %f) #0 {
; CHECK-LABEL: strict_vector_frint_v16f32:
; CHECK: # %bb.0:
@@ -270,6 +288,15 @@ define <16 x float> @strict_vector_fnearbyint_v16f32(<16 x float> %f) #0 {
ret <16 x float> %res
}
+define <16 x float> @strict_vector_fnearbyint_v16f32ob(<16 x float> %f) #0 {
+; CHECK-LABEL: strict_vector_fnearbyint_v16f32ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscaleps $12, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <16 x float> @llvm.nearbyint.v16f32(<16 x float> %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret <16 x float> %res
+}
+
define <8 x double> @strict_vector_fnearbyint_v8f64(<8 x double> %f) #0 {
; CHECK-LABEL: strict_vector_fnearbyint_v8f64:
; CHECK: # %bb.0:
@@ -280,4 +307,13 @@ define <8 x double> @strict_vector_fnearbyint_v8f64(<8 x double> %f) #0 {
ret <8 x double> %res
}
+define <8 x double> @strict_vector_fnearbyint_v8f64ob(<8 x double> %f) #0 {
+; CHECK-LABEL: strict_vector_fnearbyint_v8f64ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscalepd $12, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <8 x double> @llvm.nearbyint.v8f64(<8 x double> %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret <8 x double> %res
+}
+
attributes #0 = { strictfp }
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index 1e1df1b0da96a..f96d519870aaa 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -51,48 +51,48 @@ entry:
ret double %fadd
}
-; Test multiple fp.control bundles.
-; CHECK-NEXT: Multiple "fp.control" operand bundles
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+; Test multiple fp.round bundles.
+; CHECK-NEXT: Multiple "fp.round" operand bundles
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
define double @f6(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
ret double %ftrunc
}
-; Test fp.control bundle that has more than one rounding mode specification.
+; Test fp.round bundle that has more than one rounding mode specification.
; CHECK-NEXT: Rounding mode is specified more that once
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
define double @f7(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
ret double %ftrunc
}
-; Test fp.control bundle that has non-metadata operand.
-; CHECK-NEXT: Value of a "fp.control" bundle operand must be a metadata
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(i32 0) ]
+; Test fp.round bundle that has non-metadata operand.
+; CHECK-NEXT: Value of a "fp.round" bundle operand must be a metadata
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(i32 0) ]
define double @f8(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(i32 0) ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(i32 0) ]
ret double %ftrunc
}
-; Test fp.control bundle that has non-string operand.
-; CHECK-NEXT: Value of a "fp.control" bundle operand must be a string
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata i64 3) ]
+; Test fp.round bundle that has non-string operand.
+; CHECK-NEXT: Value of a "fp.round" bundle operand must be a string
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata i64 3) ]
define double @f9(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !{i64 3}) ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !{i64 3}) ]
ret double %ftrunc
}
-; Test fp.control bundle that specifies incorrect value.
-; CHECK-NEXT: Unrecognized value in "fp.control" bundle operand
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"qqq") ]
+; Test fp.round bundle that specifies incorrect value.
+; CHECK-NEXT: Unrecognized value in "fp.round" bundle operand
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"qqq") ]
define double @f10(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"qqq") ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"qqq") ]
ret double %ftrunc
}
@@ -141,5 +141,14 @@ entry:
ret double %ftrunc
}
+; Test fp.except bundle in default mode cannot be other than "ignore".
+; CHECK-NEXT: Value of a "fp.except" bundle operand in default mode must be "ignore"
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) [ "fp.except"(metadata !"strict") ]
+define double @f16(double %a) {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) [ "fp.except"(metadata !"strict") ]
+ ret double %ftrunc
+}
+
attributes #0 = { strictfp }
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index 84f44eb0a6499..8352a53ac5e3a 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -330,7 +330,9 @@ TEST_F(IRBuilderTest, ConstrainedFP) {
// See if we get constrained intrinsics instead of non-constrained
// instructions.
- Builder.setIsFPConstrained(true);
+ Builder.setDefaultConstrainedRounding(RoundingMode::Dynamic);
+ Builder.setDefaultConstrainedExcept(fp::ebStrict);
+ Builder.setIsFPConstrained(true, false);
auto Parent = BB->getParent();
Parent->addFnAttr(Attribute::StrictFP);
@@ -528,7 +530,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
{
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -538,13 +540,13 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
}
// Check call with FP bundles, rounding is set to default value.
- // nearbyint(%x) [ "fp.control" (metadata !"rte") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"tonearest") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -557,10 +559,10 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
// nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -571,14 +573,15 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
// Check call with FP bundles, both rounding mode and exception behavior are
// set.
- // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"tonearest"),
+ // "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -586,6 +589,40 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
+
+ // If the builder object specifies a rounding mode, the resulting call will
+ // include the corresponding "fp.round" operand bundle.
+ {
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardNegative);
+ SmallVector<OperandBundleDef, 1> Bundles;
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::TowardNegative, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // If the builder object specifies a rounding mode but the operand bundles
+ // already contain an "fp.round" bundle, the builder's specified mode
+ // is ignored.
+ {
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardNegative);
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
}
TEST_F(IRBuilderTest, FPBundlesStrict) {
@@ -607,7 +644,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
{
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -617,13 +654,13 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
}
// Check call with FP bundles, with default (dynamic) rounding mode
- // nearbyint(%x) [ "fp.control" (metadata !"dyn") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"dynamic") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -633,13 +670,13 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
}
// Check call with FP bundles, with specific rounding mode
- // nearbyint(%x) [ "fp.control" (metadata !"rtz") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"towardzero") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
@@ -652,10 +689,10 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
// nearbyint(%x) [ "fp.except" (metadata !"strict") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebStrict);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebStrict);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -668,10 +705,10 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
// nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -681,15 +718,15 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
}
// Check call with both FP bundles.
- // nearbyint(%x) [ "fp.control" (metadata !"rtz"),
+ // nearbyint(%x) [ "fp.round" (metadata !"towardzero"),
// "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -706,7 +743,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
>From db5d67dfa040a5ddbf987c845961757cede8cdd3 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Tue, 21 Oct 2025 23:58:04 +0700
Subject: [PATCH 07/24] Address review comments
- Add IRBuilder::resetModeToStrictFP instead of adding extra argument to
setIsFPConstrained.
- Change wording in the Release Notes.
- Modify printing MemoryEffects of FP operation.
- Fix IRBuilder::CreateCall.
- Clean up unit test.
---
clang/lib/CodeGen/CodeGenFunction.cpp | 2 +-
llvm/docs/ReleaseNotes.md | 2 +
llvm/include/llvm/IR/FPEnv.h | 5 ++
llvm/include/llvm/IR/IRBuilder.h | 22 ++++---
llvm/lib/CodeGen/AtomicExpandPass.cpp | 4 +-
llvm/lib/CodeGen/HardwareLoops.cpp | 6 +-
.../SelectionDAG/SelectionDAGBuilder.cpp | 29 +--------
.../SelectionDAG/SelectionDAGBuilder.h | 2 -
llvm/lib/IR/AsmWriter.cpp | 55 +++++++++-------
llvm/lib/IR/FPEnv.cpp | 33 ++++------
llvm/lib/IR/IRBuilder.cpp | 31 ++++-----
.../Target/AMDGPU/AMDGPUAtomicOptimizer.cpp | 2 +-
llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp | 2 +-
.../Transforms/Utils/LibCallsShrinkWrap.cpp | 2 +-
llvm/lib/Transforms/Utils/LowerAtomic.cpp | 2 +-
llvm/test/Other/print-fp-memeffects.ll | 15 +++++
llvm/unittests/IR/IRBuilderTest.cpp | 63 +++++++++++++++----
17 files changed, 154 insertions(+), 123 deletions(-)
create mode 100644 llvm/test/Other/print-fp-memeffects.ll
diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp
index 2112bb6729714..b14d9d7e8d060 100644
--- a/clang/lib/CodeGen/CodeGenFunction.cpp
+++ b/clang/lib/CodeGen/CodeGenFunction.cpp
@@ -1089,7 +1089,7 @@ void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
if ((FD && (FD->UsesFPIntrin() || FD->hasAttr<StrictFPAttr>())) ||
(!FD && (FPExceptionBehavior != llvm::fp::ebIgnore ||
RM != llvm::RoundingMode::NearestTiesToEven))) {
- Builder.setIsFPConstrained(true, false);
+ Builder.setIsFPConstrained(true);
Fn->addFnAttr(llvm::Attribute::StrictFP);
}
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 90e7ca5e98a5f..b125ca3644cb2 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -63,6 +63,8 @@ Changes to the LLVM IR
intrinsics. These are equivalent to `fptrunc` and `fpext` with half
with a bitcast.
* Floating-point operand bundles have been added.
+* Calls to floating-point intrinsics can have operand bundles "fp.round" and
+ "fp.except", which specify effective rounding mode and exception behavior.
* "denormal-fp-math" and "denormal-fp-math-f32" string attributes were
migrated to first-class denormal_fpenv attribute.
diff --git a/llvm/include/llvm/IR/FPEnv.h b/llvm/include/llvm/IR/FPEnv.h
index 78a5d6d5b75f5..6f95b7c688a9d 100644
--- a/llvm/include/llvm/IR/FPEnv.h
+++ b/llvm/include/llvm/IR/FPEnv.h
@@ -81,6 +81,11 @@ LLVM_ABI std::optional<StringRef>
std::optional<StringRef>
convertExceptionBehaviorToBundle(fp::ExceptionBehavior);
+inline raw_ostream &operator<<(raw_ostream &OS, fp::ExceptionBehavior EB) {
+ OS << convertExceptionBehaviorToBundle(EB).value_or("invalid");
+ return OS;
+}
+
/// Returns true if the exception handling behavior and rounding mode
/// match what is used in the default floating point environment.
inline bool isDefaultFPEnvironment(fp::ExceptionBehavior EB, RoundingMode RM) {
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index ef386930cab53..77e642f07b38c 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -348,17 +348,19 @@ class IRBuilderBase {
/// enabled the CreateF<op>() calls instead create constrained
/// floating point intrinsic calls. Fast math flags are unaffected
/// by this setting.
- void setIsFPConstrained(bool IsCon, bool AndReset = true) {
- if (AndReset) {
- if (IsCon) {
- setDefaultConstrainedRounding(RoundingMode::Dynamic);
- setDefaultConstrainedExcept(fp::ebStrict);
- } else {
- setDefaultConstrainedRounding(RoundingMode::NearestTiesToEven);
- setDefaultConstrainedExcept(fp::ebIgnore);
- }
+ void setIsFPConstrained(bool IsCon) { IsFPConstrained = IsCon; }
+
+ /// Enable/Disable use of constrained floating point math and reset FP options
+ /// according to the selected mode.
+ void resetModeToStrictFP(bool IsCon) {
+ if (IsCon) {
+ setDefaultConstrainedRounding(RoundingMode::Dynamic);
+ setDefaultConstrainedExcept(fp::ebStrict);
+ } else {
+ setDefaultConstrainedRounding(RoundingMode::NearestTiesToEven);
+ setDefaultConstrainedExcept(fp::ebIgnore);
}
- IsFPConstrained = IsCon;
+ setIsFPConstrained(IsCon);
}
/// Query for the use of constrained floating point math
diff --git a/llvm/lib/CodeGen/AtomicExpandPass.cpp b/llvm/lib/CodeGen/AtomicExpandPass.cpp
index 341a4618dbb47..4527dd545a3af 100644
--- a/llvm/lib/CodeGen/AtomicExpandPass.cpp
+++ b/llvm/lib/CodeGen/AtomicExpandPass.cpp
@@ -166,7 +166,7 @@ struct ReplacementIRBuilder
SetInsertPoint(I);
this->CollectMetadataToCopy(I, {LLVMContext::MD_pcsections});
if (BB->getParent()->getAttributes().hasFnAttr(Attribute::StrictFP))
- this->setIsFPConstrained(true);
+ this->resetModeToStrictFP(true);
MMRAMD = I->getMetadata(LLVMContext::MD_mmra);
}
@@ -1746,7 +1746,7 @@ bool AtomicExpandImpl::tryExpandAtomicCmpXchg(AtomicCmpXchgInst *CI) {
bool llvm::expandAtomicRMWToCmpXchg(AtomicRMWInst *AI,
CreateCmpXchgInstFun CreateCmpXchg) {
ReplacementIRBuilder Builder(AI, AI->getDataLayout());
- Builder.setIsFPConstrained(
+ Builder.resetModeToStrictFP(
AI->getFunction()->hasFnAttribute(Attribute::StrictFP));
// FIXME: If FP exceptions are observable, we should force them off for the
diff --git a/llvm/lib/CodeGen/HardwareLoops.cpp b/llvm/lib/CodeGen/HardwareLoops.cpp
index 5c0252614e281..7a7a0c11bf05e 100644
--- a/llvm/lib/CodeGen/HardwareLoops.cpp
+++ b/llvm/lib/CodeGen/HardwareLoops.cpp
@@ -491,7 +491,7 @@ Value *HardwareLoop::InitLoopCount() {
Value* HardwareLoop::InsertIterationSetup(Value *LoopCountInit) {
IRBuilder<> Builder(BeginBB->getTerminator());
if (BeginBB->getParent()->getAttributes().hasFnAttr(Attribute::StrictFP))
- Builder.setIsFPConstrained(true);
+ Builder.resetModeToStrictFP(true);
Type *Ty = LoopCountInit->getType();
bool UsePhi = UsePHICounter || Opts.ForcePhi;
Intrinsic::ID ID = UseLoopGuard
@@ -525,7 +525,7 @@ void HardwareLoop::InsertLoopDec() {
IRBuilder<> CondBuilder(ExitBranch);
if (ExitBranch->getParent()->getParent()->getAttributes().hasFnAttr(
Attribute::StrictFP))
- CondBuilder.setIsFPConstrained(true);
+ CondBuilder.resetModeToStrictFP(true);
Value *Ops[] = { LoopDecrement };
Value *NewCond = CondBuilder.CreateIntrinsic(Intrinsic::loop_decrement,
@@ -548,7 +548,7 @@ Instruction* HardwareLoop::InsertLoopRegDec(Value *EltsRem) {
IRBuilder<> CondBuilder(ExitBranch);
if (ExitBranch->getParent()->getParent()->getAttributes().hasFnAttr(
Attribute::StrictFP))
- CondBuilder.setIsFPConstrained(true);
+ CondBuilder.resetModeToStrictFP(true);
Value *Ops[] = { EltsRem, LoopDecrement };
Value *Call = CondBuilder.CreateIntrinsic(Intrinsic::loop_decrement_reg,
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 2d2363b87d55a..eab2058de36bf 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -8502,33 +8502,6 @@ void SelectionDAGBuilder::pushFPOpOutChain(SDValue Result,
}
}
-void SelectionDAGBuilder::pushOutChain(SDValue Result,
- fp::ExceptionBehavior EB) {
- assert(Result.getNode()->getNumValues() == 2);
-
- // Push node to the appropriate list so that future instructions can be
- // chained up correctly.
- SDValue OutChain = Result.getValue(1);
- switch (EB) {
- case fp::ExceptionBehavior::ebIgnore:
- // The only reason why ebIgnore nodes still need to be chained is that
- // they might depend on the current rounding mode, and therefore must
- // not be moved across instruction that may change that mode.
- [[fallthrough]];
- case fp::ExceptionBehavior::ebMayTrap:
- // These must not be moved across calls or instructions that may change
- // floating-point exception masks.
- PendingConstrainedFP.push_back(OutChain);
- break;
- case fp::ExceptionBehavior::ebStrict:
- // These must not be moved across calls or instructions that may change
- // floating-point exception masks or read floating-point exception flags.
- // In addition, they cannot be optimized out even if unused.
- PendingConstrainedFPStrict.push_back(OutChain);
- break;
- }
-}
-
void SelectionDAGBuilder::visitConstrainedFPIntrinsic(
const ConstrainedFPIntrinsic &FPI) {
SDLoc sdl = getCurSDLoc();
@@ -9675,7 +9648,7 @@ bool SelectionDAGBuilder::visitFPOperation(const CallInst &I, unsigned Opcode) {
SDLoc sdl = getCurSDLoc();
SDValue Result = DAG.getNode(Opcode, sdl, NodeVT, Operands, Flags);
if (HasChain)
- pushOutChain(Result, EB);
+ pushFPOpOutChain(Result, EB);
SDValue FPResult = Result.getValue(0);
setValue(&I, FPResult);
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
index 61347438fa719..1291137126101 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
@@ -380,8 +380,6 @@ class SelectionDAGBuilder {
/// Evict any dangling debug information, attempting to salvage it first.
void resolveOrClearDbgInfo();
- void pushOutChain(SDValue Result, fp::ExceptionBehavior EB);
-
SDValue getValue(const Value *V);
SDValue getNonRegisterValue(const Value *V);
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index db584029178e7..1b8e78d3297df 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -111,6 +111,10 @@ static cl::opt<bool> PrintAddrspaceName("print-addrspace-name", cl::Hidden,
cl::init(false),
cl::desc("Print address space names"));
+static cl::opt<bool> PrintFPMemoryEffects(
+ "print-fp-memory-effects", cl::Hidden,
+ cl::desc("Pretty print floating-point memory effects when dumping"));
+
// Make virtual table appear in this compilation unit.
AssemblyAnnotationWriter::~AssemblyAnnotationWriter() = default;
@@ -4342,6 +4346,25 @@ void AssemblyWriter::printGCRelocateComment(const GCRelocateInst &Relocate) {
Out << ")";
}
+static void printFPMemoryEffects(raw_ostream &Out, MemoryEffects ME) {
+ ModRefInfo MR = ME.getModRef(IRMemLocation::InaccessibleMem);
+ Out << " ; fpe=[";
+ switch (MR) {
+ case ModRefInfo::NoModRef:
+ break;
+ case ModRefInfo::Ref:
+ Out << "r";
+ break;
+ case ModRefInfo::Mod:
+ Out << "w";
+ break;
+ case ModRefInfo::ModRef:
+ Out << "rw";
+ break;
+ }
+ Out << "]";
+}
+
/// printInfoComment - Print a little comment after the instruction indicating
/// which slot it occupies.
void AssemblyWriter::printInfoComment(const Value &V, bool isMaterializable) {
@@ -4371,30 +4394,14 @@ void AssemblyWriter::printInfoComment(const Value &V, bool isMaterializable) {
if (PrintInstAddrs)
Out << " ; " << &V;
- if (auto *CI = dyn_cast<CallInst>(&V))
- if (Intrinsic::ID IID = CI->getIntrinsicID())
- if (IntrinsicInst::isFloatingPointOperation(IID))
- if (const BasicBlock *BB = CI->getParent())
- if (const Function *F = BB->getParent())
- if (F->hasFnAttribute(Attribute::StrictFP)) {
- MemoryEffects ME = CI->getMemoryEffects();
- ModRefInfo MR = ME.getModRef(IRMemLocation::InaccessibleMem);
- Out << " ; fpe=[";
- switch (MR) {
- case ModRefInfo::NoModRef:
- break;
- case ModRefInfo::Ref:
- Out << "r";
- break;
- case ModRefInfo::Mod:
- Out << "w";
- break;
- case ModRefInfo::ModRef:
- Out << "rw";
- break;
- }
- Out << "]";
- }
+ if (PrintFPMemoryEffects) {
+ if (auto *CI = dyn_cast<CallInst>(&V))
+ if (Intrinsic::ID IID = CI->getIntrinsicID())
+ if (const Function *F = CI->getFunction())
+ if (IntrinsicInst::isFloatingPointOperation(IID) &&
+ F->hasFnAttribute(Attribute::StrictFP))
+ printFPMemoryEffects(Out, CI->getMemoryEffects());
+ }
}
static void maybePrintCallAddrSpace(const Value *Operand, const Instruction *I,
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index dd6668e48cbb7..86f4283efa6d9 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -80,27 +80,20 @@ llvm::convertRoundingModeToBundle(RoundingMode UseRounding) {
std::optional<StringRef> RoundingStr;
switch (UseRounding) {
case RoundingMode::Dynamic:
- RoundingStr = "dynamic";
- break;
+ return "dynamic";
case RoundingMode::NearestTiesToEven:
- RoundingStr = "tonearest";
- break;
+ return "tonearest";
case RoundingMode::NearestTiesToAway:
- RoundingStr = "tonearestaway";
- break;
+ return "tonearestaway";
case RoundingMode::TowardNegative:
- RoundingStr = "downward";
- break;
+ return "downward";
case RoundingMode::TowardPositive:
- RoundingStr = "upward";
- break;
+ return "upward";
case RoundingMode::TowardZero:
- RoundingStr = "towardzero";
- break;
+ return "towardzero";
default:
- break;
+ return std::nullopt;
}
- return RoundingStr;
}
std::optional<fp::ExceptionBehavior>
@@ -143,16 +136,14 @@ llvm::convertExceptionBehaviorToBundle(fp::ExceptionBehavior UseExcept) {
std::optional<StringRef> ExceptStr;
switch (UseExcept) {
case fp::ebStrict:
- ExceptStr = "strict";
- break;
+ return "strict";
case fp::ebIgnore:
- ExceptStr = "ignore";
- break;
+ return "ignore";
case fp::ebMayTrap:
- ExceptStr = "maytrap";
- break;
+ return "maytrap";
+ default:
+ return std::nullopt;
}
- return ExceptStr;
}
Intrinsic::ID llvm::getConstrainedIntrinsicID(const Instruction &Instr) {
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index a4f05b32f16cf..9a0961ac6c37a 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -199,7 +199,7 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
if (const auto *Func = dyn_cast<Function>(Callee))
if (Intrinsic::ID ID = Func->getIntrinsicID())
if (IntrinsicInst::isFloatingPointOperation(ID)) {
- // If the builder has non-default floating-point options, add
+ // If the builder specifies non-default floating-point options, add
// corresponding operand bundle unless a bundle with such tag is already
// present.
bool NeedRounding;
@@ -214,23 +214,24 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
assert(DefaultConstrainedExcept == fp::ebIgnore &&
"FP exception in default mode must be ignored");
}
+ // Options specified by bundles have higher precedence.
+ for (const auto &Bundle : OpBundles) {
+ if (NeedRounding && Bundle.getTag() == "fp.round")
+ NeedRounding = false;
+ if (NeedExceptions && Bundle.getTag() == "fp.except")
+ NeedExceptions = false;
+ }
if (NeedRounding || NeedExceptions) {
- for (const auto &Bundle : OpBundles) {
- if (NeedRounding && Bundle.getTag() == "fp.round")
- NeedRounding = false;
- if (NeedExceptions && Bundle.getTag() == "fp.except")
- NeedExceptions = false;
- }
- if (NeedRounding || NeedExceptions) {
- ActualBundles.append(OpBundles.begin(), OpBundles.end());
- if (NeedRounding)
- createRoundingBundle(ActualBundles, DefaultConstrainedRounding);
- if (NeedExceptions)
- createExceptionBundle(ActualBundles, DefaultConstrainedExcept);
- ActualBundlesRef = ActualBundles;
- }
+ ActualBundles.append(OpBundles.begin(), OpBundles.end());
+ if (NeedRounding)
+ createRoundingBundle(ActualBundles, DefaultConstrainedRounding);
+ if (NeedExceptions)
+ createExceptionBundle(ActualBundles, DefaultConstrainedExcept);
+ ActualBundlesRef = ActualBundles;
}
if (IsFPConstrained) {
+ // Due to potential reading FP exception bits, in strictfp mode the
+ // memory effects must include read/write access to FPE.
MemoryEffects FME = Func->getMemoryEffects();
NeedUpdateMemoryEffects = !FME.doesAccessInaccessibleMem();
}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp b/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
index 784ee36d55c1e..434dda6416123 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
@@ -649,7 +649,7 @@ void AMDGPUAtomicOptimizerImpl::optimizeAtomic(Instruction &I,
IRBuilder<> B(&I);
if (AtomicRMWInst::isFPOperation(Op)) {
- B.setIsFPConstrained(I.getFunction()->hasFnAttribute(Attribute::StrictFP));
+ B.resetModeToStrictFP(I.getFunction()->hasFnAttribute(Attribute::StrictFP));
}
// If we are in a pixel shader, because of how we have to mask out helper
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp b/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
index 4f97e5e117bf4..2cffc5aab1b24 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
@@ -621,7 +621,7 @@ bool AMDGPULibCalls::fold(CallInst *CI) {
IRBuilder<> B(CI);
if (CI->isStrictFP())
- B.setIsFPConstrained(true);
+ B.resetModeToStrictFP(true);
if (FPMathOperator *FPOp = dyn_cast<FPMathOperator>(CI)) {
// Under unsafe-math, evaluate calls if possible.
diff --git a/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp b/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
index fca09c678eba4..8cf57bbc6ad78 100644
--- a/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
+++ b/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
@@ -103,7 +103,7 @@ class LibCallsShrinkWrap : public InstVisitor<LibCallsShrinkWrap> {
if (!Arg->getType()->isFloatTy())
V = ConstantFoldCastInstruction(Instruction::FPExt, V, Arg->getType());
if (BBBuilder.GetInsertBlock()->getParent()->hasFnAttribute(Attribute::StrictFP))
- BBBuilder.setIsFPConstrained(true);
+ BBBuilder.resetModeToStrictFP(true);
return BBBuilder.CreateFCmp(Cmp, Arg, V);
}
diff --git a/llvm/lib/Transforms/Utils/LowerAtomic.cpp b/llvm/lib/Transforms/Utils/LowerAtomic.cpp
index e8b06415d4062..b0c788fa74a5d 100644
--- a/llvm/lib/Transforms/Utils/LowerAtomic.cpp
+++ b/llvm/lib/Transforms/Utils/LowerAtomic.cpp
@@ -124,7 +124,7 @@ Value *llvm::buildAtomicRMWValue(AtomicRMWInst::BinOp Op,
bool llvm::lowerAtomicRMWInst(AtomicRMWInst *RMWI) {
IRBuilder<> Builder(RMWI);
- Builder.setIsFPConstrained(
+ Builder.resetModeToStrictFP(
RMWI->getFunction()->hasFnAttribute(Attribute::StrictFP));
Value *Ptr = RMWI->getPointerOperand();
diff --git a/llvm/test/Other/print-fp-memeffects.ll b/llvm/test/Other/print-fp-memeffects.ll
new file mode 100644
index 0000000000000..937e31456b26b
--- /dev/null
+++ b/llvm/test/Other/print-fp-memeffects.ll
@@ -0,0 +1,15 @@
+; RUN: opt %s -print-fp-memory-effects -S | FileCheck %s --match-full-lines
+
+define float @test_00(float %x) {
+ %res = call float @llvm.nearbyint.f32(float %x) [ "fpe.round"(metadata !"upward"), "fpe.except"(metadata !"ignore") ]
+ ret float %res
+; CHECK-LABEL: define float @test_00({{.*}}
+; CHECK: {{.*}} = call float @llvm.nearbyint.f32({{.*}}]
+}
+
+define float @test_01(float %x) strictfp {
+ %res = call float @llvm.nearbyint.f32(float %x) [ "fpe.round"(metadata !"upward"), "fpe.except"(metadata !"strict") ]
+ ret float %res
+; CHECK-LABEL: define float @test_01({{.*}}
+; CHECK: {{.*}} = call float @llvm.nearbyint.f32({{.*}}] ; fpe=[rw]
+}
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index 8352a53ac5e3a..c3388d64c664b 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -332,7 +332,7 @@ TEST_F(IRBuilderTest, ConstrainedFP) {
// instructions.
Builder.setDefaultConstrainedRounding(RoundingMode::Dynamic);
Builder.setDefaultConstrainedExcept(fp::ebStrict);
- Builder.setIsFPConstrained(true, false);
+ Builder.setIsFPConstrained(true);
auto Parent = BB->getParent();
Parent->addFnAttr(Attribute::StrictFP);
@@ -500,12 +500,12 @@ TEST_F(IRBuilderTest, ConstrainedFPFunctionCall) {
Function::Create(FTy, Function::ExternalLinkage, "", M.get());
BasicBlock *CalleeBB = BasicBlock::Create(Ctx, "", Callee);
IRBuilder<> CalleeBuilder(CalleeBB);
- CalleeBuilder.setIsFPConstrained(true);
+ CalleeBuilder.resetModeToStrictFP(true);
CalleeBuilder.setConstrainedFPFunctionAttr();
CalleeBuilder.CreateRetVoid();
// Now call the empty constrained FP function.
- Builder.setIsFPConstrained(true);
+ Builder.resetModeToStrictFP(true);
Builder.setConstrainedFPFunctionAttr();
CallInst *FCall = Builder.CreateCall(Callee, {});
@@ -629,9 +629,6 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
F->addFnAttr(Attribute::StrictFP);
IRBuilder<> Builder(BB);
- Builder.setDefaultConstrainedExcept(fp::ebStrict);
- Builder.setDefaultConstrainedRounding(RoundingMode::TowardZero);
- Builder.setIsFPConstrained(true);
GlobalVariable *GVDouble = new GlobalVariable(
*M, Type::getDoubleTy(Ctx), true, GlobalValue::ExternalLinkage, nullptr);
@@ -639,6 +636,8 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
Function *Fn = Intrinsic::getOrInsertDeclaration(
M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
+ Builder.resetModeToStrictFP(true);
+
// A floating-point operation has side effects in strictfp environment even
// if it has no FP bundles.
{
@@ -653,7 +652,8 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // Check call with FP bundles, with default (dynamic) rounding mode
+ // A call may have FP bundles, in this case it has a bundle that specifies
+ // default (dynamic) rounding mode.
// nearbyint(%x) [ "fp.round" (metadata !"dynamic") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -669,7 +669,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // Check call with FP bundles, with specific rounding mode
+ // FP bundle may specify a non-default rounding mode.
// nearbyint(%x) [ "fp.round" (metadata !"towardzero") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -685,7 +685,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // Check call with FP bundles, exception behavior is set to default value.
+ // FP bundles may specify exception behavior, here it is the default value.
// nearbyint(%x) [ "fp.except" (metadata !"strict") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -701,7 +701,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // Check call with FP bundles, exception behavior is set to specific value.
+ // FP bundles may specify exception behavior, including non-default values.
// nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -717,19 +717,56 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // Check call with both FP bundles.
+ // FP bundles may specify both rounding mode and exception behavior, here is
+ // the case of default values.
+ // nearbyint(%x) [ "fp.round" (metadata !"dynamic"),
+ // "fp.except" (metadata !"strict") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebStrict);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // FP bundles may specify both rounding mode and exception behavior, here is
+ // the case of non-default values.
// nearbyint(%x) [ "fp.round" (metadata !"towardzero"),
// "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
- EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // FP bundles are added when IRBuilder specifies rounding or exception
+ // behavior, non-default for the selected mode.
+ Builder.setDefaultConstrainedExcept(fp::ebIgnore);
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardZero);
+ Builder.setIsFPConstrained(true);
+ {
+ Value *V = Builder.CreateCall(Fn, {FnArg});
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
>From d0b9f5402de76542a0ccae44e1f56367e4e3df52 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Sun, 2 Nov 2025 14:46:55 +0700
Subject: [PATCH 08/24] Fix warnings
---
llvm/lib/IR/FPEnv.cpp | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index 86f4283efa6d9..927bfc23183c6 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -77,7 +77,6 @@ llvm::convertRoundingModeToStr(RoundingMode UseRounding) {
std::optional<StringRef>
llvm::convertRoundingModeToBundle(RoundingMode UseRounding) {
- std::optional<StringRef> RoundingStr;
switch (UseRounding) {
case RoundingMode::Dynamic:
return "dynamic";
@@ -133,7 +132,6 @@ llvm::convertExceptionBehaviorToStr(fp::ExceptionBehavior UseExcept) {
std::optional<StringRef>
llvm::convertExceptionBehaviorToBundle(fp::ExceptionBehavior UseExcept) {
- std::optional<StringRef> ExceptStr;
switch (UseExcept) {
case fp::ebStrict:
return "strict";
@@ -141,9 +139,8 @@ llvm::convertExceptionBehaviorToBundle(fp::ExceptionBehavior UseExcept) {
return "ignore";
case fp::ebMayTrap:
return "maytrap";
- default:
- return std::nullopt;
}
+ return std::nullopt;
}
Intrinsic::ID llvm::getConstrainedIntrinsicID(const Instruction &Instr) {
>From dc6cec36f336b5f82fe27008b7ce2ea471c6ece0 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Mon, 1 Dec 2025 23:42:47 +0700
Subject: [PATCH 09/24] Put back 'maytrap' and fix some errors
---
llvm/docs/LangRef.rst | 5 +++++
llvm/lib/IR/IRBuilder.cpp | 5 +++--
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index c5869daf2a05c..a1685b70e8cad 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3319,6 +3319,7 @@ value, which can have one of the following:
::
"ignore"
+ "maytrap"
"strict"
If the argument is "ignore", floating-point exceptions raised by the call are not
@@ -3334,6 +3335,10 @@ unless it can be proven unobservable. No new observable floating-point exception
may be introduced. This value may only be used only in functions with
``strictfp`` attribute.
+The value "maytrap" is almost same as "strict", but transformations are not
+required to preserve all exceptions that are implied by the original code. For
+example, exceptions may be potentially hidden by constant folding.
+
.. _moduleasm:
Module-Level Inline Assembly
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 9a0961ac6c37a..c1448f8590fc4 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -230,8 +230,9 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
ActualBundlesRef = ActualBundles;
}
if (IsFPConstrained) {
- // Due to potential reading FP exception bits, in strictfp mode the
- // memory effects must include read/write access to FPE.
+ // Due to potential setting FP exception bits, in modes other than
+ // the default, the memory effects must include read/write access
+ // to FPE.
MemoryEffects FME = Func->getMemoryEffects();
NeedUpdateMemoryEffects = !FME.doesAccessInaccessibleMem();
}
>From 2480fadcbcc81904bc1f95941c18492366e1a3cc Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 4 Dec 2025 13:30:39 +0700
Subject: [PATCH 10/24] Fix copying math flags
---
llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index eab2058de36bf..14ec379cf7c7f 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -9627,11 +9627,11 @@ bool SelectionDAGBuilder::visitFPOperation(const CallInst &I, unsigned Opcode) {
NodeVT = DAG.getVTList(VT);
SDNodeFlags Flags;
+ if (auto *FPOp = dyn_cast<FPMathOperator>(&I))
+ Flags.copyFMF(*FPOp);
fp::ExceptionBehavior EB = I.getExceptionBehavior();
if (EB == fp::ExceptionBehavior::ebIgnore)
Flags.setNoFPExcept(true);
- if (auto *FPOp = dyn_cast<FPMathOperator>(&I))
- Flags.copyFMF(*FPOp);
// Temporary solution: use STRICT_* nodes.
if (HasChain)
>From 2ad22847cb94c0305a085df455f27187dd84b206 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Sun, 7 Dec 2025 22:20:10 +0700
Subject: [PATCH 11/24] When ilniling do not convert calls with bundles
---
llvm/include/llvm/IR/InstrTypes.h | 3 +++
llvm/lib/IR/FPEnv.cpp | 8 ++++++++
llvm/lib/IR/Instructions.cpp | 12 ++++++++++++
3 files changed, 23 insertions(+)
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index a0d4176bc646b..744aebbfab3d9 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -2178,6 +2178,9 @@ class CallBase : public Instruction {
return false;
}
+ /// Returns true, if this call has a floating-point operand bundle.
+ bool hasFloatingPointOperandBundle() const;
+
/// Return the effective rounding mode for this call.
RoundingMode getRoundingMode() const;
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index 927bfc23183c6..d8d9ab1d817da 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -145,6 +145,14 @@ llvm::convertExceptionBehaviorToBundle(fp::ExceptionBehavior UseExcept) {
Intrinsic::ID llvm::getConstrainedIntrinsicID(const Instruction &Instr) {
Intrinsic::ID IID = Intrinsic::not_intrinsic;
+
+ if (auto *CB = dyn_cast<CallBase>(&Instr)) {
+ // If the instruction is a call, do not convert it if it has
+ // floating-point operand bundles.
+ if (CB->hasFloatingPointOperandBundle())
+ return IID;
+ }
+
switch (Instr.getOpcode()) {
case Instruction::FCmp:
// Unlike other instructions FCmp can be mapped to one of two intrinsic
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index c8713821333d5..afec918cecff7 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -631,6 +631,18 @@ bool CallBase::hasClobberingOperandBundles() const {
getIntrinsicID() != Intrinsic::assume;
}
+bool CallBase::hasFloatingPointOperandBundle() const {
+ for (unsigned i = 0, e = getNumOperandBundles(); i != e; ++i) {
+ OperandBundleUse U = getOperandBundleAt(i);
+ switch (U.getTagID()) {
+ case LLVMContext::OB_fp_round:
+ case LLVMContext::OB_fp_except:
+ return true;
+ }
+ }
+ return false;
+}
+
RoundingMode CallBase::getRoundingMode() const {
// Try reading rounding mode from operand bundle.
if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_round)) {
>From 7a24b8d44d5cb93a1f9ccf555aa615331972196d Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Mon, 15 Dec 2025 18:59:09 +0700
Subject: [PATCH 12/24] Inlining instructions with FP bundles
---
llvm/lib/Transforms/Utils/CloneFunction.cpp | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/llvm/lib/Transforms/Utils/CloneFunction.cpp b/llvm/lib/Transforms/Utils/CloneFunction.cpp
index 192d313b23798..26b1b01855ac1 100644
--- a/llvm/lib/Transforms/Utils/CloneFunction.cpp
+++ b/llvm/lib/Transforms/Utils/CloneFunction.cpp
@@ -520,6 +520,24 @@ PruningFunctionCloner::cloneInstruction(BasicBlock::const_iterator II) {
MetadataAsValue::get(Ctx, MDString::get(Ctx, "fpexcept.ignore")));
NewInst = CallInst::Create(IFn, Args, OldInst.getName() + ".strict");
+ } else if (auto *ICall = dyn_cast<IntrinsicInst>(&OldInst)) {
+ Intrinsic::ID IID = ICall->getIntrinsicID();
+ if (IntrinsicInst::isFloatingPointOperation(IID)) {
+ // The function call needs to be cloned, but the bundles attached to it
+ // may require update.
+ if (const BasicBlock *BB = OldInst.getParent())
+ if (const Function *F = BB->getParent())
+ // Instructions taken from default mode functions acquire "ignore"
+ // exception handling attribute.
+ if (!F->getAttributes().hasFnAttr(Attribute::StrictFP) &&
+ !ICall->getOperandBundle(LLVMContext::OB_fp_except)) {
+ SmallVector<OperandBundleDef, 1> NewBundles;
+ ICall->getOperandBundlesAsDefs(NewBundles);
+ addExceptionBundle(F->getContext(), NewBundles, fp::ebIgnore);
+ NewInst = CallInst::Create(const_cast<IntrinsicInst *>(ICall),
+ NewBundles);
+ }
+ }
}
}
if (!NewInst)
>From 28c79eb6b0f9283effe008a7f0571756889c2629 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Wed, 10 Dec 2025 00:00:31 +0700
Subject: [PATCH 13/24] Do not set fp.round for functions that do not depend on
rounding mode
---
llvm/include/llvm/IR/FloatingPointOps.def | 9 ++--
llvm/include/llvm/IR/IntrinsicInst.h | 4 ++
.../SelectionDAG/SelectionDAGBuilder.cpp | 4 +-
llvm/lib/IR/IRBuilder.cpp | 52 +++++++++++++------
llvm/lib/IR/IntrinsicInst.cpp | 13 ++++-
llvm/lib/IR/Verifier.cpp | 7 ++-
llvm/test/Verifier/fp-intrinsics.ll | 29 +++++++----
llvm/unittests/IR/IRBuilderTest.cpp | 48 ++++++++++++++++-
8 files changed, 132 insertions(+), 34 deletions(-)
diff --git a/llvm/include/llvm/IR/FloatingPointOps.def b/llvm/include/llvm/IR/FloatingPointOps.def
index 465be751a0087..67a66a84223b4 100644
--- a/llvm/include/llvm/IR/FloatingPointOps.def
+++ b/llvm/include/llvm/IR/FloatingPointOps.def
@@ -13,18 +13,19 @@
// Describes floating-point operation.
// Arguments of the entries are:
// N - intrinsic function name,
+// R - true if the operation depends on rounding mode
// D - DAG node corresponding to the intrinsic.
#ifndef FUNCTION
-#define FUNCTION(N,D)
+#define FUNCTION(N,R,D)
#endif
// Describes floating-point operation, which lowers to STRICT_* nodes in DAG.
#ifndef LEGACY_DAG
-#define LEGACY_DAG(N,D) FUNCTION(N,D)
+#define LEGACY_DAG(N,R,D) FUNCTION(N,R,D)
#endif
-LEGACY_DAG(nearbyint, FNEARBYINT)
-LEGACY_DAG(trunc, FTRUNC)
+LEGACY_DAG(nearbyint, 1, FNEARBYINT)
+LEGACY_DAG(trunc, 0, FTRUNC)
#undef FUNCTION
#undef LEGACY_DAG
diff --git a/llvm/include/llvm/IR/IntrinsicInst.h b/llvm/include/llvm/IR/IntrinsicInst.h
index 98a2987afbba0..9f14244da7117 100644
--- a/llvm/include/llvm/IR/IntrinsicInst.h
+++ b/llvm/include/llvm/IR/IntrinsicInst.h
@@ -143,6 +143,10 @@ class IntrinsicInst : public CallInst {
/// instructions ordering.
static bool isFloatingPointOperation(Intrinsic::ID IID);
+ /// Check if \p ID represents a function that may access FP environment and
+ /// depends on rounding mode.
+ static bool dependsOnRoundingMode(Intrinsic::ID IID);
+
/// Methods for support type inquiry through isa, cast, and dyn_cast:
static bool classof(const CallInst *I) {
auto *F = dyn_cast_or_null<Function>(I->getCalledOperand());
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 14ec379cf7c7f..6612562b3113a 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -7080,7 +7080,7 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
#include "llvm/IR/VPIntrinsics.def"
visitVectorPredicationIntrinsic(cast<VPIntrinsic>(I));
return;
-#define FUNCTION(NAME, DAGN) \
+#define FUNCTION(NAME, RM, DAGN) \
case Intrinsic::NAME: \
visitFPOperation(I, ISD::DAGN); \
break;
@@ -9638,7 +9638,7 @@ bool SelectionDAGBuilder::visitFPOperation(const CallInst &I, unsigned Opcode) {
switch (Opcode) {
default:
break;
-#define LEGACY_DAG(NAME, DAGN) \
+#define LEGACY_DAG(NAME, RM, DAGN) \
case ISD::DAGN: \
Opcode = ISD::STRICT_##DAGN; \
break;
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index c1448f8590fc4..23fb782d4de83 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -202,33 +202,55 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
// If the builder specifies non-default floating-point options, add
// corresponding operand bundle unless a bundle with such tag is already
// present.
- bool NeedRounding;
- bool NeedExceptions;
+ bool AddRounding;
+ bool AddExceptions;
if (IsFPConstrained) {
- NeedRounding = DefaultConstrainedRounding != RoundingMode::Dynamic;
- NeedExceptions = DefaultConstrainedExcept != fp::ebStrict;
+ AddRounding = DefaultConstrainedRounding != RoundingMode::Dynamic;
+ AddExceptions = DefaultConstrainedExcept != fp::ebStrict;
} else {
- NeedRounding =
+ AddRounding =
DefaultConstrainedRounding != RoundingMode::NearestTiesToEven;
- NeedExceptions = false;
+ AddExceptions = false;
assert(DefaultConstrainedExcept == fp::ebIgnore &&
"FP exception in default mode must be ignored");
}
- // Options specified by bundles have higher precedence.
+
+ // Options specified by bundle arguments have higher precedence. Some FP
+ // operations do not depend on rounding mode, do not add "fp.round"
+ // to them.
+ bool NoRoundingMode = !IntrinsicInst::dependsOnRoundingMode(ID);
+ bool RoundingModeIsSpecified = false;
for (const auto &Bundle : OpBundles) {
- if (NeedRounding && Bundle.getTag() == "fp.round")
- NeedRounding = false;
- if (NeedExceptions && Bundle.getTag() == "fp.except")
- NeedExceptions = false;
+ if (!RoundingModeIsSpecified)
+ RoundingModeIsSpecified = Bundle.getTag() == "fp.round";
+ if (AddRounding && RoundingModeIsSpecified)
+ AddRounding = false;
+ if (AddExceptions && Bundle.getTag() == "fp.except")
+ AddExceptions = false;
}
- if (NeedRounding || NeedExceptions) {
- ActualBundles.append(OpBundles.begin(), OpBundles.end());
- if (NeedRounding)
+
+ if (AddRounding && NoRoundingMode)
+ AddRounding = false;
+ if (!OpBundles.empty()) {
+ if (RoundingModeIsSpecified && NoRoundingMode) {
+ std::copy_if(OpBundles.begin(), OpBundles.end(),
+ std::back_inserter(ActualBundles),
+ [](const OperandBundleDef &Bundle) {
+ return Bundle.getTag() != "fp.round";
+ });
+ } else {
+ ActualBundles.append(OpBundles.begin(), OpBundles.end());
+ }
+ ActualBundlesRef = ActualBundles;
+ }
+ if (AddRounding || AddExceptions) {
+ if (AddRounding)
createRoundingBundle(ActualBundles, DefaultConstrainedRounding);
- if (NeedExceptions)
+ if (AddExceptions)
createExceptionBundle(ActualBundles, DefaultConstrainedExcept);
ActualBundlesRef = ActualBundles;
}
+
if (IsFPConstrained) {
// Due to potential setting FP exception bits, in modes other than
// the default, the memory effects must include read/write access
diff --git a/llvm/lib/IR/IntrinsicInst.cpp b/llvm/lib/IR/IntrinsicInst.cpp
index 8d7c781e9f47a..a493d6497e39c 100644
--- a/llvm/lib/IR/IntrinsicInst.cpp
+++ b/llvm/lib/IR/IntrinsicInst.cpp
@@ -68,7 +68,7 @@ bool IntrinsicInst::mayLowerToFunctionCall(Intrinsic::ID IID) {
bool IntrinsicInst::isFloatingPointOperation(Intrinsic::ID IID) {
switch (IID) {
-#define FUNCTION(NAME, D) case Intrinsic::NAME:
+#define FUNCTION(NAME, R, D) case Intrinsic::NAME:
#include "llvm/IR/FloatingPointOps.def"
return true;
default:
@@ -76,6 +76,17 @@ bool IntrinsicInst::isFloatingPointOperation(Intrinsic::ID IID) {
}
}
+bool IntrinsicInst::dependsOnRoundingMode(Intrinsic::ID IID) {
+ switch (IID) {
+#define FUNCTION(NAME, R, D) \
+ case Intrinsic::NAME: \
+ return R;
+#include "llvm/IR/FloatingPointOps.def"
+ default:
+ return false;
+ }
+}
+
//===----------------------------------------------------------------------===//
/// DbgVariableIntrinsic - This is the common base class for debug info
/// intrinsics for variables.
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index e9201936b2310..b9692c2732d8d 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -4045,7 +4045,8 @@ void Verifier::visitCallBase(CallBase &Call) {
"Return type cannot be x86_amx for indirect call!");
}
- if (Intrinsic::ID ID = Call.getIntrinsicID())
+ Intrinsic::ID ID = Call.getIntrinsicID();
+ if (ID)
visitIntrinsicCall(ID, Call);
// Verify that a callsite has at most one "deopt", at most one "funclet", at
@@ -4122,6 +4123,10 @@ void Verifier::visitCallBase(CallBase &Call) {
} else if (Tag == LLVMContext::OB_fp_round) {
Check(!FoundFpeRoundBundle, "Multiple \"fp.round\" operand bundles",
Call);
+ Check(IntrinsicInst::dependsOnRoundingMode(ID),
+ "\"fp.round\" operand bundles cannot be specified on an intrinsic "
+ "that does not depend on rounding mode",
+ Call);
bool FoundRoundingMode = false;
for (auto &U : BU.Inputs) {
Value *V = U.get();
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index f96d519870aaa..af659cb2dd2f0 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -53,46 +53,46 @@ entry:
; Test multiple fp.round bundles.
; CHECK-NEXT: Multiple "fp.round" operand bundles
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
+; CHECK-NEXT: %ftrunc = call double @llvm.nearbyint.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
define double @f6(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
+ %ftrunc = call double @llvm.nearbyint.f64(double %a) #0 [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
ret double %ftrunc
}
; Test fp.round bundle that has more than one rounding mode specification.
; CHECK-NEXT: Rounding mode is specified more that once
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
+; CHECK-NEXT: %ftrunc = call double @llvm.nearbyint.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
define double @f7(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
+ %ftrunc = call double @llvm.nearbyint.f64(double %a) #0 [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
ret double %ftrunc
}
; Test fp.round bundle that has non-metadata operand.
; CHECK-NEXT: Value of a "fp.round" bundle operand must be a metadata
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(i32 0) ]
+; CHECK-NEXT: %ftrunc = call double @llvm.nearbyint.f64(double %a) #{{[0-9]+}} [ "fp.round"(i32 0) ]
define double @f8(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(i32 0) ]
+ %ftrunc = call double @llvm.nearbyint.f64(double %a) #0 [ "fp.round"(i32 0) ]
ret double %ftrunc
}
; Test fp.round bundle that has non-string operand.
; CHECK-NEXT: Value of a "fp.round" bundle operand must be a string
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata i64 3) ]
+; CHECK-NEXT: %ftrunc = call double @llvm.nearbyint.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata i64 3) ]
define double @f9(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !{i64 3}) ]
+ %ftrunc = call double @llvm.nearbyint.f64(double %a) #0 [ "fp.round"(metadata !{i64 3}) ]
ret double %ftrunc
}
; Test fp.round bundle that specifies incorrect value.
; CHECK-NEXT: Unrecognized value in "fp.round" bundle operand
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"qqq") ]
+; CHECK-NEXT: %ftrunc = call double @llvm.nearbyint.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"qqq") ]
define double @f10(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"qqq") ]
+ %ftrunc = call double @llvm.nearbyint.f64(double %a) #0 [ "fp.round"(metadata !"qqq") ]
ret double %ftrunc
}
@@ -150,5 +150,14 @@ entry:
ret double %ftrunc
}
+; Test fp.round bundle that was attached to a function that does not depend on rounding mode.
+; CHECK-NEXT: "fp.round" operand bundles cannot be specified on an intrinsic that does not depend on rounding mode
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero") ]
+define double @f17(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"towardzero") ]
+ ret double %ftrunc
+}
+
attributes #0 = { strictfp }
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index c3388d64c664b..a39a5102a98fb 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -524,6 +524,8 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
Value *FnArg = Builder.CreateLoad(GVDouble->getValueType(), GVDouble);
Function *Fn = Intrinsic::getOrInsertDeclaration(
M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
+ Function *Fn2 = Intrinsic::getOrInsertDeclaration(M.get(), Intrinsic::trunc,
+ {Type::getDoubleTy(Ctx)});
// A floating-point operation does not have side effects in default
// environment even.
@@ -555,6 +557,18 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_TRUE(ME.doesNotAccessMemory());
}
+ // If the called intrinsic does not depend on rounding mode, the "fp.round"
+ // bundle is not added.
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ Value *V = Builder.CreateCall(Fn2, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ }
+
// Check call with FP bundles, exception behavior is set to default value.
// nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
{
@@ -576,7 +590,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
// nearbyint(%x) [ "fp.round" (metadata !"tonearest"),
// "fp.except" (metadata !"ignore") ]
{
- SmallVector<OperandBundleDef, 1> Bundles;
+ SmallVector<OperandBundleDef, 2> Bundles;
llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
@@ -590,6 +604,23 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_TRUE(ME.doesNotAccessMemory());
}
+ // If the called intrinsic does not depend on rounding mode, the "fp.round"
+ // bundle is not added, even if it is specified.
+ {
+ SmallVector<OperandBundleDef, 2> Bundles;
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn2, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
// If the builder object specifies a rounding mode, the resulting call will
// include the corresponding "fp.round" operand bundle.
{
@@ -623,6 +654,21 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
+
+ // If the builder object specifies a non-default rounding mode and the operand
+ // bundles contain an "fp.round" bundle, but the called function does not
+ // depend on rounding mode, the resulting call should not have "fp.round"
+ // operand bundle.
+ {
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardNegative);
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
+ Value *V = Builder.CreateCall(Fn2, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ }
}
TEST_F(IRBuilderTest, FPBundlesStrict) {
>From a56d3d1550a287774a67ae214d14be5b8a2b4dd5 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 11 Dec 2025 00:11:46 +0700
Subject: [PATCH 14/24] Support of assumed rounding mode
---
llvm/include/llvm/IR/FPEnv.h | 46 ++++++--
llvm/include/llvm/IR/InstrTypes.h | 22 +++-
llvm/lib/IR/FPEnv.cpp | 46 ++++++--
llvm/lib/IR/IRBuilder.cpp | 17 ++-
llvm/lib/IR/Instructions.cpp | 37 +++---
llvm/lib/IR/Verifier.cpp | 2 +-
llvm/test/Verifier/fp-intrinsics.ll | 9 ++
llvm/unittests/IR/IRBuilderTest.cpp | 173 ++++++++++++++++------------
8 files changed, 242 insertions(+), 110 deletions(-)
diff --git a/llvm/include/llvm/IR/FPEnv.h b/llvm/include/llvm/IR/FPEnv.h
index 6f95b7c688a9d..968ccc35bc934 100644
--- a/llvm/include/llvm/IR/FPEnv.h
+++ b/llvm/include/llvm/IR/FPEnv.h
@@ -44,22 +44,54 @@ enum ExceptionBehavior : uint8_t {
}
+/// Keeps information about rounding mode used in a floating-point operation.
+///
+/// In addition to rounding mode used for the execution (the effective mode),
+/// this class stores the method how the mode is specified. It may be a "static"
+/// rounding node, where the rounding is encoded directly in the operation, or
+/// an "assumed" mode, which assumes that the relevant mode is loaded into the
+/// FP control register.
+class RoundingSpec {
+ RoundingMode Mode;
+ bool IsAssumed;
+
+public:
+ RoundingSpec(RoundingMode RM, bool D) : Mode(RM), IsAssumed(D) {}
+
+ static RoundingSpec makeStatic(RoundingMode RM) { return {RM, false}; }
+ static RoundingSpec makeAssumed(RoundingMode RM) { return {RM, true}; }
+ static RoundingSpec makeDynamic() { return {RoundingMode::Dynamic, true}; }
+
+ RoundingMode getEffective() const { return Mode; }
+ void setEffective(RoundingMode RM) { Mode = RM; }
+ bool isAssumed() const { return IsAssumed; }
+ void setAssumed(bool D) { IsAssumed = D; }
+ bool isDynamic() const { return IsAssumed || Mode == RoundingMode::Dynamic; }
+ bool isStatic() const { return !isDynamic(); }
+ bool isDefault() const {
+ return Mode == RoundingMode::NearestTiesToEven && IsAssumed;
+ }
+};
+
+LLVM_ABI std::optional<RoundingSpec> readRoundingSpec(StringRef);
+LLVM_ABI std::string printRoundingSpec(RoundingSpec R);
+
+/// Returns a valid RoundingMode enumerator given a string that represents
+/// rounding mode in operand bundles.
+LLVM_ABI RoundingMode readRoundingMode(StringRef);
+
/// Returns a valid RoundingMode enumerator when given a string
/// that is valid as input in constrained intrinsic rounding mode
/// metadata.
LLVM_ABI std::optional<RoundingMode> convertStrToRoundingMode(StringRef);
-/// Returns a valid RoundingMode enumerator given a string that is used as
-/// rounding mode specifier in operand bundles.
-std::optional<RoundingMode> convertBundleToRoundingMode(StringRef);
-
/// For any RoundingMode enumerator, returns a string valid as input in
/// constrained intrinsic rounding mode metadata.
LLVM_ABI std::optional<StringRef> convertRoundingModeToStr(RoundingMode);
/// For any RoundingMode enumerator, returns a string to be used in operand
/// bundles.
-std::optional<StringRef> convertRoundingModeToBundle(RoundingMode);
+LLVM_ABI std::optional<StringRef> convertRoundingModeToBundle(RoundingMode);
/// Returns a valid ExceptionBehavior enumerator when given a string
/// valid as input in constrained intrinsic exception behavior metadata.
@@ -68,7 +100,7 @@ LLVM_ABI std::optional<fp::ExceptionBehavior>
/// Returns a valid ExceptionBehavior enumerator given a string from the operand
/// bundle argument.
-std::optional<fp::ExceptionBehavior>
+LLVM_ABI std::optional<fp::ExceptionBehavior>
convertBundleToExceptionBehavior(StringRef);
/// For any ExceptionBehavior enumerator, returns a string valid as
@@ -78,7 +110,7 @@ LLVM_ABI std::optional<StringRef>
/// Return string representing the given exception behavior for use in operand
/// bundles
-std::optional<StringRef>
+LLVM_ABI std::optional<StringRef>
convertExceptionBehaviorToBundle(fp::ExceptionBehavior);
inline raw_ostream &operator<<(raw_ostream &OS, fp::ExceptionBehavior EB) {
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index 744aebbfab3d9..d8362ae2ff92b 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -1025,6 +1025,14 @@ struct OperandBundleUse {
return false;
}
+ /// Return the value of the given input as a string. The input must be
+ /// a string metadata.
+ StringRef getInputAsString(size_t Ndx) const {
+ const Value *V = Inputs[Ndx].get();
+ const auto *MD = cast<MetadataAsValue>(V)->getMetadata();
+ return cast<MDString>(MD)->getString();
+ }
+
/// Return the tag of this operand bundle as a string.
StringRef getTagName() const {
return Tag->getKey();
@@ -1088,6 +1096,14 @@ template <typename InputTy> class OperandBundleDefT {
input_iterator input_begin() const { return Inputs.begin(); }
input_iterator input_end() const { return Inputs.end(); }
+ /// Return the value of the given input as a string. The input must be
+ /// a string metadata.
+ StringRef getInputAsString(size_t Ndx) const {
+ const Value *V = Inputs[Ndx];
+ const auto *MD = cast<MetadataAsValue>(V)->getMetadata();
+ return cast<MDString>(MD)->getString();
+ }
+
StringRef getTag() const { return Tag; }
};
@@ -1098,7 +1114,7 @@ using ConstOperandBundleDef = OperandBundleDefT<const Value *>;
/// bundle set.
void addRoundingBundle(LLVMContext &Ctx,
SmallVectorImpl<OperandBundleDef> &Bundles,
- RoundingMode Rounding);
+ RoundingMode Rounding, bool Assumed = false);
/// Add a bundle with tag "fp.except" and the specified exception behavior to
/// the given bundle set.
@@ -2181,6 +2197,10 @@ class CallBase : public Instruction {
/// Returns true, if this call has a floating-point operand bundle.
bool hasFloatingPointOperandBundle() const;
+ /// Return information about the rounding mode to be used for evaluation of
+ /// the called instruction.
+ RoundingSpec getRoundingSpec() const;
+
/// Return the effective rounding mode for this call.
RoundingMode getRoundingMode() const;
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index d8d9ab1d817da..db9cf9a07a197 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -21,6 +21,40 @@
using namespace llvm;
+std::optional<RoundingSpec> llvm::readRoundingSpec(StringRef Str) {
+ SmallVector<StringRef, 2> Parts;
+ Str.split(Parts, ',');
+ if (Parts.size() < 1 || Parts.size() > 2)
+ return std::nullopt;
+
+ RoundingMode Effective = readRoundingMode(Parts.front());
+ if (Effective == RoundingMode::Invalid)
+ return std::nullopt;
+ if (Parts.size() == 1)
+ return RoundingSpec::makeStatic(Effective);
+
+ RoundingMode Dynamic = readRoundingMode(Parts[1]);
+ if (Dynamic == RoundingMode::Invalid)
+ return std::nullopt;
+ if (Dynamic != RoundingMode::Dynamic)
+ std::swap(Effective, Dynamic);
+ if (Dynamic != RoundingMode::Dynamic)
+ return std::nullopt;
+
+ return RoundingSpec(Effective, true);
+}
+
+RoundingMode llvm::readRoundingMode(StringRef RoundingArg) {
+ return StringSwitch<RoundingMode>(RoundingArg)
+ .Case("dynamic", RoundingMode::Dynamic)
+ .Case("tonearest", RoundingMode::NearestTiesToEven)
+ .Case("tonearestaway", RoundingMode::NearestTiesToAway)
+ .Case("downward", RoundingMode::TowardNegative)
+ .Case("upward", RoundingMode::TowardPositive)
+ .Case("towardzero", RoundingMode::TowardZero)
+ .Default(RoundingMode::Invalid);
+}
+
std::optional<RoundingMode>
llvm::convertStrToRoundingMode(StringRef RoundingArg) {
// For dynamic rounding mode, we use round to nearest but we will set the
@@ -35,18 +69,6 @@ llvm::convertStrToRoundingMode(StringRef RoundingArg) {
.Default(std::nullopt);
}
-std::optional<RoundingMode>
-llvm::convertBundleToRoundingMode(StringRef RoundingArg) {
- return StringSwitch<std::optional<RoundingMode>>(RoundingArg)
- .Case("dynamic", RoundingMode::Dynamic)
- .Case("tonearest", RoundingMode::NearestTiesToEven)
- .Case("tonearestaway", RoundingMode::NearestTiesToAway)
- .Case("downward", RoundingMode::TowardNegative)
- .Case("upward", RoundingMode::TowardPositive)
- .Case("towardzero", RoundingMode::TowardZero)
- .Default(std::nullopt);
-}
-
std::optional<StringRef>
llvm::convertRoundingModeToStr(RoundingMode UseRounding) {
std::optional<StringRef> RoundingStr;
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 23fb782d4de83..25f1d9e10e5f9 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -215,7 +215,8 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
"FP exception in default mode must be ignored");
}
- // Options specified by bundle arguments have higher precedence. Some FP
+ // The options specified by the specified bundles have higher
+ // precedence than the options specified in the builder. Some FP
// operations do not depend on rounding mode, do not add "fp.round"
// to them.
bool NoRoundingMode = !IntrinsicInst::dependsOnRoundingMode(ID);
@@ -228,9 +229,9 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
if (AddExceptions && Bundle.getTag() == "fp.except")
AddExceptions = false;
}
-
if (AddRounding && NoRoundingMode)
AddRounding = false;
+
if (!OpBundles.empty()) {
if (RoundingModeIsSpecified && NoRoundingMode) {
std::copy_if(OpBundles.begin(), OpBundles.end(),
@@ -240,6 +241,18 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
});
} else {
ActualBundles.append(OpBundles.begin(), OpBundles.end());
+#ifndef NDEBUG
+ // In the default floating-point mode assumed rounding is not
+ // allowed, except when rounding mode is NearestTiesToEven.
+ if (!IsFPConstrained)
+ for (const OperandBundleDef &Bundle : ActualBundles) {
+ if (Bundle.getTag() == "fp.round") {
+ StringRef MDS = Bundle.getInputAsString(0);
+ RoundingSpec RS = *readRoundingSpec(MDS);
+ assert(RS.isStatic() || RS.isDefault());
+ }
+ }
+#endif
}
ActualBundlesRef = ActualBundles;
}
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index afec918cecff7..70d0c63f411b8 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -643,24 +643,25 @@ bool CallBase::hasFloatingPointOperandBundle() const {
return false;
}
-RoundingMode CallBase::getRoundingMode() const {
- // Try reading rounding mode from operand bundle.
- if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_round)) {
- Value *V = RoundingBundle->Inputs.front();
- Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
- if (auto RM = convertBundleToRoundingMode(cast<MDString>(MD)->getString()))
- return *RM;
- }
+RoundingSpec CallBase::getRoundingSpec() const {
+ // Try reading the rounding mode from the operand bundles.
+ if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_round))
+ if (auto R = readRoundingSpec(RoundingBundle->getInputAsString(0)))
+ return *R;
- // No FP bundle, try to guess from attributes of the current function.
+ // No FP bundle, try to guess from the attributes of the current function.
if (const BasicBlock *BB = getParent())
if (const Function *F = BB->getParent())
return F->getAttributes().hasFnAttr(Attribute::StrictFP)
- ? RoundingMode::Dynamic
- : RoundingMode::NearestTiesToEven;
+ ? RoundingSpec::makeDynamic()
+ : RoundingSpec::makeAssumed(RoundingMode::NearestTiesToEven);
- // Isolated call. Assume default environment.
- return RoundingMode::NearestTiesToEven;
+ // An isolated call. Assume the default environment.
+ return RoundingSpec::makeAssumed(RoundingMode::NearestTiesToEven);
+}
+
+RoundingMode CallBase::getRoundingMode() const {
+ return getRoundingSpec().getEffective();
}
fp::ExceptionBehavior CallBase::getExceptionBehavior() const {
@@ -811,7 +812,7 @@ bool CallBase::hasArgumentWithAdditionalReturnCaptureComponents() const {
void llvm::addRoundingBundle(LLVMContext &Ctx,
SmallVectorImpl<OperandBundleDef> &Bundles,
- RoundingMode Rounding) {
+ RoundingMode Rounding, bool Assumed) {
assert(std::find_if(Bundles.begin(), Bundles.end(),
[](const OperandBundleDef &B) {
return B.getTag() == "fp.round";
@@ -819,7 +820,13 @@ void llvm::addRoundingBundle(LLVMContext &Ctx,
"Bundle 'fp.round' already exists");
std::optional<StringRef> RndStr = convertRoundingModeToBundle(Rounding);
assert(RndStr && "Garbage rounding mode!");
- auto *RoundingMDS = MDString::get(Ctx, *RndStr);
+ StringRef RndValue = *RndStr;
+ std::string RndSpec;
+ if (Assumed && Rounding != RoundingMode::Dynamic) {
+ RndSpec = RndValue.str() + ",dynamic";
+ RndValue = StringRef(RndSpec);
+ }
+ auto *RoundingMDS = MDString::get(Ctx, RndValue);
auto *RM = MetadataAsValue::get(Ctx, RoundingMDS);
Bundles.emplace_back("fp.round", RM);
}
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index b9692c2732d8d..2b2ac9d954c70 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -4137,7 +4137,7 @@ void Verifier::visitCallBase(CallBase &Call) {
Check(isa<MDString>(MD),
"Value of a \"fp.round\" bundle operand must be a string", Call);
StringRef Item = cast<MDString>(MD)->getString();
- if (convertBundleToRoundingMode(Item)) {
+ if (readRoundingSpec(Item)) {
Check(!FoundRoundingMode, "Rounding mode is specified more that once",
Call);
FoundRoundingMode = true;
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index af659cb2dd2f0..4922687ba6a9b 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -159,5 +159,14 @@ entry:
ret double %ftrunc
}
+; Test fp.round bundle that specifies value in incorrext format.
+; CHECK-NEXT: Unrecognized value in "fp.round" bundle operand
+; CHECK-NEXT: %ftrunc = call double @llvm.nearbyint.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero,upward") ]
+define double @f18(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.nearbyint.f64(double %a) #0 [ "fp.round"(metadata !"towardzero,upward") ]
+ ret double %ftrunc
+}
+
attributes #0 = { strictfp }
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index a39a5102a98fb..0846888389361 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -527,8 +527,9 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
Function *Fn2 = Intrinsic::getOrInsertDeclaration(M.get(), Intrinsic::trunc,
{Type::getDoubleTy(Ctx)});
- // A floating-point operation does not have side effects in default
- // environment even.
+ // Check the state of a call without FP bundles. A floating-point operation
+ // does not have side effects in the default environment.
+ // nearbyint(%x)
{
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
@@ -536,12 +537,16 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, RS.getEffective());
+ EXPECT_TRUE(RS.isAssumed());
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
- // Check call with FP bundles, rounding is set to default value.
+ // Check the state of a call with "fp.round" bundle only. The rounding is set
+ // to the default value for the default mode but is static.
// nearbyint(%x) [ "fp.round" (metadata !"tonearest") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -552,60 +557,50 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, RS.getEffective());
+ EXPECT_TRUE(RS.isStatic());
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
- // If the called intrinsic does not depend on rounding mode, the "fp.round"
- // bundle is not added.
- {
- SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
- Value *V = Builder.CreateCall(Fn2, {FnArg}, Bundles);
- auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
- EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
- EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
- }
-
- // Check call with FP bundles, exception behavior is set to default value.
- // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ // Check the state of a call with "fp.round" bundle only. The rounding is set
+ // to the default value for the default mode and is assumed.
+ // nearbyint(%x) [ "fp.round" (metadata !"tonearest,dynamic") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven,
+ true);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, RS.getEffective());
+ EXPECT_TRUE(RS.isAssumed());
+ EXPECT_TRUE(RS.isDynamic());
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
- // Check call with FP bundles, both rounding mode and exception behavior are
- // set.
- // nearbyint(%x) [ "fp.round" (metadata !"tonearest"),
- // "fp.except" (metadata !"ignore") ]
+ // If the called intrinsic does not depend on rounding mode, the "fp.round"
+ // bundle is not added.
{
- SmallVector<OperandBundleDef, 2> Bundles;
+ SmallVector<OperandBundleDef, 1> Bundles;
llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
- llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
- Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ Value *V = Builder.CreateCall(Fn2, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
- EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
- EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
- MemoryEffects ME = I->getMemoryEffects();
- EXPECT_TRUE(ME.doesNotAccessMemory());
}
// If the called intrinsic does not depend on rounding mode, the "fp.round"
- // bundle is not added, even if it is specified.
+ // bundle is not added, even if it is explicitly specified.
{
SmallVector<OperandBundleDef, 2> Bundles;
llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
@@ -632,13 +627,17 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::TowardNegative, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::TowardNegative, RS.getEffective());
+ EXPECT_TRUE(RS.isStatic());
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
+ Builder.resetModeToStrictFP(false);
}
- // If the builder object specifies a rounding mode but the operand bundles
- // already contain an "fp.round" bundle, the builder's specified mode
+ // If the builder object specifies a rounding mode but the provided operand
+ // bundles already contain an "fp.round" bundle, the builder's specified mode
// is ignored.
{
Builder.setDefaultConstrainedRounding(RoundingMode::TowardNegative);
@@ -650,9 +649,13 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::TowardZero, RS.getEffective());
+ EXPECT_TRUE(RS.isStatic());
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
+ Builder.resetModeToStrictFP(false);
}
// If the builder object specifies a non-default rounding mode and the operand
@@ -668,6 +671,49 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ Builder.resetModeToStrictFP(false);
+ }
+
+ // Check the state of a call with "fp.except" bundle only.
+ // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, RS.getEffective());
+ EXPECT_TRUE(RS.isAssumed());
+ EXPECT_TRUE(RS.isDynamic());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // Check call with FP bundles, both rounding mode and exception behavior are
+ // set.
+ // nearbyint(%x) [ "fp.round" (metadata !"tonearest"),
+ // "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 2> Bundles;
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, RS.getEffective());
+ EXPECT_TRUE(RS.isStatic());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
}
}
@@ -684,8 +730,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
Builder.resetModeToStrictFP(true);
- // A floating-point operation has side effects in strictfp environment even
- // if it has no FP bundles.
+ // Check the state of a call without FP bundles.
{
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
@@ -698,8 +743,8 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // A call may have FP bundles, in this case it has a bundle that specifies
- // default (dynamic) rounding mode.
+ // Check the state of a call with "fp.round" bundle only. The rounding is set
+ // to the default value for the strictfp mode.
// nearbyint(%x) [ "fp.round" (metadata !"dynamic") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -715,7 +760,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // FP bundle may specify a non-default rounding mode.
+ // FP bundle may specify a static rounding mode.
// nearbyint(%x) [ "fp.round" (metadata !"towardzero") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -726,28 +771,32 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::TowardZero, RS.getEffective());
+ EXPECT_TRUE(RS.isStatic());
+ EXPECT_FALSE(RS.isDynamic());
EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // FP bundles may specify exception behavior, here it is the default value.
- // nearbyint(%x) [ "fp.except" (metadata !"strict") ]
+ // FP bundle may specify an assumed rounding mode.
+ // nearbyint(%x) [ "fp.round" (metadata !"towardzero,dynamic") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addExceptionBundle(Ctx, Bundles, fp::ebStrict);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero, true);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
- EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
- EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
- MemoryEffects ME = I->getMemoryEffects();
- EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ RoundingSpec RS = I->getRoundingSpec();
+ EXPECT_EQ(RoundingMode::TowardZero, RS.getEffective());
+ EXPECT_TRUE(RS.isAssumed());
+ EXPECT_TRUE(RS.isDynamic());
}
- // FP bundles may specify exception behavior, including non-default values.
+ // FP bundles may specify exception behavior.
// nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
@@ -763,27 +812,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
EXPECT_TRUE(ME.doesAccessInaccessibleMem());
}
- // FP bundles may specify both rounding mode and exception behavior, here is
- // the case of default values.
- // nearbyint(%x) [ "fp.round" (metadata !"dynamic"),
- // "fp.except" (metadata !"strict") ]
- {
- SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
- llvm::addExceptionBundle(Ctx, Bundles, fp::ebStrict);
- Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
- auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
- EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
- EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
- EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
- MemoryEffects ME = I->getMemoryEffects();
- EXPECT_TRUE(ME.doesAccessInaccessibleMem());
- }
-
- // FP bundles may specify both rounding mode and exception behavior, here is
- // the case of non-default values.
+ // FP bundles may specify both rounding mode and exception behavior.
// nearbyint(%x) [ "fp.round" (metadata !"towardzero"),
// "fp.except" (metadata !"ignore") ]
{
>From 84efc9b6b964ce01f8f3e3c37d5a69f0931d3a5e Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Fri, 19 Dec 2025 22:42:54 +0700
Subject: [PATCH 15/24] Cleanup
---
llvm/include/llvm/IR/Function.h | 3 +++
llvm/include/llvm/IR/IRBuilder.h | 9 ++++----
llvm/include/llvm/IR/InstrTypes.h | 6 ++---
llvm/include/llvm/IR/IntrinsicInst.h | 12 ----------
llvm/include/llvm/IR/Intrinsics.h | 12 ++++++++++
llvm/lib/CodeGen/AtomicExpandPass.cpp | 5 ++---
llvm/lib/CodeGen/HardwareLoops.cpp | 6 ++---
llvm/lib/IR/AsmWriter.cpp | 2 +-
llvm/lib/IR/Function.cpp | 4 ++++
llvm/lib/IR/IRBuilder.cpp | 4 ++--
llvm/lib/IR/Instructions.cpp | 21 +++++++++++-------
llvm/lib/IR/IntrinsicInst.cpp | 21 ------------------
llvm/lib/IR/Intrinsics.cpp | 22 +++++++++++++++++++
llvm/lib/IR/Verifier.cpp | 2 +-
.../Target/AMDGPU/AMDGPUAtomicOptimizer.cpp | 2 +-
llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp | 2 +-
llvm/lib/Transforms/Utils/CloneFunction.cpp | 2 +-
.../Transforms/Utils/LibCallsShrinkWrap.cpp | 2 +-
llvm/lib/Transforms/Utils/LowerAtomic.cpp | 3 +--
llvm/unittests/IR/IRBuilderTest.cpp | 12 +++++-----
20 files changed, 81 insertions(+), 71 deletions(-)
diff --git a/llvm/include/llvm/IR/Function.h b/llvm/include/llvm/IR/Function.h
index f39fe509a49a4..5303aa60be219 100644
--- a/llvm/include/llvm/IR/Function.h
+++ b/llvm/include/llvm/IR/Function.h
@@ -260,6 +260,9 @@ class LLVM_ABI Function : public GlobalObject, public ilist_node<Function> {
/// getIntrinsicID() returns Intrinsic::not_intrinsic.
bool isConstrainedFPIntrinsic() const;
+ /// Returns true if the function is a floating-point operations.
+ bool isFPOperation() const;
+
/// Update internal caches that depend on the function name (such as the
/// intrinsic ID and libcall cache).
/// Note, this method does not need to be called directly, as it is called
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index 77e642f07b38c..dde38674d0519 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -350,17 +350,16 @@ class IRBuilderBase {
/// by this setting.
void setIsFPConstrained(bool IsCon) { IsFPConstrained = IsCon; }
- /// Enable/Disable use of constrained floating point math and reset FP options
- /// according to the selected mode.
- void resetModeToStrictFP(bool IsCon) {
- if (IsCon) {
+ /// Enable/Disable code generation for strictfp functions.
+ void setFPMode(bool IsStrictFP) {
+ if (IsStrictFP) {
setDefaultConstrainedRounding(RoundingMode::Dynamic);
setDefaultConstrainedExcept(fp::ebStrict);
} else {
setDefaultConstrainedRounding(RoundingMode::NearestTiesToEven);
setDefaultConstrainedExcept(fp::ebIgnore);
}
- setIsFPConstrained(IsCon);
+ setIsFPConstrained(IsStrictFP);
}
/// Query for the use of constrained floating point math
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index d8362ae2ff92b..8405bf94eb9af 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -1181,9 +1181,6 @@ class CallBase : public Instruction {
/// number of extra operands.
LLVM_ABI unsigned getNumSubclassExtraOperandsDynamic() const;
- /// Get memory effects specific to floating-point operations.
- MemoryEffects getFloatingPointMemoryEffects() const;
-
public:
using Instruction::getContext;
@@ -1223,6 +1220,9 @@ class CallBase : public Instruction {
return nullptr;
}
+ /// Get memory effects specific to floating-point operations.
+ std::optional<MemoryEffects> getFloatingPointMemoryEffects() const;
+
static bool classof(const Instruction *I) {
return I->getOpcode() == Instruction::Call ||
I->getOpcode() == Instruction::Invoke ||
diff --git a/llvm/include/llvm/IR/IntrinsicInst.h b/llvm/include/llvm/IR/IntrinsicInst.h
index 9f14244da7117..784e599c53f44 100644
--- a/llvm/include/llvm/IR/IntrinsicInst.h
+++ b/llvm/include/llvm/IR/IntrinsicInst.h
@@ -135,18 +135,6 @@ class IntrinsicInst : public CallInst {
/// course of IR transformations
LLVM_ABI static bool mayLowerToFunctionCall(Intrinsic::ID IID);
- /// Check if \p ID represents a function that may access FP environment and
- /// may have FP operand bundles.
- ///
- /// Access to FP environment means that in the strict FP environment the
- /// function has read/write memory effect, which is used to maintain proper
- /// instructions ordering.
- static bool isFloatingPointOperation(Intrinsic::ID IID);
-
- /// Check if \p ID represents a function that may access FP environment and
- /// depends on rounding mode.
- static bool dependsOnRoundingMode(Intrinsic::ID IID);
-
/// Methods for support type inquiry through isa, cast, and dyn_cast:
static bool classof(const CallInst *I) {
auto *F = dyn_cast_or_null<Function>(I->getCalledOperand());
diff --git a/llvm/include/llvm/IR/Intrinsics.h b/llvm/include/llvm/IR/Intrinsics.h
index 5aecec9fd5925..445ce9f7f133e 100644
--- a/llvm/include/llvm/IR/Intrinsics.h
+++ b/llvm/include/llvm/IR/Intrinsics.h
@@ -149,6 +149,18 @@ namespace Intrinsic {
/// Floating-Point Intrinsics" that take rounding mode metadata.
LLVM_ABI bool hasConstrainedFPRoundingModeOperand(ID QID);
+ /// Returns true if \p ID represents an intrinsic function that may access FP
+ /// environment and may have FP operand bundles.
+ ///
+ /// Access to FP environment means that in the strict FP environment the
+ /// function has read/write memory effect, which is used to maintain proper
+ /// instructions ordering.
+ LLVM_ABI bool isFPOperation(ID IID);
+
+ /// Returns true if \p ID represents a function that may access FP environment
+ /// and depends on rounding mode.
+ LLVM_ABI bool dependsOnRoundingMode(ID IID);
+
/// This is a type descriptor which explains the type requirements of an
/// intrinsic. This is returned by getIntrinsicInfoTableEntries.
struct IITDescriptor {
diff --git a/llvm/lib/CodeGen/AtomicExpandPass.cpp b/llvm/lib/CodeGen/AtomicExpandPass.cpp
index 4527dd545a3af..cefe2f2f37ab2 100644
--- a/llvm/lib/CodeGen/AtomicExpandPass.cpp
+++ b/llvm/lib/CodeGen/AtomicExpandPass.cpp
@@ -166,7 +166,7 @@ struct ReplacementIRBuilder
SetInsertPoint(I);
this->CollectMetadataToCopy(I, {LLVMContext::MD_pcsections});
if (BB->getParent()->getAttributes().hasFnAttr(Attribute::StrictFP))
- this->resetModeToStrictFP(true);
+ this->setFPMode(true);
MMRAMD = I->getMetadata(LLVMContext::MD_mmra);
}
@@ -1746,8 +1746,7 @@ bool AtomicExpandImpl::tryExpandAtomicCmpXchg(AtomicCmpXchgInst *CI) {
bool llvm::expandAtomicRMWToCmpXchg(AtomicRMWInst *AI,
CreateCmpXchgInstFun CreateCmpXchg) {
ReplacementIRBuilder Builder(AI, AI->getDataLayout());
- Builder.resetModeToStrictFP(
- AI->getFunction()->hasFnAttribute(Attribute::StrictFP));
+ Builder.setFPMode(AI->getFunction()->hasFnAttribute(Attribute::StrictFP));
// FIXME: If FP exceptions are observable, we should force them off for the
// loop for the FP atomics.
diff --git a/llvm/lib/CodeGen/HardwareLoops.cpp b/llvm/lib/CodeGen/HardwareLoops.cpp
index 7a7a0c11bf05e..d55f20bdf2b71 100644
--- a/llvm/lib/CodeGen/HardwareLoops.cpp
+++ b/llvm/lib/CodeGen/HardwareLoops.cpp
@@ -491,7 +491,7 @@ Value *HardwareLoop::InitLoopCount() {
Value* HardwareLoop::InsertIterationSetup(Value *LoopCountInit) {
IRBuilder<> Builder(BeginBB->getTerminator());
if (BeginBB->getParent()->getAttributes().hasFnAttr(Attribute::StrictFP))
- Builder.resetModeToStrictFP(true);
+ Builder.setFPMode(true);
Type *Ty = LoopCountInit->getType();
bool UsePhi = UsePHICounter || Opts.ForcePhi;
Intrinsic::ID ID = UseLoopGuard
@@ -525,7 +525,7 @@ void HardwareLoop::InsertLoopDec() {
IRBuilder<> CondBuilder(ExitBranch);
if (ExitBranch->getParent()->getParent()->getAttributes().hasFnAttr(
Attribute::StrictFP))
- CondBuilder.resetModeToStrictFP(true);
+ CondBuilder.setFPMode(true);
Value *Ops[] = { LoopDecrement };
Value *NewCond = CondBuilder.CreateIntrinsic(Intrinsic::loop_decrement,
@@ -548,7 +548,7 @@ Instruction* HardwareLoop::InsertLoopRegDec(Value *EltsRem) {
IRBuilder<> CondBuilder(ExitBranch);
if (ExitBranch->getParent()->getParent()->getAttributes().hasFnAttr(
Attribute::StrictFP))
- CondBuilder.resetModeToStrictFP(true);
+ CondBuilder.setFPMode(true);
Value *Ops[] = { EltsRem, LoopDecrement };
Value *Call = CondBuilder.CreateIntrinsic(Intrinsic::loop_decrement_reg,
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index 1b8e78d3297df..33bdbd195ee08 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -4398,7 +4398,7 @@ void AssemblyWriter::printInfoComment(const Value &V, bool isMaterializable) {
if (auto *CI = dyn_cast<CallInst>(&V))
if (Intrinsic::ID IID = CI->getIntrinsicID())
if (const Function *F = CI->getFunction())
- if (IntrinsicInst::isFloatingPointOperation(IID) &&
+ if (Intrinsic::isFPOperation(IID) &&
F->hasFnAttribute(Attribute::StrictFP))
printFPMemoryEffects(Out, CI->getMemoryEffects());
}
diff --git a/llvm/lib/IR/Function.cpp b/llvm/lib/IR/Function.cpp
index ef94c218edb27..1babd9ef7c454 100644
--- a/llvm/lib/IR/Function.cpp
+++ b/llvm/lib/IR/Function.cpp
@@ -556,6 +556,10 @@ bool Function::isConstrainedFPIntrinsic() const {
return Intrinsic::isConstrainedFPIntrinsic(getIntrinsicID());
}
+bool Function::isFPOperation() const {
+ return Intrinsic::isFPOperation(getIntrinsicID());
+}
+
void Function::clearArguments() {
for (Argument &A : makeArgArray(Arguments, NumArgs)) {
A.setName("");
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 25f1d9e10e5f9..30a27c53b3e6a 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -198,7 +198,7 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
bool NeedUpdateMemoryEffects = false;
if (const auto *Func = dyn_cast<Function>(Callee))
if (Intrinsic::ID ID = Func->getIntrinsicID())
- if (IntrinsicInst::isFloatingPointOperation(ID)) {
+ if (Intrinsic::isFPOperation(ID)) {
// If the builder specifies non-default floating-point options, add
// corresponding operand bundle unless a bundle with such tag is already
// present.
@@ -219,7 +219,7 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
// precedence than the options specified in the builder. Some FP
// operations do not depend on rounding mode, do not add "fp.round"
// to them.
- bool NoRoundingMode = !IntrinsicInst::dependsOnRoundingMode(ID);
+ bool NoRoundingMode = !Intrinsic::dependsOnRoundingMode(ID);
bool RoundingModeIsSpecified = false;
for (const auto &Bundle : OpBundles) {
if (!RoundingModeIsSpecified)
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index 70d0c63f411b8..b6e64b697bbb2 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -684,24 +684,28 @@ fp::ExceptionBehavior CallBase::getExceptionBehavior() const {
return fp::ebIgnore;
}
-MemoryEffects CallBase::getFloatingPointMemoryEffects() const {
+std::optional<MemoryEffects> CallBase::getFloatingPointMemoryEffects() const {
if (Intrinsic::ID IntrID = getIntrinsicID())
if (const BasicBlock *BB = getParent())
if (const Function *F = BB->getParent())
- if (F->hasFnAttribute(Attribute::StrictFP) &&
- IntrinsicInst::isFloatingPointOperation(IntrID)) {
- // Floating-point operations in strictfp function always have side
- // effect at least because they can raise exceptions.
- return MemoryEffects::inaccessibleMemOnly();
+ if (Intrinsic::isFPOperation(IntrID)) {
+ if (F->hasFnAttribute(Attribute::StrictFP))
+ // Floating-point operations in strictfp function always have side
+ // effect at least because they can raise exceptions.
+ return MemoryEffects::inaccessibleMemOnly();
+ return MemoryEffects::none();
}
- return MemoryEffects::none();
+ return std::nullopt;
}
MemoryEffects CallBase::getMemoryEffects() const {
MemoryEffects ME = getAttributes().getMemoryEffects();
if (auto *Fn = dyn_cast<Function>(getCalledOperand())) {
MemoryEffects FnME = Fn->getMemoryEffects();
- FnME |= getFloatingPointMemoryEffects();
+ if (auto FPME = getFloatingPointMemoryEffects())
+ FnME =
+ FnME.getWithModRef(IRMemLocation::InaccessibleMem,
+ FPME->getModRef(IRMemLocation::InaccessibleMem));
if (hasOperandBundles()) {
// TODO: Add a method to get memory effects for operand bundles instead.
if (hasReadingOperandBundles())
@@ -717,6 +721,7 @@ MemoryEffects CallBase::getMemoryEffects() const {
}
return ME;
}
+
void CallBase::setMemoryEffects(MemoryEffects ME) {
addFnAttr(Attribute::getWithMemoryEffects(getContext(), ME));
}
diff --git a/llvm/lib/IR/IntrinsicInst.cpp b/llvm/lib/IR/IntrinsicInst.cpp
index a493d6497e39c..281cbd4388b58 100644
--- a/llvm/lib/IR/IntrinsicInst.cpp
+++ b/llvm/lib/IR/IntrinsicInst.cpp
@@ -66,27 +66,6 @@ bool IntrinsicInst::mayLowerToFunctionCall(Intrinsic::ID IID) {
}
}
-bool IntrinsicInst::isFloatingPointOperation(Intrinsic::ID IID) {
- switch (IID) {
-#define FUNCTION(NAME, R, D) case Intrinsic::NAME:
-#include "llvm/IR/FloatingPointOps.def"
- return true;
- default:
- return false;
- }
-}
-
-bool IntrinsicInst::dependsOnRoundingMode(Intrinsic::ID IID) {
- switch (IID) {
-#define FUNCTION(NAME, R, D) \
- case Intrinsic::NAME: \
- return R;
-#include "llvm/IR/FloatingPointOps.def"
- default:
- return false;
- }
-}
-
//===----------------------------------------------------------------------===//
/// DbgVariableIntrinsic - This is the common base class for debug info
/// intrinsics for variables.
diff --git a/llvm/lib/IR/Intrinsics.cpp b/llvm/lib/IR/Intrinsics.cpp
index 186bd1edb8c52..30c8afa3d5081 100644
--- a/llvm/lib/IR/Intrinsics.cpp
+++ b/llvm/lib/IR/Intrinsics.cpp
@@ -834,6 +834,28 @@ bool Intrinsic::hasConstrainedFPRoundingModeOperand(Intrinsic::ID QID) {
}
}
+bool Intrinsic::isFPOperation(ID IID) {
+ switch (IID) {
+#define FUNCTION(NAME, ROUND_MODE, DAGN) case Intrinsic::NAME:
+#include "llvm/IR/FloatingPointOps.def"
+#undef INSTRUCTION
+ return true;
+ default:
+ return false;
+ }
+}
+
+bool Intrinsic::dependsOnRoundingMode(Intrinsic::ID IID) {
+ switch (IID) {
+#define FUNCTION(NAME, R, D) \
+ case Intrinsic::NAME: \
+ return R;
+#include "llvm/IR/FloatingPointOps.def"
+ default:
+ return false;
+ }
+}
+
using DeferredIntrinsicMatchPair =
std::pair<Type *, ArrayRef<Intrinsic::IITDescriptor>>;
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 2b2ac9d954c70..b96df1f691837 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -4123,7 +4123,7 @@ void Verifier::visitCallBase(CallBase &Call) {
} else if (Tag == LLVMContext::OB_fp_round) {
Check(!FoundFpeRoundBundle, "Multiple \"fp.round\" operand bundles",
Call);
- Check(IntrinsicInst::dependsOnRoundingMode(ID),
+ Check(Intrinsic::dependsOnRoundingMode(ID),
"\"fp.round\" operand bundles cannot be specified on an intrinsic "
"that does not depend on rounding mode",
Call);
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp b/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
index 434dda6416123..008dbd93e90ba 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
@@ -649,7 +649,7 @@ void AMDGPUAtomicOptimizerImpl::optimizeAtomic(Instruction &I,
IRBuilder<> B(&I);
if (AtomicRMWInst::isFPOperation(Op)) {
- B.resetModeToStrictFP(I.getFunction()->hasFnAttribute(Attribute::StrictFP));
+ B.setFPMode(I.getFunction()->hasFnAttribute(Attribute::StrictFP));
}
// If we are in a pixel shader, because of how we have to mask out helper
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp b/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
index 2cffc5aab1b24..72786dbba9d41 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
@@ -621,7 +621,7 @@ bool AMDGPULibCalls::fold(CallInst *CI) {
IRBuilder<> B(CI);
if (CI->isStrictFP())
- B.resetModeToStrictFP(true);
+ B.setFPMode(true);
if (FPMathOperator *FPOp = dyn_cast<FPMathOperator>(CI)) {
// Under unsafe-math, evaluate calls if possible.
diff --git a/llvm/lib/Transforms/Utils/CloneFunction.cpp b/llvm/lib/Transforms/Utils/CloneFunction.cpp
index 26b1b01855ac1..fb4259fb08c03 100644
--- a/llvm/lib/Transforms/Utils/CloneFunction.cpp
+++ b/llvm/lib/Transforms/Utils/CloneFunction.cpp
@@ -522,7 +522,7 @@ PruningFunctionCloner::cloneInstruction(BasicBlock::const_iterator II) {
NewInst = CallInst::Create(IFn, Args, OldInst.getName() + ".strict");
} else if (auto *ICall = dyn_cast<IntrinsicInst>(&OldInst)) {
Intrinsic::ID IID = ICall->getIntrinsicID();
- if (IntrinsicInst::isFloatingPointOperation(IID)) {
+ if (Intrinsic::isFPOperation(IID)) {
// The function call needs to be cloned, but the bundles attached to it
// may require update.
if (const BasicBlock *BB = OldInst.getParent())
diff --git a/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp b/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
index 8cf57bbc6ad78..463c0b308f494 100644
--- a/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
+++ b/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
@@ -103,7 +103,7 @@ class LibCallsShrinkWrap : public InstVisitor<LibCallsShrinkWrap> {
if (!Arg->getType()->isFloatTy())
V = ConstantFoldCastInstruction(Instruction::FPExt, V, Arg->getType());
if (BBBuilder.GetInsertBlock()->getParent()->hasFnAttribute(Attribute::StrictFP))
- BBBuilder.resetModeToStrictFP(true);
+ BBBuilder.setFPMode(true);
return BBBuilder.CreateFCmp(Cmp, Arg, V);
}
diff --git a/llvm/lib/Transforms/Utils/LowerAtomic.cpp b/llvm/lib/Transforms/Utils/LowerAtomic.cpp
index b0c788fa74a5d..2c9245fd83796 100644
--- a/llvm/lib/Transforms/Utils/LowerAtomic.cpp
+++ b/llvm/lib/Transforms/Utils/LowerAtomic.cpp
@@ -124,8 +124,7 @@ Value *llvm::buildAtomicRMWValue(AtomicRMWInst::BinOp Op,
bool llvm::lowerAtomicRMWInst(AtomicRMWInst *RMWI) {
IRBuilder<> Builder(RMWI);
- Builder.resetModeToStrictFP(
- RMWI->getFunction()->hasFnAttribute(Attribute::StrictFP));
+ Builder.setFPMode(RMWI->getFunction()->hasFnAttribute(Attribute::StrictFP));
Value *Ptr = RMWI->getPointerOperand();
Value *Val = RMWI->getValOperand();
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index 0846888389361..cacbf7ebd272e 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -500,12 +500,12 @@ TEST_F(IRBuilderTest, ConstrainedFPFunctionCall) {
Function::Create(FTy, Function::ExternalLinkage, "", M.get());
BasicBlock *CalleeBB = BasicBlock::Create(Ctx, "", Callee);
IRBuilder<> CalleeBuilder(CalleeBB);
- CalleeBuilder.resetModeToStrictFP(true);
+ CalleeBuilder.setFPMode(true);
CalleeBuilder.setConstrainedFPFunctionAttr();
CalleeBuilder.CreateRetVoid();
// Now call the empty constrained FP function.
- Builder.resetModeToStrictFP(true);
+ Builder.setFPMode(true);
Builder.setConstrainedFPFunctionAttr();
CallInst *FCall = Builder.CreateCall(Callee, {});
@@ -633,7 +633,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
- Builder.resetModeToStrictFP(false);
+ Builder.setFPMode(false);
}
// If the builder object specifies a rounding mode but the provided operand
@@ -655,7 +655,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
- Builder.resetModeToStrictFP(false);
+ Builder.setFPMode(false);
}
// If the builder object specifies a non-default rounding mode and the operand
@@ -671,7 +671,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
- Builder.resetModeToStrictFP(false);
+ Builder.setFPMode(false);
}
// Check the state of a call with "fp.except" bundle only.
@@ -728,7 +728,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
Function *Fn = Intrinsic::getOrInsertDeclaration(
M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
- Builder.resetModeToStrictFP(true);
+ Builder.setFPMode(true);
// Check the state of a call without FP bundles.
{
>From 8d74152468a0bb9183ac9460ac9149185481b90b Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 26 Feb 2026 13:21:00 +0700
Subject: [PATCH 16/24] Fix tests after rebase
---
.../AMDGPU/amdgpu-simplify-libcall-pow.ll | 19 +++++++------------
1 file changed, 7 insertions(+), 12 deletions(-)
diff --git a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll
index b2d5bb2faeca7..662dbb67d7b31 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll
@@ -1211,16 +1211,16 @@ define float @test_pow_afn_f32_strictfp(float %x, float %y) #2 {
; NOPRELINK-NEXT: [[TMP6:%.*]] = call nnan nsz afn float @llvm.log2.f32(float [[TMP5]]) #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP7:%.*]] = call nnan nsz afn float @llvm.experimental.constrained.fmul.f32(float [[TMP2]], float [[TMP6]], metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP8:%.*]] = call nnan nsz afn float @llvm.exp2.f32(float [[TMP7]]) #[[ATTR3]]
-; NOPRELINK-NEXT: [[TMP9:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP2]]) #[[ATTR3]]
+; NOPRELINK-NEXT: [[TMP9:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP2]]) #[[ATTR5:[0-9]+]]
; NOPRELINK-NEXT: [[TMP10:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f32(float [[TMP9]], float [[TMP2]], metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP11:%.*]] = call nnan nsz afn float @llvm.experimental.constrained.fmul.f32(float [[TMP2]], float 5.000000e-01, metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR3]]
-; NOPRELINK-NEXT: [[TMP12:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP11]]) #[[ATTR3]]
+; NOPRELINK-NEXT: [[TMP12:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP11]]) #[[ATTR5]]
; NOPRELINK-NEXT: [[TMP13:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f32(float [[TMP12]], float [[TMP11]], metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP14:%.*]] = xor i1 [[TMP13]], true
; NOPRELINK-NEXT: [[TMP15:%.*]] = and i1 [[TMP10]], [[TMP14]]
; NOPRELINK-NEXT: [[TMP16:%.*]] = select nnan nsz afn i1 [[TMP15]], float [[TMP4]], float 1.000000e+00
; NOPRELINK-NEXT: [[TMP17:%.*]] = call nnan nsz afn float @llvm.copysign.f32(float [[TMP8]], float [[TMP16]]) #[[ATTR3]]
-; NOPRELINK-NEXT: [[TMP18:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP2]]) #[[ATTR3]]
+; NOPRELINK-NEXT: [[TMP18:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP2]]) #[[ATTR5]]
; NOPRELINK-NEXT: [[TMP19:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f32(float [[TMP18]], float [[TMP2]], metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP20:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f32(float [[TMP4]], float 0.000000e+00, metadata !"olt", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP21:%.*]] = xor i1 [[TMP19]], true
@@ -1256,15 +1256,10 @@ define float @test_pow_afn_f32_strictfp(float %x, float %y) #2 {
}
define float @test_pow_fast_f32_nobuiltin(float %x, float %y) {
-; PRELINK-LABEL: define float @test_pow_fast_f32_nobuiltin
-; PRELINK-SAME: (float [[X:%.*]], float [[Y:%.*]]) {
-; PRELINK-NEXT: [[POW:%.*]] = tail call fast float @_Z3powff(float [[X]], float [[Y]]) #[[ATTR6:[0-9]+]]
-; PRELINK-NEXT: ret float [[POW]]
-;
-; NOPRELINK-LABEL: define float @test_pow_fast_f32_nobuiltin
-; NOPRELINK-SAME: (float [[X:%.*]], float [[Y:%.*]]) {
-; NOPRELINK-NEXT: [[POW:%.*]] = tail call fast float @_Z3powff(float [[X]], float [[Y]]) #[[ATTR5:[0-9]+]]
-; NOPRELINK-NEXT: ret float [[POW]]
+; CHECK-LABEL: define float @test_pow_fast_f32_nobuiltin
+; CHECK-SAME: (float [[X:%.*]], float [[Y:%.*]]) {
+; CHECK-NEXT: [[POW:%.*]] = tail call fast float @_Z3powff(float [[X]], float [[Y]]) #[[ATTR6:[0-9]+]]
+; CHECK-NEXT: ret float [[POW]]
;
%pow = tail call fast float @_Z3powff(float %x, float %y) #3
ret float %pow
>From ed280c6e5c07c894eb5958e21a4e4492196cad46 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Fri, 19 Dec 2025 22:59:23 +0700
Subject: [PATCH 17/24] Fix behavior in Alias Analysis
---
.../CodeGen/strictfp-elementwise-builtins.cpp | 8 ++++----
llvm/lib/Analysis/BasicAliasAnalysis.cpp | 18 ++++++++++++++++++
llvm/lib/Analysis/GlobalsModRef.cpp | 7 ++++++-
3 files changed, 28 insertions(+), 5 deletions(-)
diff --git a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
index 6453d50f044aa..696f3f65236fc 100644
--- a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
+++ b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
@@ -48,9 +48,9 @@ float4 strict_elementwise_min(float4 a, float4 b) {
}
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z26strict_elementwise_maximumDv4_fS_
-// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR2]] {
+// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_MAXIMUM:%.*]] = tail call <4 x float> @llvm.maximum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR4]]
+// CHECK-NEXT: [[ELT_MAXIMUM:%.*]] = tail call <4 x float> @llvm.maximum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR5:[0-9]+]]
// CHECK-NEXT: ret <4 x float> [[ELT_MAXIMUM]]
//
float4 strict_elementwise_maximum(float4 a, float4 b) {
@@ -58,9 +58,9 @@ float4 strict_elementwise_maximum(float4 a, float4 b) {
}
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z26strict_elementwise_minimumDv4_fS_
-// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR2]] {
+// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_MINIMUM:%.*]] = tail call <4 x float> @llvm.minimum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR4]]
+// CHECK-NEXT: [[ELT_MINIMUM:%.*]] = tail call <4 x float> @llvm.minimum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR5]]
// CHECK-NEXT: ret <4 x float> [[ELT_MINIMUM]]
//
float4 strict_elementwise_minimum(float4 a, float4 b) {
diff --git a/llvm/lib/Analysis/BasicAliasAnalysis.cpp b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
index 64e035436a414..81d4aa0d4c6e3 100644
--- a/llvm/lib/Analysis/BasicAliasAnalysis.cpp
+++ b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
@@ -837,6 +837,14 @@ MemoryEffects BasicAAResult::getMemoryEffects(const CallBase *Call,
if (const Function *F = dyn_cast<Function>(Call->getCalledOperand())) {
MemoryEffects FuncME = AAQI.AAR.getMemoryEffects(F);
+
+ // Floating-point operations have memory effects that describe interaction
+ // with the floating-point environment. These memory effects depend on
+ // the attributes of the containing function.
+ if (auto FPME = Call->getFloatingPointMemoryEffects())
+ FuncME = FuncME.getWithModRef(IRMemLocation::InaccessibleMem,
+ FPME->getModRef(IRMemLocation::InaccessibleMem));
+
// Operand bundles on the call may also read or write memory, in addition
// to the behavior of the called function.
if (Call->hasReadingOperandBundles())
@@ -863,6 +871,16 @@ MemoryEffects BasicAAResult::getMemoryEffects(const Function *F) {
// inaccessible memory to model control dependence.
return MemoryEffects::readOnly() |
MemoryEffects::inaccessibleMemOnly(ModRefInfo::ModRef);
+#define FUNCTION(NAME,R,D) case Intrinsic::NAME:
+#include "llvm/IR/FloatingPointOps.def"
+ // Floating-point operations may have or may not have side effects due to
+ // the interaction with floating-point environment. Which case is realized,
+ // it depends on the corresponding call site bundles and attribute of the
+ // containing function. Here we conservatively assume that in stricfp
+ // function the side effects exist.
+ if (F->getAttributes().hasFnAttr(llvm::Attribute::StrictFP))
+ return MemoryEffects::inaccessibleMemOnly(ModRefInfo::ModRef);
+ return MemoryEffects::none();
}
return F->getMemoryEffects();
diff --git a/llvm/lib/Analysis/GlobalsModRef.cpp b/llvm/lib/Analysis/GlobalsModRef.cpp
index 295e267848b23..0f639126c7e0a 100644
--- a/llvm/lib/Analysis/GlobalsModRef.cpp
+++ b/llvm/lib/Analysis/GlobalsModRef.cpp
@@ -537,7 +537,12 @@ void GlobalsAAResult::AnalyzeCallGraph(CallGraph &CG, Module &M) {
if (F->isDeclaration() || F->hasOptNone()) {
// Try to get mod/ref behaviour from function attributes.
- if (F->doesNotAccessMemory()) {
+ if (F->isFPOperation()) {
+ // Floating-point operations have mod/ref behaviour that depends on
+ // the call site properties and containing function attributes.
+ // Conservatively assume RW access to the floating-point environment.
+ FI.addModRefInfo(ModRefInfo::ModRef);
+ } else if (F->doesNotAccessMemory()) {
// Can't do better than that!
} else if (F->onlyReadsMemory()) {
FI.addModRefInfo(ModRefInfo::Ref);
>From 057c81e9ecfbd1c080210433772880442e5fc67d Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 26 Feb 2026 22:41:51 +0700
Subject: [PATCH 18/24] Fix setting NoFPExcept flag
---
llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 6612562b3113a..01b831aa23999 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -9630,8 +9630,10 @@ bool SelectionDAGBuilder::visitFPOperation(const CallInst &I, unsigned Opcode) {
if (auto *FPOp = dyn_cast<FPMathOperator>(&I))
Flags.copyFMF(*FPOp);
fp::ExceptionBehavior EB = I.getExceptionBehavior();
- if (EB == fp::ExceptionBehavior::ebIgnore)
+ if (DAG.getMachineFunction().getFunction().getAttributes().hasFnAttr(
+ llvm::Attribute::StrictFP) && EB == fp::ExceptionBehavior::ebIgnore) {
Flags.setNoFPExcept(true);
+ }
// Temporary solution: use STRICT_* nodes.
if (HasChain)
>From 953199e0b09e804f553b608d4dec93fcab7438bc Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Tue, 10 Mar 2026 19:47:00 +0700
Subject: [PATCH 19/24] Restore test
---
clang/test/CodeGen/strictfp-elementwise-builtins.cpp | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
index 696f3f65236fc..6453d50f044aa 100644
--- a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
+++ b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
@@ -48,9 +48,9 @@ float4 strict_elementwise_min(float4 a, float4 b) {
}
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z26strict_elementwise_maximumDv4_fS_
-// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR2]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_MAXIMUM:%.*]] = tail call <4 x float> @llvm.maximum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR5:[0-9]+]]
+// CHECK-NEXT: [[ELT_MAXIMUM:%.*]] = tail call <4 x float> @llvm.maximum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR4]]
// CHECK-NEXT: ret <4 x float> [[ELT_MAXIMUM]]
//
float4 strict_elementwise_maximum(float4 a, float4 b) {
@@ -58,9 +58,9 @@ float4 strict_elementwise_maximum(float4 a, float4 b) {
}
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z26strict_elementwise_minimumDv4_fS_
-// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR2]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_MINIMUM:%.*]] = tail call <4 x float> @llvm.minimum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR5]]
+// CHECK-NEXT: [[ELT_MINIMUM:%.*]] = tail call <4 x float> @llvm.minimum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR4]]
// CHECK-NEXT: ret <4 x float> [[ELT_MINIMUM]]
//
float4 strict_elementwise_minimum(float4 a, float4 b) {
>From e1e67758aaa1754a0c107d236f2a28905595f6b0 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Fri, 13 Mar 2026 16:05:46 +0700
Subject: [PATCH 20/24] Fix format errors
---
llvm/lib/Analysis/BasicAliasAnalysis.cpp | 7 ++++---
llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp | 3 ++-
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/llvm/lib/Analysis/BasicAliasAnalysis.cpp b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
index 81d4aa0d4c6e3..907efe0c8cce0 100644
--- a/llvm/lib/Analysis/BasicAliasAnalysis.cpp
+++ b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
@@ -842,8 +842,9 @@ MemoryEffects BasicAAResult::getMemoryEffects(const CallBase *Call,
// with the floating-point environment. These memory effects depend on
// the attributes of the containing function.
if (auto FPME = Call->getFloatingPointMemoryEffects())
- FuncME = FuncME.getWithModRef(IRMemLocation::InaccessibleMem,
- FPME->getModRef(IRMemLocation::InaccessibleMem));
+ FuncME =
+ FuncME.getWithModRef(IRMemLocation::InaccessibleMem,
+ FPME->getModRef(IRMemLocation::InaccessibleMem));
// Operand bundles on the call may also read or write memory, in addition
// to the behavior of the called function.
@@ -871,7 +872,7 @@ MemoryEffects BasicAAResult::getMemoryEffects(const Function *F) {
// inaccessible memory to model control dependence.
return MemoryEffects::readOnly() |
MemoryEffects::inaccessibleMemOnly(ModRefInfo::ModRef);
-#define FUNCTION(NAME,R,D) case Intrinsic::NAME:
+#define FUNCTION(NAME, R, D) case Intrinsic::NAME:
#include "llvm/IR/FloatingPointOps.def"
// Floating-point operations may have or may not have side effects due to
// the interaction with floating-point environment. Which case is realized,
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 01b831aa23999..1fdf372393d13 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -9631,7 +9631,8 @@ bool SelectionDAGBuilder::visitFPOperation(const CallInst &I, unsigned Opcode) {
Flags.copyFMF(*FPOp);
fp::ExceptionBehavior EB = I.getExceptionBehavior();
if (DAG.getMachineFunction().getFunction().getAttributes().hasFnAttr(
- llvm::Attribute::StrictFP) && EB == fp::ExceptionBehavior::ebIgnore) {
+ llvm::Attribute::StrictFP) &&
+ EB == fp::ExceptionBehavior::ebIgnore) {
Flags.setNoFPExcept(true);
}
>From d2486793ce66ab8ac0d0f582ad3d83885fe3a4da Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Wed, 3 Dec 2025 16:01:46 +0700
Subject: [PATCH 21/24] Allow more intrinsics to have FP bundles
With this change all intrinsics that have constrained counterparts are
allowed to have floating-point operand bundles as well.
---
.../CodeGen/strictfp-elementwise-builtins.cpp | 8 +-
llvm/include/llvm/IR/FloatingPointOps.def | 47 ++++++-
.../SelectionDAG/SelectionDAGBuilder.cpp | 118 ++----------------
.../AMDGPU/amdgpu-simplify-libcall-pow.ll | 6 +-
.../AMDGPU/amdgpu-simplify-libcall-pown.ll | 4 +-
.../AMDGPU/amdgpu-simplify-libcall-rootn.ll | 51 ++++----
6 files changed, 92 insertions(+), 142 deletions(-)
diff --git a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
index 6453d50f044aa..696f3f65236fc 100644
--- a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
+++ b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
@@ -48,9 +48,9 @@ float4 strict_elementwise_min(float4 a, float4 b) {
}
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z26strict_elementwise_maximumDv4_fS_
-// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR2]] {
+// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_MAXIMUM:%.*]] = tail call <4 x float> @llvm.maximum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR4]]
+// CHECK-NEXT: [[ELT_MAXIMUM:%.*]] = tail call <4 x float> @llvm.maximum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR5:[0-9]+]]
// CHECK-NEXT: ret <4 x float> [[ELT_MAXIMUM]]
//
float4 strict_elementwise_maximum(float4 a, float4 b) {
@@ -58,9 +58,9 @@ float4 strict_elementwise_maximum(float4 a, float4 b) {
}
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z26strict_elementwise_minimumDv4_fS_
-// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR2]] {
+// CHECK-SAME: (<4 x float> noundef [[A:%.*]], <4 x float> noundef [[B:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_MINIMUM:%.*]] = tail call <4 x float> @llvm.minimum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR4]]
+// CHECK-NEXT: [[ELT_MINIMUM:%.*]] = tail call <4 x float> @llvm.minimum.v4f32(<4 x float> [[A]], <4 x float> [[B]]) #[[ATTR5]]
// CHECK-NEXT: ret <4 x float> [[ELT_MINIMUM]]
//
float4 strict_elementwise_minimum(float4 a, float4 b) {
diff --git a/llvm/include/llvm/IR/FloatingPointOps.def b/llvm/include/llvm/IR/FloatingPointOps.def
index 67a66a84223b4..e533689bd995e 100644
--- a/llvm/include/llvm/IR/FloatingPointOps.def
+++ b/llvm/include/llvm/IR/FloatingPointOps.def
@@ -19,13 +19,54 @@
#define FUNCTION(N,R,D)
#endif
-// Describes floating-point operation, which lowers to STRICT_* nodes in DAG.
+// Describes a floating-point operation that is lowered to DAG nodes STRICT_*
+// in strictfp functions.
#ifndef LEGACY_DAG
#define LEGACY_DAG(N,R,D) FUNCTION(N,R,D)
#endif
-LEGACY_DAG(nearbyint, 1, FNEARBYINT)
-LEGACY_DAG(trunc, 0, FTRUNC)
+// Describes a floating-point operation that is lowered to DAG nodes STRICT_*
+// in strictfp functions, and requires special expansion in the DAG builder.
+#ifndef LEGACY_EXP
+#define LEGACY_EXP(N,R,D) FUNCTION(N,R,D)
+#endif
+
+LEGACY_DAG(nearbyint, 1, FNEARBYINT)
+LEGACY_DAG(trunc, 0, FTRUNC)
+LEGACY_DAG(ceil, 0, FCEIL)
+LEGACY_DAG(floor, 0, FFLOOR)
+LEGACY_DAG(round, 0, FROUND)
+LEGACY_DAG(roundeven, 0, FROUNDEVEN)
+LEGACY_DAG(rint, 1, FRINT)
+LEGACY_DAG(lround, 0, LROUND)
+LEGACY_DAG(llround, 0, LLROUND)
+LEGACY_DAG(lrint, 1, LRINT)
+LEGACY_DAG(llrint, 1, LLRINT)
+LEGACY_DAG(minnum, 0, FMINNUM)
+LEGACY_DAG(maxnum, 0, FMAXNUM)
+LEGACY_DAG(minimum, 0, FMINIMUM)
+LEGACY_DAG(maximum, 0, FMAXIMUM)
+LEGACY_DAG(sqrt, 1, FSQRT)
+LEGACY_EXP(exp, 1, FEXP)
+LEGACY_EXP(exp2, 1, FEXP2)
+LEGACY_EXP(log, 1, FLOG)
+LEGACY_EXP(log2, 1, FLOG2)
+LEGACY_EXP(log10, 1, FLOG10)
+LEGACY_EXP(pow, 1, FPOW)
+LEGACY_EXP(powi, 1, FPOWI)
+LEGACY_DAG(sin, 1, FSIN)
+LEGACY_DAG(cos, 1, FCOS)
+LEGACY_DAG(tan, 1, FTAN)
+LEGACY_DAG(sinh, 1, FSINH)
+LEGACY_DAG(cosh, 1, FCOSH)
+LEGACY_DAG(tanh, 1, FTANH)
+LEGACY_DAG(asin, 1, FASIN)
+LEGACY_DAG(acos, 1, FACOS)
+LEGACY_DAG(atan, 1, FATAN)
+LEGACY_DAG(atan2, 1, FATAN2)
+LEGACY_DAG(ldexp, 1, FLDEXP)
+LEGACY_DAG(fma, 1, FMA)
#undef FUNCTION
#undef LEGACY_DAG
+#undef LEGACY_EXP
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 1fdf372393d13..4e42683afec6f 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -6905,107 +6905,24 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
setValue(&I,
expandExp2(sdl, getValue(I.getArgOperand(0)), DAG, TLI, Flags));
return;
+ case Intrinsic::exp10:
+ setValue(&I, DAG.getNode(ISD::FEXP10, sdl,
+ getValue(I.getArgOperand(0)).getValueType(),
+ getValue(I.getArgOperand(0)), Flags));
+ return;
case Intrinsic::pow:
setValue(&I, expandPow(sdl, getValue(I.getArgOperand(0)),
getValue(I.getArgOperand(1)), DAG, TLI, Flags));
return;
- case Intrinsic::sqrt:
case Intrinsic::fabs:
- case Intrinsic::sin:
- case Intrinsic::cos:
- case Intrinsic::tan:
- case Intrinsic::asin:
- case Intrinsic::acos:
- case Intrinsic::atan:
- case Intrinsic::sinh:
- case Intrinsic::cosh:
- case Intrinsic::tanh:
- case Intrinsic::exp10:
- case Intrinsic::floor:
- case Intrinsic::ceil:
- case Intrinsic::rint:
- case Intrinsic::round:
- case Intrinsic::roundeven:
- case Intrinsic::canonicalize: {
- unsigned Opcode;
- // clang-format off
- switch (Intrinsic) {
- default: llvm_unreachable("Impossible intrinsic"); // Can't reach here.
- case Intrinsic::sqrt: Opcode = ISD::FSQRT; break;
- case Intrinsic::fabs: Opcode = ISD::FABS; break;
- case Intrinsic::sin: Opcode = ISD::FSIN; break;
- case Intrinsic::cos: Opcode = ISD::FCOS; break;
- case Intrinsic::tan: Opcode = ISD::FTAN; break;
- case Intrinsic::asin: Opcode = ISD::FASIN; break;
- case Intrinsic::acos: Opcode = ISD::FACOS; break;
- case Intrinsic::atan: Opcode = ISD::FATAN; break;
- case Intrinsic::sinh: Opcode = ISD::FSINH; break;
- case Intrinsic::cosh: Opcode = ISD::FCOSH; break;
- case Intrinsic::tanh: Opcode = ISD::FTANH; break;
- case Intrinsic::exp10: Opcode = ISD::FEXP10; break;
- case Intrinsic::floor: Opcode = ISD::FFLOOR; break;
- case Intrinsic::ceil: Opcode = ISD::FCEIL; break;
- case Intrinsic::rint: Opcode = ISD::FRINT; break;
- case Intrinsic::round: Opcode = ISD::FROUND; break;
- case Intrinsic::roundeven: Opcode = ISD::FROUNDEVEN; break;
- case Intrinsic::canonicalize: Opcode = ISD::FCANONICALIZE; break;
- }
- // clang-format on
-
- setValue(&I, DAG.getNode(Opcode, sdl,
+ setValue(&I, DAG.getNode(ISD::FABS, sdl,
getValue(I.getArgOperand(0)).getValueType(),
getValue(I.getArgOperand(0)), Flags));
return;
- }
- case Intrinsic::atan2:
- setValue(&I, DAG.getNode(ISD::FATAN2, sdl,
- getValue(I.getArgOperand(0)).getValueType(),
- getValue(I.getArgOperand(0)),
- getValue(I.getArgOperand(1)), Flags));
- return;
- case Intrinsic::lround:
- case Intrinsic::llround:
- case Intrinsic::lrint:
- case Intrinsic::llrint: {
- unsigned Opcode;
- // clang-format off
- switch (Intrinsic) {
- default: llvm_unreachable("Impossible intrinsic"); // Can't reach here.
- case Intrinsic::lround: Opcode = ISD::LROUND; break;
- case Intrinsic::llround: Opcode = ISD::LLROUND; break;
- case Intrinsic::lrint: Opcode = ISD::LRINT; break;
- case Intrinsic::llrint: Opcode = ISD::LLRINT; break;
- }
- // clang-format on
-
- EVT RetVT = TLI.getValueType(DAG.getDataLayout(), I.getType());
- setValue(&I, DAG.getNode(Opcode, sdl, RetVT,
- getValue(I.getArgOperand(0))));
- return;
- }
- case Intrinsic::minnum:
- setValue(&I, DAG.getNode(ISD::FMINNUM, sdl,
- getValue(I.getArgOperand(0)).getValueType(),
- getValue(I.getArgOperand(0)),
- getValue(I.getArgOperand(1)), Flags));
- return;
- case Intrinsic::maxnum:
- setValue(&I, DAG.getNode(ISD::FMAXNUM, sdl,
- getValue(I.getArgOperand(0)).getValueType(),
- getValue(I.getArgOperand(0)),
- getValue(I.getArgOperand(1)), Flags));
- return;
- case Intrinsic::minimum:
- setValue(&I, DAG.getNode(ISD::FMINIMUM, sdl,
- getValue(I.getArgOperand(0)).getValueType(),
- getValue(I.getArgOperand(0)),
- getValue(I.getArgOperand(1)), Flags));
- return;
- case Intrinsic::maximum:
- setValue(&I, DAG.getNode(ISD::FMAXIMUM, sdl,
+ case Intrinsic::canonicalize:
+ setValue(&I, DAG.getNode(ISD::FCANONICALIZE, sdl,
getValue(I.getArgOperand(0)).getValueType(),
- getValue(I.getArgOperand(0)),
- getValue(I.getArgOperand(1)), Flags));
+ getValue(I.getArgOperand(0)), Flags));
return;
case Intrinsic::minimumnum:
setValue(&I, DAG.getNode(ISD::FMINIMUMNUM, sdl,
@@ -7025,12 +6942,6 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
getValue(I.getArgOperand(0)),
getValue(I.getArgOperand(1)), Flags));
return;
- case Intrinsic::ldexp:
- setValue(&I, DAG.getNode(ISD::FLDEXP, sdl,
- getValue(I.getArgOperand(0)).getValueType(),
- getValue(I.getArgOperand(0)),
- getValue(I.getArgOperand(1)), Flags));
- return;
case Intrinsic::modf:
case Intrinsic::sincos:
case Intrinsic::sincospi:
@@ -7065,12 +6976,6 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
getValue(I.getArgOperand(0)), Flags));
return;
}
- case Intrinsic::fma:
- setValue(&I, DAG.getNode(
- ISD::FMA, sdl, getValue(I.getArgOperand(0)).getValueType(),
- getValue(I.getArgOperand(0)), getValue(I.getArgOperand(1)),
- getValue(I.getArgOperand(2)), Flags));
- return;
#define INSTRUCTION(NAME, NARG, ROUND_MODE, INTRINSIC) \
case Intrinsic::INTRINSIC:
#include "llvm/IR/ConstrainedOps.def"
@@ -7080,10 +6985,11 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
#include "llvm/IR/VPIntrinsics.def"
visitVectorPredicationIntrinsic(cast<VPIntrinsic>(I));
return;
-#define FUNCTION(NAME, RM, DAGN) \
+#define LEGACY_EXP(NAME, R, DAGN)
+#define FUNCTION(NAME, R, DAGN) \
case Intrinsic::NAME: \
visitFPOperation(I, ISD::DAGN); \
- break;
+ return;
#include "llvm/IR/FloatingPointOps.def"
case Intrinsic::fptrunc_round: {
// Get the last argument, the metadata and convert it to an integer in the
diff --git a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll
index 662dbb67d7b31..77630846f49db 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow.ll
@@ -1208,10 +1208,10 @@ define float @test_pow_afn_f32_strictfp(float %x, float %y) #2 {
; NOPRELINK-NEXT: [[TMP3:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f32(float [[TMP2]], float 0.000000e+00, metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP4:%.*]] = select nnan nsz afn i1 [[TMP3]], float 1.000000e+00, float [[X]]
; NOPRELINK-NEXT: [[TMP5:%.*]] = call nnan nsz afn float @llvm.fabs.f32(float [[TMP4]]) #[[ATTR3]]
-; NOPRELINK-NEXT: [[TMP6:%.*]] = call nnan nsz afn float @llvm.log2.f32(float [[TMP5]]) #[[ATTR3]]
+; NOPRELINK-NEXT: [[TMP6:%.*]] = call nnan nsz afn float @llvm.log2.f32(float [[TMP5]]) #[[ATTR5:[0-9]+]]
; NOPRELINK-NEXT: [[TMP7:%.*]] = call nnan nsz afn float @llvm.experimental.constrained.fmul.f32(float [[TMP2]], float [[TMP6]], metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR3]]
-; NOPRELINK-NEXT: [[TMP8:%.*]] = call nnan nsz afn float @llvm.exp2.f32(float [[TMP7]]) #[[ATTR3]]
-; NOPRELINK-NEXT: [[TMP9:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP2]]) #[[ATTR5:[0-9]+]]
+; NOPRELINK-NEXT: [[TMP8:%.*]] = call nnan nsz afn float @llvm.exp2.f32(float [[TMP7]]) #[[ATTR5]]
+; NOPRELINK-NEXT: [[TMP9:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP2]]) #[[ATTR5]]
; NOPRELINK-NEXT: [[TMP10:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f32(float [[TMP9]], float [[TMP2]], metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP11:%.*]] = call nnan nsz afn float @llvm.experimental.constrained.fmul.f32(float [[TMP2]], float 5.000000e-01, metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR3]]
; NOPRELINK-NEXT: [[TMP12:%.*]] = call nnan nsz afn float @llvm.trunc.f32(float [[TMP11]]) #[[ATTR5]]
diff --git a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pown.ll b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pown.ll
index b4182ccbf77a4..935ebc6dc41bf 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pown.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pown.ll
@@ -883,10 +883,10 @@ define float @test_pown_fast_f32_strictfp(float %x, i32 %y) #1 {
; CHECK-SAME: (float [[X:%.*]], i32 [[Y:%.*]]) #[[ATTR0:[0-9]+]] {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[__FABS:%.*]] = call fast float @llvm.fabs.f32(float [[X]]) #[[ATTR0]]
-; CHECK-NEXT: [[__LOG2:%.*]] = call fast float @llvm.log2.f32(float [[__FABS]]) #[[ATTR0]]
+; CHECK-NEXT: [[__LOG2:%.*]] = call fast float @llvm.log2.f32(float [[__FABS]]) #[[ATTR5:[0-9]+]]
; CHECK-NEXT: [[POWNI2F:%.*]] = call fast float @llvm.experimental.constrained.sitofp.f32.i32(i32 [[Y]], metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR0]]
; CHECK-NEXT: [[__YLOGX:%.*]] = call fast float @llvm.experimental.constrained.fmul.f32(float [[POWNI2F]], float [[__LOG2]], metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR0]]
-; CHECK-NEXT: [[__EXP2:%.*]] = call fast nofpclass(nan ninf nzero nsub nnorm) float @llvm.exp2.f32(float [[__YLOGX]]) #[[ATTR0]]
+; CHECK-NEXT: [[__EXP2:%.*]] = call fast nofpclass(nan ninf nzero nsub nnorm) float @llvm.exp2.f32(float [[__YLOGX]]) #[[ATTR5]]
; CHECK-NEXT: [[__YEVEN:%.*]] = shl i32 [[Y]], 31
; CHECK-NEXT: [[TMP0:%.*]] = bitcast float [[X]] to i32
; CHECK-NEXT: [[__POW_SIGN:%.*]] = and i32 [[__YEVEN]], [[TMP0]]
diff --git a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-rootn.ll b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-rootn.ll
index 00b4583494c75..1e67c323a7e32 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-rootn.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-rootn.ll
@@ -271,7 +271,7 @@ define half @test_rootn_f16_1(half %x) {
define half @test_rootn_f16_2(half %x) {
; CHECK-LABEL: define half @test_rootn_f16_2(
; CHECK-SAME: half [[X:%.*]]) {
-; CHECK-NEXT: [[CALL:%.*]] = call half @llvm.sqrt.f16(half [[X]]), !fpmath [[META0:![0-9]+]]
+; CHECK-NEXT: [[CALL:%.*]] = call half @llvm.sqrt.f16(half [[X]]), !fpmath [[META1:![0-9]+]]
; CHECK-NEXT: ret half [[CALL]]
;
%call = tail call half @_Z5rootnDhi(half %x, i32 2)
@@ -307,7 +307,7 @@ define half @test_rootn_f16_neg2(half %x) {
; CHECK-LABEL: define half @test_rootn_f16_neg2(
; CHECK-SAME: half [[X:%.*]]) {
; CHECK-NEXT: [[TMP1:%.*]] = call contract half @llvm.sqrt.f16(half [[X]])
-; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract half 0xH3C00, [[TMP1]], !fpmath [[META0]]
+; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract half 0xH3C00, [[TMP1]], !fpmath [[META1]]
; CHECK-NEXT: ret half [[__ROOTN2RSQRT]]
;
%call = tail call half @_Z5rootnDhi(half %x, i32 -2)
@@ -356,7 +356,7 @@ define <2 x half> @test_rootn_v2f16_1(<2 x half> %x) {
define <2 x half> @test_rootn_v2f16_2(<2 x half> %x) {
; CHECK-LABEL: define <2 x half> @test_rootn_v2f16_2(
; CHECK-SAME: <2 x half> [[X:%.*]]) {
-; CHECK-NEXT: [[CALL:%.*]] = call <2 x half> @llvm.sqrt.v2f16(<2 x half> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call <2 x half> @llvm.sqrt.v2f16(<2 x half> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <2 x half> [[CALL]]
;
%call = tail call <2 x half> @_Z5rootnDv2_DhDv2_i(<2 x half> %x, <2 x i32> <i32 2, i32 2>)
@@ -377,7 +377,7 @@ define <2 x half> @test_rootn_v2f16_neg2(<2 x half> %x) {
; CHECK-LABEL: define <2 x half> @test_rootn_v2f16_neg2(
; CHECK-SAME: <2 x half> [[X:%.*]]) {
; CHECK-NEXT: [[TMP1:%.*]] = call contract <2 x half> @llvm.sqrt.v2f16(<2 x half> [[X]])
-; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract <2 x half> splat (half 0xH3C00), [[TMP1]], !fpmath [[META0]]
+; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract <2 x half> splat (half 0xH3C00), [[TMP1]], !fpmath [[META1]]
; CHECK-NEXT: ret <2 x half> [[__ROOTN2RSQRT]]
;
%call = tail call <2 x half> @_Z5rootnDv2_DhDv2_i(<2 x half> %x, <2 x i32> <i32 -2, i32 -2>)
@@ -618,7 +618,7 @@ define float @test_rootn_f32__y_2(float %x) {
; CHECK-LABEL: define float @test_rootn_f32__y_2(
; CHECK-SAME: float [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call float @llvm.sqrt.f32(float [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call float @llvm.sqrt.f32(float [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret float [[CALL]]
;
entry:
@@ -630,7 +630,7 @@ define float @test_rootn_f32__y_2_flags(float %x) {
; CHECK-LABEL: define float @test_rootn_f32__y_2_flags(
; CHECK-SAME: float [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call nnan nsz float @llvm.sqrt.f32(float [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call nnan nsz float @llvm.sqrt.f32(float [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret float [[CALL]]
;
entry:
@@ -643,7 +643,7 @@ define float @test_rootn_f32__y_2_fpmath_3(float %x) {
; CHECK-LABEL: define float @test_rootn_f32__y_2_fpmath_3(
; CHECK-SAME: float [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call nnan nsz float @llvm.sqrt.f32(float [[X]]), !fpmath [[META1:![0-9]+]]
+; CHECK-NEXT: [[CALL:%.*]] = call nnan nsz float @llvm.sqrt.f32(float [[X]]), !fpmath [[META2:![0-9]+]]
; CHECK-NEXT: ret float [[CALL]]
;
entry:
@@ -655,7 +655,7 @@ define <2 x float> @test_rootn_v2f32__y_2_flags(<2 x float> %x) {
; CHECK-LABEL: define <2 x float> @test_rootn_v2f32__y_2_flags(
; CHECK-SAME: <2 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call nnan nsz <2 x float> @llvm.sqrt.v2f32(<2 x float> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call nnan nsz <2 x float> @llvm.sqrt.v2f32(<2 x float> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <2 x float> [[CALL]]
;
entry:
@@ -667,7 +667,7 @@ define <3 x float> @test_rootn_v3f32__y_2(<3 x float> %x) {
; CHECK-LABEL: define <3 x float> @test_rootn_v3f32__y_2(
; CHECK-SAME: <3 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call <3 x float> @llvm.sqrt.v3f32(<3 x float> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call <3 x float> @llvm.sqrt.v3f32(<3 x float> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <3 x float> [[CALL]]
;
entry:
@@ -679,7 +679,7 @@ define <3 x float> @test_rootn_v3f32__y_2_undef(<3 x float> %x) {
; CHECK-LABEL: define <3 x float> @test_rootn_v3f32__y_2_undef(
; CHECK-SAME: <3 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call <3 x float> @llvm.sqrt.v3f32(<3 x float> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call <3 x float> @llvm.sqrt.v3f32(<3 x float> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <3 x float> [[CALL]]
;
entry:
@@ -691,7 +691,7 @@ define <4 x float> @test_rootn_v4f32__y_2(<4 x float> %x) {
; CHECK-LABEL: define <4 x float> @test_rootn_v4f32__y_2(
; CHECK-SAME: <4 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call <4 x float> @llvm.sqrt.v4f32(<4 x float> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call <4 x float> @llvm.sqrt.v4f32(<4 x float> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <4 x float> [[CALL]]
;
entry:
@@ -703,7 +703,7 @@ define <8 x float> @test_rootn_v8f32__y_2(<8 x float> %x) {
; CHECK-LABEL: define <8 x float> @test_rootn_v8f32__y_2(
; CHECK-SAME: <8 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call <8 x float> @llvm.sqrt.v8f32(<8 x float> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call <8 x float> @llvm.sqrt.v8f32(<8 x float> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <8 x float> [[CALL]]
;
entry:
@@ -715,7 +715,7 @@ define <16 x float> @test_rootn_v16f32__y_2(<16 x float> %x) {
; CHECK-LABEL: define <16 x float> @test_rootn_v16f32__y_2(
; CHECK-SAME: <16 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call <16 x float> @llvm.sqrt.v16f32(<16 x float> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call <16 x float> @llvm.sqrt.v16f32(<16 x float> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <16 x float> [[CALL]]
;
entry:
@@ -775,7 +775,7 @@ define <2 x float> @test_rootn_v2f32__y_nonsplat_2_poison(<2 x float> %x) {
; CHECK-LABEL: define <2 x float> @test_rootn_v2f32__y_nonsplat_2_poison(
; CHECK-SAME: <2 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[CALL:%.*]] = call <2 x float> @llvm.sqrt.v2f32(<2 x float> [[X]]), !fpmath [[META0]]
+; CHECK-NEXT: [[CALL:%.*]] = call <2 x float> @llvm.sqrt.v2f32(<2 x float> [[X]]), !fpmath [[META1]]
; CHECK-NEXT: ret <2 x float> [[CALL]]
;
entry:
@@ -884,7 +884,7 @@ define float @test_rootn_f32__y_neg2(float %x) {
; CHECK-SAME: float [[X:%.*]]) {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = call contract float @llvm.sqrt.f32(float [[X]])
-; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract float 1.000000e+00, [[TMP0]], !fpmath [[META0]]
+; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract float 1.000000e+00, [[TMP0]], !fpmath [[META1]]
; CHECK-NEXT: ret float [[__ROOTN2RSQRT]]
;
entry:
@@ -897,7 +897,7 @@ define float @test_rootn_f32__y_neg2__flags(float %x) {
; CHECK-SAME: float [[X:%.*]]) {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = call nnan nsz contract float @llvm.sqrt.f32(float [[X]])
-; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv nnan nsz contract float 1.000000e+00, [[TMP0]], !fpmath [[META0]]
+; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv nnan nsz contract float 1.000000e+00, [[TMP0]], !fpmath [[META1]]
; CHECK-NEXT: ret float [[__ROOTN2RSQRT]]
;
entry:
@@ -946,7 +946,7 @@ define <2 x float> @test_rootn_v2f32__y_neg2(<2 x float> %x) {
; CHECK-SAME: <2 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = call contract <2 x float> @llvm.sqrt.v2f32(<2 x float> [[X]])
-; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract <2 x float> splat (float 1.000000e+00), [[TMP0]], !fpmath [[META0]]
+; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv contract <2 x float> splat (float 1.000000e+00), [[TMP0]], !fpmath [[META1]]
; CHECK-NEXT: ret <2 x float> [[__ROOTN2RSQRT]]
;
entry:
@@ -959,7 +959,7 @@ define <2 x float> @test_rootn_v2f32__y_neg2__flags(<2 x float> %x) {
; CHECK-SAME: <2 x float> [[X:%.*]]) {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = call nnan nsz contract <2 x float> @llvm.sqrt.v2f32(<2 x float> [[X]])
-; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv nnan nsz contract <2 x float> splat (float 1.000000e+00), [[TMP0]], !fpmath [[META0]]
+; CHECK-NEXT: [[__ROOTN2RSQRT:%.*]] = fdiv nnan nsz contract <2 x float> splat (float 1.000000e+00), [[TMP0]], !fpmath [[META1]]
; CHECK-NEXT: ret <2 x float> [[__ROOTN2RSQRT]]
;
entry:
@@ -1282,9 +1282,9 @@ define float @test_rootn_fast_f32_strictfp(float %x, i32 %y) #1 {
; NOPRELINK-NEXT: [[TMP0:%.*]] = call fast float @llvm.experimental.constrained.sitofp.f32.i32(i32 [[Y]], metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR0]]
; NOPRELINK-NEXT: [[TMP1:%.*]] = call fast float @llvm.amdgcn.rcp.f32(float [[TMP0]]) #[[ATTR0]]
; NOPRELINK-NEXT: [[TMP2:%.*]] = call fast float @llvm.fabs.f32(float [[X]]) #[[ATTR0]]
-; NOPRELINK-NEXT: [[TMP3:%.*]] = call fast float @llvm.log2.f32(float [[TMP2]]) #[[ATTR0]]
+; NOPRELINK-NEXT: [[TMP3:%.*]] = call fast float @llvm.log2.f32(float [[TMP2]]) #[[ATTR5:[0-9]+]]
; NOPRELINK-NEXT: [[TMP4:%.*]] = call fast float @llvm.experimental.constrained.fmul.f32(float [[TMP1]], float [[TMP3]], metadata !"round.dynamic", metadata !"fpexcept.strict") #[[ATTR0]]
-; NOPRELINK-NEXT: [[TMP5:%.*]] = call fast float @llvm.exp2.f32(float [[TMP4]]) #[[ATTR0]]
+; NOPRELINK-NEXT: [[TMP5:%.*]] = call fast float @llvm.exp2.f32(float [[TMP4]]) #[[ATTR5]]
; NOPRELINK-NEXT: [[TMP6:%.*]] = and i32 [[Y]], 1
; NOPRELINK-NEXT: [[DOTNOT:%.*]] = icmp eq i32 [[TMP6]], 0
; NOPRELINK-NEXT: [[TMP7:%.*]] = select fast i1 [[DOTNOT]], float 1.000000e+00, float [[X]]
@@ -1971,10 +1971,13 @@ attributes #2 = { noinline }
; NOPRELINK: attributes #[[ATTR2:[0-9]+]] = { nocallback nofree nosync nounwind strictfp willreturn memory(inaccessiblemem: readwrite) }
; NOPRELINK: attributes #[[ATTR3]] = { noinline }
; NOPRELINK: attributes #[[ATTR4]] = { nobuiltin }
+; NOPRELINK: attributes #[[ATTR5]] = { strictfp memory(inaccessiblemem: readwrite) }
;.
-; PRELINK: [[META0]] = !{float 2.000000e+00}
-; PRELINK: [[META1]] = !{float 3.000000e+00}
+; PRELINK: [[META0:![0-9]+]] = !{i32 1, !"amdgpu-libcall-have-fast-pow", i32 1}
+; PRELINK: [[META1]] = !{float 2.000000e+00}
+; PRELINK: [[META2]] = !{float 3.000000e+00}
;.
-; NOPRELINK: [[META0]] = !{float 2.000000e+00}
-; NOPRELINK: [[META1]] = !{float 3.000000e+00}
+; NOPRELINK: [[META0:![0-9]+]] = !{i32 1, !"amdgpu-libcall-have-fast-pow", i32 1}
+; NOPRELINK: [[META1]] = !{float 2.000000e+00}
+; NOPRELINK: [[META2]] = !{float 3.000000e+00}
;.
>From dcd7ce0e78aee0c94e3caf01e673910ebf2ca87c Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Sat, 14 Mar 2026 19:43:30 +0700
Subject: [PATCH 22/24] Updated documentation
---
llvm/docs/LangRef.rst | 289 ++++++++++++++++++++++++++++++--------
llvm/docs/ReleaseNotes.md | 9 +-
2 files changed, 238 insertions(+), 60 deletions(-)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index a1685b70e8cad..fc3f37cc66f7f 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3288,14 +3288,18 @@ target instructions (see the AArch64 psABI for details).
Floating-point Operand Bundles
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-These operand bundles are used for calls that involve floating-point
-operations and interact with :ref:`floating-point environment <floatenv>` or
-depend on floating-point options, such as rounding mode, denormal modes, etc.
+A floating-point bundle may be used on a call that implements a
+:ref:`floating-point operation <floatop>`. It can specify execution options,
+provide optimization hints or convey any other information about the operation
+and its context that may be useful to the compiler.
-An operand bundle tagged with "fp.round" contains information about the
-rounding mode used for the operation execution (the effective rounding mode).
-This mode is represented by a metadata string value and specifies the mode used
-for the operation evaluation. Possible values are:
+Currently the following tags are supported in floating-point bundles:
+* "fp.round"
+* "fp.except"
+
+An operand bundle tagged with ``"fp.round"`` specifies the rounding mode used
+for the operation's evaluation (the effective rounding mode). This mode is
+represented by a metadata string. Possible values are:
::
@@ -3306,15 +3310,21 @@ for the operation evaluation. Possible values are:
"tonearestaway"
"dynamic"
-Only one value may be specified. If the "fp.round" bundle is absent, and
-the operation depends on rounding mode, the default behavior is to use the value
-from a control register (dynamic rounding). In the specific case of
+Example:
+
+.. code-block:: llvm
+
+ %y = call @llvm.trunc.f64(double%x) [ "fp.round"(metadata !"upward") ]
+
+Only one value with this tag may be specified. If the "fp.round" bundle is
+absent and the operation depends on the rounding mode, the default behavior is
+to use the value from the control register (dynamic rounding). In the
:ref:`default floating-point environment <floatenv>`, the register is assumed to
-set rounding to nearest, ties to even.
+be set to rounding to nearest, ties to even.
-An operand bundle tagged with "fp.except" may be associated with operations
-that can raise floating-point exceptions. It contains a single metadata string
-value, which can have one of the following:
+An operand bundle tagged with "fp.except" may be associated with operations that
+can raise floating-point exceptions. It contains a single metadata string value,
+which can be one of the following:
::
@@ -3322,22 +3332,26 @@ value, which can have one of the following:
"maytrap"
"strict"
-If the argument is "ignore", floating-point exceptions raised by the call are not
-intended to be observed. Optimization transformations may reorder such operations
-or omit them in some cases. For example, if the call result is unused, the call
-may be removed, even if it could raise exceptions. This is the only permitted value
-in the :ref:`default floating-point environment <floatenv>`.
+If the argument is ``"ignore"``, any floating-point exceptions raised by the
+call are not intended to be observed. Optimizers may reorder such operations
+or even eliminate them in some cases. For example, if the call's result is
+unused, the call may be removed entirely, even if it could raise exceptions.
+This is the only permitted value in the
+:ref:`default floating-point environment <floatenv>`.
+
+If the ``"fp.except"`` argument is ``"strict"``, any exceptions observed during
+code execution must be preserved by transformations, including indirect
+observation through traps handling. No new observable floating-point exceptions
+may be introduced. This value may only be used in functions with ``strictfp``
+attribute.
-If the argument of "fp.except" is "strict", all transformations must preserve the
-floating-point exception semantics of the original code. Any exception that would
-have been raised by the original code must also be raised by the transformed code
-unless it can be proven unobservable. No new observable floating-point exceptions
-may be introduced. This value may only be used only in functions with
-``strictfp`` attribute.
+If the ``"fp.except"`` bundle is absent, the containing function has the
+``strictfp`` attribute, and the operation may raise floating-point exceptions,
+``"strict"`` is used as the default exception behavior.
-The value "maytrap" is almost same as "strict", but transformations are not
-required to preserve all exceptions that are implied by the original code. For
-example, exceptions may be potentially hidden by constant folding.
+The value ``"maytrap"`` is almost same as ``"strict"``, but transformations are
+not required to preserve all exceptions that are implied by the original code.
+For example, exceptions may be potentially hidden by constant folding.
.. _moduleasm:
@@ -4085,19 +4099,63 @@ seq\_cst total orderings of other operations that are not marked
Floating-Point Environment
--------------------------
-The default LLVM floating-point environment assumes that traps are disabled and
-status flags are not observable. Therefore, floating-point math operations do
-not have side effects and may be speculated freely. Results assume the
-round-to-nearest rounding mode, and subnormals are assumed to be preserved.
+The execution of an operation on floating-point values is often a more complex
+process than simply evaluating a function of its input arguments. First, it can
+depend on various parameters like rounding mode, denormal behavior, trap masks
+and so on. These are referenced to as "control modes" and are stored in
+floating-point control registers. In addition, the operation may set status bits
+in a status register. Floating-point environment is a collection of registers
+that hold both control modes and status bits.
+
+Interaction with the floating-point environment, including reading control
+modes, writing status bits and trapping, is regarded as side effects. Depending
+on how the side effects are treated, compilation occurs in one of two modes.
+
+In the ``strict mode``, all side effects produced by the floating-point
+operations are taken into account. Modifications to the floating-point
+environment are allowed only in this mode.
+
+In the ``unconstrained mode``, control modes are not modified and status bits
+are not observed. This allows floating-point operations to be considered free
+of side effects, which facilitates code optimizations. An important case of this
+mode is the ``default mode``, in which the control modes have default values:
+rounding mode is "round to nearest, ties to even", traps are disabled, and
+subnormals are assumed to be preserved.
+
+The compilation mode is defined for an entire function and is specified by
+``strictfp`` attribute. If this attribute is set, compilation occurs in strict
+mode. The value of the floating-point environment is specified by function
+attributes (such as :ref:`denormal_fpenv <denormal_fpenv>) and can be modified
+either by intrinsic functions like ``llvm.set_rounding`` or external functions
+like ``fesetround``.
+
+.. _floatop:
+
+Floating-point operations
+-------------------------
-Running LLVM code in an environment where these assumptions are not met
-typically leads to undefined behavior. The ``strictfp`` and
-:ref:`denormal_fpenv <denormal_fpenv>` attributes as well as
-:ref:`Constrained Floating-Point Intrinsics <constrainedfp>` and
-:ref:`floating-point operand bundles<ob_fp> can be
-used to weaken LLVM's assumptions and ensure defined behavior in
-non-default floating-point environments; see their respective
-documentation for details.
+Whether an operation interacts with the floating-point environment depends on
+the operation itself, the attributes of its containing function and
+floating-point bundles specified with its call. Operations that can exhibit such
+interaction, and which may be ignored in the unconstrained mode, are referred to
+as ``floating-point operations``. These are computational operations that
+produce floating-point or integer results, round all results according to the
+value of the floating-point environment, and might signal floating-point
+exceptions.
+
+Some operations on floating-point values are not classified as floating-point
+operations. For instance, ``llvm.copysign``, ``llvm.fabs`` or
+``llvm.is_fpclass`` do not depend on control modes and cannot raise exceptions.
+The operations like ``llvm.set_rounding`` or ``llvm.set_fpenv`` interact with
+the floating-point environment, but they are not computational and their
+interaction cannot be ignored.
+
+In addition to optional side effects, floating-point operations may have
+:ref:`floating-point operand bundles <ob_fp>`. These bundles can specify
+evaluation options (such as rounding mode), optimization hints, or any other
+information that may be useful to the compiler. They can even remove side
+effects from an operation - for example, when a dynamic rounding mode is
+replaced with a static one.
.. _floatnan:
@@ -16601,6 +16659,10 @@ matches a conforming libm implementation.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.powi.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16649,6 +16711,10 @@ function may non-deterministically treat signaling NaNs as quiet NaNs. For
example, `powi(QNaN, 0)` returns `1.0`, and `powi(SNaN, 0)` may
non-deterministically return `1.0` or a NaN.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _t_llvm_sin:
'``llvm.sin.*``' Intrinsic
@@ -16688,6 +16754,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _t_llvm_cos:
'``llvm.cos.*``' Intrinsic
@@ -16727,6 +16797,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.tan.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16764,6 +16838,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.asin.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16801,6 +16879,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.acos.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16838,6 +16920,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.atan.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16875,6 +16961,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.atan2.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16913,6 +17003,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.sinh.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16950,6 +17044,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.cosh.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -16987,6 +17085,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.tanh.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -17024,6 +17126,9 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
'``llvm.sincos.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -17218,6 +17323,10 @@ function may non-deterministically treat signaling NaNs as quiet NaNs. For
example, `pow(QNaN, 0.0)` returns `1.0`, and `pow(SNaN, 0.0)` may
non-deterministically return `1.0` or a NaN.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_exp:
'``llvm.exp.*``' Intrinsic
@@ -17258,6 +17367,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_exp2:
'``llvm.exp2.*``' Intrinsic
@@ -17298,6 +17411,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_exp10:
'``llvm.exp10.*``' Intrinsic
@@ -17380,6 +17497,10 @@ value is returned. If the result underflows a zero with the same sign
is returned. If the result overflows, the result is an infinity with
the same sign.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_frexp:
'``llvm.frexp.*``' Intrinsic
@@ -17476,6 +17597,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_log10:
'``llvm.log10.*``' Intrinsic
@@ -17516,6 +17641,9 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
.. _int_log2:
@@ -17557,6 +17685,10 @@ trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_fma:
'``llvm.fma.*``' Intrinsic
@@ -17596,6 +17728,10 @@ is assumed to not trap or set ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_fabs:
'``llvm.fabs.*``' Intrinsic
@@ -17738,6 +17874,10 @@ which follow :ref:`LLVM's usual signaling NaN behavior <floatnan>` instead.
The ``llvm.minnum`` intrinsic can be refined into ``llvm.minimumnum``, as the
latter exhibits a subset of behaviors of the former.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. warning::
If the intrinsic is used without nsz, not all backends currently respect the
@@ -17804,6 +17944,10 @@ which follow :ref:`LLVM's usual signaling NaN behavior <floatnan>` instead.
The ``llvm.maxnum`` intrinsic can be refined into ``llvm.maximumnum``, as the
latter exhibits a subset of behaviors of the former.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. warning::
If the intrinsic is used without nsz, not all backends currently respect the
@@ -17859,6 +18003,10 @@ If the ``nsz`` flag is specified, ``llvm.maximum`` with one +0.0 and one
``nsz`` semantics, if both operands have the same sign, the result must also
have the same sign.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _i_maximum:
'``llvm.maximum.*``' Intrinsic
@@ -17907,6 +18055,10 @@ If the ``nsz`` flag is specified, ``llvm.maximum`` with one +0.0 and one
``nsz`` semantics, if both operands have the same sign, the result must also
have the same sign.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _i_minimumnum:
'``llvm.minimumnum.*``' Intrinsic
@@ -18092,6 +18244,10 @@ Semantics:
This function returns the same values as the libm ``floor`` functions
would, and handles error conditions in the same way.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_ceil:
'``llvm.ceil.*``' Intrinsic
@@ -18129,6 +18285,9 @@ Semantics:
This function returns the same values as the libm ``ceil`` functions
would, and handles error conditions in the same way.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
.. _int_llvm_trunc:
@@ -18168,6 +18327,10 @@ Semantics:
This function returns the same values as the libm ``trunc`` functions
would, and handles error conditions in the same way.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_rint:
'``llvm.rint.*``' Intrinsic
@@ -18205,11 +18368,13 @@ Semantics:
""""""""""
This function returns the same values as the libm ``rint`` functions
-would, and handles error conditions in the same way. Since LLVM assumes the
-:ref:`default floating-point environment <floatenv>`, the rounding mode is
-assumed to be set to "nearest", so halfway cases are rounded to the even
-integer. Use :ref:`Constrained Floating-Point Intrinsics <constrainedfp>`
-to avoid that assumption.
+would, and handles error conditions in the same way.
+
+In the :ref:`default floating-point environment <floatenv>`, the rounding mode is
+assumed to be "round to nearest, ties to even".
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
.. _int_nearbyint:
@@ -18247,11 +18412,13 @@ Semantics:
""""""""""
This function returns the same values as the libm ``nearbyint``
-functions would, and handles error conditions in the same way. Since LLVM
-assumes the :ref:`default floating-point environment <floatenv>`, the rounding
-mode is assumed to be set to "nearest", so halfway cases are rounded to the even
-integer. Use :ref:`Constrained Floating-Point Intrinsics <constrainedfp>` to
-avoid that assumption.
+functions would, and handles error conditions in the same way.
+
+In the :ref:`default floating-point environment <floatenv>`, the rounding mode is
+assumed to be "round to nearest, ties to even".
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
.. _int_round:
@@ -18291,6 +18458,10 @@ Semantics:
This function returns the same values as the libm ``round``
functions would, and handles error conditions in the same way.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_roundeven:
'``llvm.roundeven.*``' Intrinsic
@@ -18330,6 +18501,9 @@ This function implements IEEE 754 operation ``roundToIntegralTiesToEven``. It
also behaves in the same way as C standard function ``roundeven``, including
that it disregards rounding mode and does not raise floating point exceptions.
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
'``llvm.lround.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -18376,6 +18550,10 @@ would, but without setting errno. If the rounded value is too large to
be stored in the result type, the return value is a non-deterministic
value (equivalent to `freeze poison`).
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
'``llvm.llround.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -18413,6 +18591,10 @@ functions would, but without setting errno. If the rounded value is
too large to be stored in the result type, the return value is a
non-deterministic value (equivalent to `freeze poison`).
+As a :ref:`floating-point operation <floatop>`, this function has side effects
+in a ``strictfp`` function and may be annotated with
+:ref:`floating-point operand bundles <ob_fp>`.
+
.. _int_lrint:
'``llvm.lrint.*``' Intrinsic
@@ -27980,13 +28162,6 @@ Constrained FP intrinsics are used to support non-default rounding modes and
accurately preserve exception behavior without compromising LLVM's ability to
optimize FP code when the default behavior is used.
-If any FP operation in a function is constrained then they all must be
-constrained. This is required for correct LLVM IR. Optimizations that
-move code around can create miscompiles if mixing of constrained and normal
-operations is done. The correct way to mix constrained and less constrained
-operations is to use the rounding mode and exception handling metadata to
-mark constrained intrinsics as having LLVM's default behavior.
-
Each of these intrinsics corresponds to a normal floating-point operation. The
data arguments and the return value are the same as the corresponding FP
operation.
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index b125ca3644cb2..2d577cafa82a8 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -62,9 +62,12 @@ Changes to the LLVM IR
* Removed `llvm.convert.to.fp16` and `llvm.convert.from.fp16`
intrinsics. These are equivalent to `fptrunc` and `fpext` with half
with a bitcast.
-* Floating-point operand bundles have been added.
-* Calls to floating-point intrinsics can have operand bundles "fp.round" and
- "fp.except", which specify effective rounding mode and exception behavior.
+
+* Calls to floating-point intrinsics implicitly acquire side effects if
+ containing function has strictfp attribute.
+
+* Operand bundles "fp.round" and "fp.except" may be used in floating-point
+ intrinsics calls.
* "denormal-fp-math" and "denormal-fp-math-f32" string attributes were
migrated to first-class denormal_fpenv attribute.
>From 6a205ed3d5915934f6409d0cbff9f2b038c25f4c Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Mon, 16 Mar 2026 18:11:23 +0700
Subject: [PATCH 23/24] Fix typo
---
llvm/docs/LangRef.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index fc3f37cc66f7f..acd9576203229 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -4125,7 +4125,7 @@ subnormals are assumed to be preserved.
The compilation mode is defined for an entire function and is specified by
``strictfp`` attribute. If this attribute is set, compilation occurs in strict
mode. The value of the floating-point environment is specified by function
-attributes (such as :ref:`denormal_fpenv <denormal_fpenv>) and can be modified
+attributes (such as :ref:`denormal_fpenv <denormal_fpenv>`) and can be modified
either by intrinsic functions like ``llvm.set_rounding`` or external functions
like ``fesetround``.
>From 6f69e39cd99f5603a8f94534ddd84930a1c2ff62 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Fri, 20 Mar 2026 13:06:51 +0700
Subject: [PATCH 24/24] Remove unnecessary changes
---
llvm/include/llvm/IR/IRBuilder.h | 16 ++--------
llvm/lib/CodeGen/AtomicExpandPass.cpp | 5 +--
llvm/lib/CodeGen/HardwareLoops.cpp | 6 ++--
llvm/lib/IR/IRBuilder.cpp | 5 +--
.../Target/AMDGPU/AMDGPUAtomicOptimizer.cpp | 2 +-
llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp | 2 +-
.../Transforms/Utils/LibCallsShrinkWrap.cpp | 2 +-
llvm/lib/Transforms/Utils/LowerAtomic.cpp | 3 +-
llvm/unittests/IR/IRBuilderTest.cpp | 32 +++----------------
9 files changed, 19 insertions(+), 54 deletions(-)
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index dde38674d0519..b30fe72787e92 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -153,8 +153,8 @@ class IRBuilderBase {
FastMathFlags FMF;
bool IsFPConstrained = false;
- fp::ExceptionBehavior DefaultConstrainedExcept = fp::ebIgnore;
- RoundingMode DefaultConstrainedRounding = RoundingMode::NearestTiesToEven;
+ fp::ExceptionBehavior DefaultConstrainedExcept = fp::ebStrict;
+ RoundingMode DefaultConstrainedRounding = RoundingMode::Dynamic;
ArrayRef<OperandBundleDef> DefaultOperandBundles;
@@ -350,18 +350,6 @@ class IRBuilderBase {
/// by this setting.
void setIsFPConstrained(bool IsCon) { IsFPConstrained = IsCon; }
- /// Enable/Disable code generation for strictfp functions.
- void setFPMode(bool IsStrictFP) {
- if (IsStrictFP) {
- setDefaultConstrainedRounding(RoundingMode::Dynamic);
- setDefaultConstrainedExcept(fp::ebStrict);
- } else {
- setDefaultConstrainedRounding(RoundingMode::NearestTiesToEven);
- setDefaultConstrainedExcept(fp::ebIgnore);
- }
- setIsFPConstrained(IsStrictFP);
- }
-
/// Query for the use of constrained floating point math
bool getIsFPConstrained() { return IsFPConstrained; }
diff --git a/llvm/lib/CodeGen/AtomicExpandPass.cpp b/llvm/lib/CodeGen/AtomicExpandPass.cpp
index cefe2f2f37ab2..341a4618dbb47 100644
--- a/llvm/lib/CodeGen/AtomicExpandPass.cpp
+++ b/llvm/lib/CodeGen/AtomicExpandPass.cpp
@@ -166,7 +166,7 @@ struct ReplacementIRBuilder
SetInsertPoint(I);
this->CollectMetadataToCopy(I, {LLVMContext::MD_pcsections});
if (BB->getParent()->getAttributes().hasFnAttr(Attribute::StrictFP))
- this->setFPMode(true);
+ this->setIsFPConstrained(true);
MMRAMD = I->getMetadata(LLVMContext::MD_mmra);
}
@@ -1746,7 +1746,8 @@ bool AtomicExpandImpl::tryExpandAtomicCmpXchg(AtomicCmpXchgInst *CI) {
bool llvm::expandAtomicRMWToCmpXchg(AtomicRMWInst *AI,
CreateCmpXchgInstFun CreateCmpXchg) {
ReplacementIRBuilder Builder(AI, AI->getDataLayout());
- Builder.setFPMode(AI->getFunction()->hasFnAttribute(Attribute::StrictFP));
+ Builder.setIsFPConstrained(
+ AI->getFunction()->hasFnAttribute(Attribute::StrictFP));
// FIXME: If FP exceptions are observable, we should force them off for the
// loop for the FP atomics.
diff --git a/llvm/lib/CodeGen/HardwareLoops.cpp b/llvm/lib/CodeGen/HardwareLoops.cpp
index d55f20bdf2b71..5c0252614e281 100644
--- a/llvm/lib/CodeGen/HardwareLoops.cpp
+++ b/llvm/lib/CodeGen/HardwareLoops.cpp
@@ -491,7 +491,7 @@ Value *HardwareLoop::InitLoopCount() {
Value* HardwareLoop::InsertIterationSetup(Value *LoopCountInit) {
IRBuilder<> Builder(BeginBB->getTerminator());
if (BeginBB->getParent()->getAttributes().hasFnAttr(Attribute::StrictFP))
- Builder.setFPMode(true);
+ Builder.setIsFPConstrained(true);
Type *Ty = LoopCountInit->getType();
bool UsePhi = UsePHICounter || Opts.ForcePhi;
Intrinsic::ID ID = UseLoopGuard
@@ -525,7 +525,7 @@ void HardwareLoop::InsertLoopDec() {
IRBuilder<> CondBuilder(ExitBranch);
if (ExitBranch->getParent()->getParent()->getAttributes().hasFnAttr(
Attribute::StrictFP))
- CondBuilder.setFPMode(true);
+ CondBuilder.setIsFPConstrained(true);
Value *Ops[] = { LoopDecrement };
Value *NewCond = CondBuilder.CreateIntrinsic(Intrinsic::loop_decrement,
@@ -548,7 +548,7 @@ Instruction* HardwareLoop::InsertLoopRegDec(Value *EltsRem) {
IRBuilder<> CondBuilder(ExitBranch);
if (ExitBranch->getParent()->getParent()->getAttributes().hasFnAttr(
Attribute::StrictFP))
- CondBuilder.setFPMode(true);
+ CondBuilder.setIsFPConstrained(true);
Value *Ops[] = { EltsRem, LoopDecrement };
Value *Call = CondBuilder.CreateIntrinsic(Intrinsic::loop_decrement_reg,
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 30a27c53b3e6a..8b71d76f78229 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -208,11 +208,8 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
AddRounding = DefaultConstrainedRounding != RoundingMode::Dynamic;
AddExceptions = DefaultConstrainedExcept != fp::ebStrict;
} else {
- AddRounding =
- DefaultConstrainedRounding != RoundingMode::NearestTiesToEven;
+ AddRounding = false;
AddExceptions = false;
- assert(DefaultConstrainedExcept == fp::ebIgnore &&
- "FP exception in default mode must be ignored");
}
// The options specified by the specified bundles have higher
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp b/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
index 008dbd93e90ba..784ee36d55c1e 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUAtomicOptimizer.cpp
@@ -649,7 +649,7 @@ void AMDGPUAtomicOptimizerImpl::optimizeAtomic(Instruction &I,
IRBuilder<> B(&I);
if (AtomicRMWInst::isFPOperation(Op)) {
- B.setFPMode(I.getFunction()->hasFnAttribute(Attribute::StrictFP));
+ B.setIsFPConstrained(I.getFunction()->hasFnAttribute(Attribute::StrictFP));
}
// If we are in a pixel shader, because of how we have to mask out helper
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp b/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
index 72786dbba9d41..4f97e5e117bf4 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULibCalls.cpp
@@ -621,7 +621,7 @@ bool AMDGPULibCalls::fold(CallInst *CI) {
IRBuilder<> B(CI);
if (CI->isStrictFP())
- B.setFPMode(true);
+ B.setIsFPConstrained(true);
if (FPMathOperator *FPOp = dyn_cast<FPMathOperator>(CI)) {
// Under unsafe-math, evaluate calls if possible.
diff --git a/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp b/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
index 463c0b308f494..fca09c678eba4 100644
--- a/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
+++ b/llvm/lib/Transforms/Utils/LibCallsShrinkWrap.cpp
@@ -103,7 +103,7 @@ class LibCallsShrinkWrap : public InstVisitor<LibCallsShrinkWrap> {
if (!Arg->getType()->isFloatTy())
V = ConstantFoldCastInstruction(Instruction::FPExt, V, Arg->getType());
if (BBBuilder.GetInsertBlock()->getParent()->hasFnAttribute(Attribute::StrictFP))
- BBBuilder.setFPMode(true);
+ BBBuilder.setIsFPConstrained(true);
return BBBuilder.CreateFCmp(Cmp, Arg, V);
}
diff --git a/llvm/lib/Transforms/Utils/LowerAtomic.cpp b/llvm/lib/Transforms/Utils/LowerAtomic.cpp
index 2c9245fd83796..e8b06415d4062 100644
--- a/llvm/lib/Transforms/Utils/LowerAtomic.cpp
+++ b/llvm/lib/Transforms/Utils/LowerAtomic.cpp
@@ -124,7 +124,8 @@ Value *llvm::buildAtomicRMWValue(AtomicRMWInst::BinOp Op,
bool llvm::lowerAtomicRMWInst(AtomicRMWInst *RMWI) {
IRBuilder<> Builder(RMWI);
- Builder.setFPMode(RMWI->getFunction()->hasFnAttribute(Attribute::StrictFP));
+ Builder.setIsFPConstrained(
+ RMWI->getFunction()->hasFnAttribute(Attribute::StrictFP));
Value *Ptr = RMWI->getPointerOperand();
Value *Val = RMWI->getValOperand();
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index cacbf7ebd272e..6e81712cd73bf 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -330,8 +330,6 @@ TEST_F(IRBuilderTest, ConstrainedFP) {
// See if we get constrained intrinsics instead of non-constrained
// instructions.
- Builder.setDefaultConstrainedRounding(RoundingMode::Dynamic);
- Builder.setDefaultConstrainedExcept(fp::ebStrict);
Builder.setIsFPConstrained(true);
auto Parent = BB->getParent();
Parent->addFnAttr(Attribute::StrictFP);
@@ -500,12 +498,12 @@ TEST_F(IRBuilderTest, ConstrainedFPFunctionCall) {
Function::Create(FTy, Function::ExternalLinkage, "", M.get());
BasicBlock *CalleeBB = BasicBlock::Create(Ctx, "", Callee);
IRBuilder<> CalleeBuilder(CalleeBB);
- CalleeBuilder.setFPMode(true);
+ CalleeBuilder.setIsFPConstrained(true);
CalleeBuilder.setConstrainedFPFunctionAttr();
CalleeBuilder.CreateRetVoid();
// Now call the empty constrained FP function.
- Builder.setFPMode(true);
+ Builder.setIsFPConstrained(true);
Builder.setConstrainedFPFunctionAttr();
CallInst *FCall = Builder.CreateCall(Callee, {});
@@ -616,26 +614,6 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_TRUE(ME.doesNotAccessMemory());
}
- // If the builder object specifies a rounding mode, the resulting call will
- // include the corresponding "fp.round" operand bundle.
- {
- Builder.setDefaultConstrainedRounding(RoundingMode::TowardNegative);
- SmallVector<OperandBundleDef, 1> Bundles;
- Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
- auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
- EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
- EXPECT_EQ(RoundingMode::TowardNegative, I->getRoundingMode());
- RoundingSpec RS = I->getRoundingSpec();
- EXPECT_EQ(RoundingMode::TowardNegative, RS.getEffective());
- EXPECT_TRUE(RS.isStatic());
- EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
- MemoryEffects ME = I->getMemoryEffects();
- EXPECT_TRUE(ME.doesNotAccessMemory());
- Builder.setFPMode(false);
- }
-
// If the builder object specifies a rounding mode but the provided operand
// bundles already contain an "fp.round" bundle, the builder's specified mode
// is ignored.
@@ -655,7 +633,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
- Builder.setFPMode(false);
+ Builder.setIsFPConstrained(false);
}
// If the builder object specifies a non-default rounding mode and the operand
@@ -671,7 +649,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_EQ(Intrinsic::trunc, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
- Builder.setFPMode(false);
+ Builder.setIsFPConstrained(false);
}
// Check the state of a call with "fp.except" bundle only.
@@ -728,7 +706,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
Function *Fn = Intrinsic::getOrInsertDeclaration(
M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
- Builder.setFPMode(true);
+ Builder.setIsFPConstrained(true);
// Check the state of a call without FP bundles.
{
More information about the llvm-commits
mailing list