[clang] [llvm] Minimal support of floating-point operand bundles (PR #135658)
Serge Pavlov via llvm-commits
llvm-commits at lists.llvm.org
Thu Aug 7 08:47:42 PDT 2025
https://github.com/spavloff updated https://github.com/llvm/llvm-project/pull/135658
>From 1cc46ac05c320b23cf1e949ab5e4980f333c57a5 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Mon, 14 Apr 2025 12:51:43 +0700
Subject: [PATCH 1/7] Minimal support of floating-point operand bundles
This is a lite version of https://github.com/llvm/llvm-project/pull/109798,
where code changes are minimized to facilitate discussion about the
implementation. The motivations and ideas behind the new floating-point
operation support are described in that PR and in the discussion
https://discourse.llvm.org/t/rfc-change-of-strict-fp-operation-representation-in-ir/85021.
There are concerns that the proposed changes are too invasive and a new
approach is required to make the transition smoother.
This implementation is essentially a subset of PR109798, where
everything beyond the minimum is removed. It tries to build eventually
the same implementation as that PR but in different steps.
The patch does not attempt to modify the existing implementation based
on the constrained intrinsics. Instead it introduces a new one using
operand bundles. This new implementation initially has very limited
functionality, which latter will be extended and finally can replace the
existing one.
This PR introduces the notion of floating-point operation, this is an
intrinsic, that is listed in the file "FloatingPointOps.def". These have
two additional properties:
1. In the strict environment (a function with strictfp attribute) calls
to these operations acquire side effect, now it is read/write access
to inaccessible memory, just as constrained intrinsics do.
2. Calls to these operations may have floating-point operand bundles.
There are two kinds of such bundles, tagged with "fp.control" and
"fp.except", which are used to carry additional information about
control modes and exception handling. Initially the set of control
modes consists of rounding mode only.
The set of operations enlisted in "FloatingPointOps.def" and in
"ConstrainedOps.def" are completely independent, an intrinsic may be in
one list or in both. The set of floating-point operations is expected to
grow and finally all FP intrinsics will be available in the new
implementation. In this patch set of intrinsics in
"FloatingPointOps.def" is minimum necessary for tests.
---
llvm/docs/LangRef.rst | 51 ++++-
llvm/docs/ReleaseNotes.md | 2 +
llvm/include/llvm/IR/FPEnv.h | 18 ++
llvm/include/llvm/IR/FloatingPointOps.def | 24 +++
llvm/include/llvm/IR/IRBuilder.h | 45 ++--
llvm/include/llvm/IR/InstrTypes.h | 16 ++
llvm/include/llvm/IR/IntrinsicInst.h | 8 +
llvm/include/llvm/IR/LLVMContext.h | 2 +
llvm/include/llvm/Support/ModRef.h | 5 +
llvm/lib/IR/FPEnv.cpp | 64 ++++++
llvm/lib/IR/IRBuilder.cpp | 75 +++++++
llvm/lib/IR/Instructions.cpp | 83 +++++++-
llvm/lib/IR/IntrinsicInst.cpp | 10 +
llvm/lib/IR/LLVMContext.cpp | 14 ++
llvm/lib/IR/Verifier.cpp | 31 ++-
.../Bitcode/operand-bundles-bc-analyzer.ll | 2 +
llvm/test/Verifier/fp-intrinsics.ll | 91 ++++++++
llvm/unittests/IR/IRBuilderTest.cpp | 197 ++++++++++++++++++
18 files changed, 717 insertions(+), 21 deletions(-)
create mode 100644 llvm/include/llvm/IR/FloatingPointOps.def
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 3a3a74f323a26..ba119ef97f765 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3090,6 +3090,51 @@ A "convergencectrl" operand bundle is only valid on a ``convergent`` operation.
When present, the operand bundle must contain exactly one value of token type.
See the :doc:`ConvergentOperations` document for details.
+.. _ob_fp:
+
+Floating-point Operand Bundles
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+These operand bundles are used for calls that involve floating-point
+operations and interact with :ref:`floating-point environment <floatenv>` or
+depend on floating-point options, such as rounding mode, denormal modes, etc.
+There are two kinds of such operand bundles, which represent the value of
+floating-point control modes and the treatment of status bits respectively.
+
+An operand bundle tagged with "fp.control" contains information about the
+control modes used for the operation execution. Operands specified in this
+bundle represent particular options. Currently, only rounding mode is supported.
+It is represented by a metadata string value, which specifies the rounding mode
+to be used for the operation evaluation. Possible values are:
+
+::
+
+ "rtz" - toward zero
+ "rte" - to nearest, ties to even
+ "rtp" - toward positive infinity
+ "rtn" - toward negative infinity
+ "rmm" - to nearest, ties away from zero
+ "dyn" - rounding mode is taken from control register
+
+If "fp.control" is absent, the default rounding rounding mode is taken from the
+control register (dynamic rounding). In the particular case of
+:ref:`default floating-point environment <floatenv>`, it must be rounding to
+nearest, ties to even.
+
+An operand bundle tagged with "fp.except" may be associated with operations
+that can read or write floating-point exception flags. It contains a single
+metadata string value, which can have one of the following values:
+
+::
+
+ "ignore"
+ "strict"
+ "maytrap"
+
+It has the same meaning as the corresponding argument in
+:ref:`constrained intrinsics <constrainedfp>`.
+
+
.. _moduleasm:
Module-Level Inline Assembly
@@ -3831,9 +3876,9 @@ round-to-nearest rounding mode, and subnormals are assumed to be preserved.
Running LLVM code in an environment where these assumptions are not met
typically leads to undefined behavior. The ``strictfp`` and ``denormal-fp-math``
attributes as well as :ref:`Constrained Floating-Point Intrinsics
-<constrainedfp>` can be used to weaken LLVM's assumptions and ensure defined
-behavior in non-default floating-point environments; see their respective
-documentation for details.
+<constrainedfp>` or :ref:`floating-point operand bundles<ob_fp>` can be used to
+weaken LLVM's assumptions and ensure defined behavior in non-default
+floating-point environments; see their respective documentation for details.
.. _floatnan:
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 0c49fc86a4232..4d7fbcfad7f5e 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -56,6 +56,8 @@ Makes programs 10x faster by doing Special New Thing.
Changes to the LLVM IR
----------------------
+* Floating-point operand bundles have been added.
+
Changes to LLVM infrastructure
------------------------------
diff --git a/llvm/include/llvm/IR/FPEnv.h b/llvm/include/llvm/IR/FPEnv.h
index 38395b15c8c09..78a5d6d5b75f5 100644
--- a/llvm/include/llvm/IR/FPEnv.h
+++ b/llvm/include/llvm/IR/FPEnv.h
@@ -49,20 +49,38 @@ enum ExceptionBehavior : uint8_t {
/// metadata.
LLVM_ABI std::optional<RoundingMode> convertStrToRoundingMode(StringRef);
+/// Returns a valid RoundingMode enumerator given a string that is used as
+/// rounding mode specifier in operand bundles.
+std::optional<RoundingMode> convertBundleToRoundingMode(StringRef);
+
/// For any RoundingMode enumerator, returns a string valid as input in
/// constrained intrinsic rounding mode metadata.
LLVM_ABI std::optional<StringRef> convertRoundingModeToStr(RoundingMode);
+/// For any RoundingMode enumerator, returns a string to be used in operand
+/// bundles.
+std::optional<StringRef> convertRoundingModeToBundle(RoundingMode);
+
/// Returns a valid ExceptionBehavior enumerator when given a string
/// valid as input in constrained intrinsic exception behavior metadata.
LLVM_ABI std::optional<fp::ExceptionBehavior>
convertStrToExceptionBehavior(StringRef);
+/// Returns a valid ExceptionBehavior enumerator given a string from the operand
+/// bundle argument.
+std::optional<fp::ExceptionBehavior>
+ convertBundleToExceptionBehavior(StringRef);
+
/// For any ExceptionBehavior enumerator, returns a string valid as
/// input in constrained intrinsic exception behavior metadata.
LLVM_ABI std::optional<StringRef>
convertExceptionBehaviorToStr(fp::ExceptionBehavior);
+/// Return string representing the given exception behavior for use in operand
+/// bundles
+std::optional<StringRef>
+ convertExceptionBehaviorToBundle(fp::ExceptionBehavior);
+
/// Returns true if the exception handling behavior and rounding mode
/// match what is used in the default floating point environment.
inline bool isDefaultFPEnvironment(fp::ExceptionBehavior EB, RoundingMode RM) {
diff --git a/llvm/include/llvm/IR/FloatingPointOps.def b/llvm/include/llvm/IR/FloatingPointOps.def
new file mode 100644
index 0000000000000..8567b5dbac302
--- /dev/null
+++ b/llvm/include/llvm/IR/FloatingPointOps.def
@@ -0,0 +1,24 @@
+//===- llvm/IR/FloatingPointOps.def - FP intrinsics -------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Defines set of intrinsics, which are classified as floating-point operations.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef FUNCTION
+#define FUNCTION(N,D)
+#endif
+
+// Arguments of the entries are:
+// - intrinsic function name,
+// - DAG node corresponding to the intrinsic.
+
+FUNCTION(nearbyint, FNEARBYINT)
+FUNCTION(trunc, FTRUNC)
+
+#undef FUNCTION
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index d106dedf814e5..e9c5bdae51cc6 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -1005,6 +1005,16 @@ class IRBuilderBase {
FMFSource FMFSource = {},
const Twine &Name = "");
+ /// Create a call to intrinsic \p ID with \p Args, mangled using \p Types and
+ /// with operand bundles.
+ /// If \p FMFSource is provided, copy fast-math-flags from that instruction to
+ /// the intrinsic.
+ CallInst *CreateIntrinsic(Intrinsic::ID ID, ArrayRef<Type *> Types,
+ ArrayRef<Value *> Args,
+ ArrayRef<OperandBundleDef> OpBundles,
+ Instruction *FMFSource = nullptr,
+ const Twine &Name = "");
+
/// Create a call to non-overloaded intrinsic \p ID with \p Args. If
/// \p FMFSource is provided, copy fast-math-flags from that instruction to
/// the intrinsic.
@@ -2510,24 +2520,13 @@ class IRBuilderBase {
CallInst *CreateCall(FunctionType *FTy, Value *Callee,
ArrayRef<Value *> Args = {}, const Twine &Name = "",
MDNode *FPMathTag = nullptr) {
- CallInst *CI = CallInst::Create(FTy, Callee, Args, DefaultOperandBundles);
- if (IsFPConstrained)
- setConstrainedFPCallAttr(CI);
- if (isa<FPMathOperator>(CI))
- setFPAttrs(CI, FPMathTag, FMF);
- return Insert(CI, Name);
+ return CreateCall(FTy, Callee, Args, DefaultOperandBundles, Name,
+ FPMathTag);
}
CallInst *CreateCall(FunctionType *FTy, Value *Callee, ArrayRef<Value *> Args,
ArrayRef<OperandBundleDef> OpBundles,
- const Twine &Name = "", MDNode *FPMathTag = nullptr) {
- CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
- if (IsFPConstrained)
- setConstrainedFPCallAttr(CI);
- if (isa<FPMathOperator>(CI))
- setFPAttrs(CI, FPMathTag, FMF);
- return Insert(CI, Name);
- }
+ const Twine &Name = "", MDNode *FPMathTag = nullptr);
CallInst *CreateCall(FunctionCallee Callee, ArrayRef<Value *> Args = {},
const Twine &Name = "", MDNode *FPMathTag = nullptr) {
@@ -2761,6 +2760,24 @@ class IRBuilderBase {
/// assumption on the provided pointer.
LLVM_ABI CallInst *CreateDereferenceableAssumption(Value *PtrValue,
Value *SizeValue);
+
+ /// Create an operand bundle in the provided bundle set to represent given FP
+ /// rounding mode.
+ ///
+ /// If the rounding mode is not defined, adds the default rounding mode,
+ /// stored in this builder object.
+ void
+ createFPRoundingBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<RoundingMode> Rounding = std::nullopt);
+
+ /// Create an operand bundle in the provided bundle set to represent FP
+ /// exception behavior.
+ ///
+ /// If the exception behavior is not defined, adds the default behavior,
+ /// stored in this builder object.
+ void createFPExceptionBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<fp::ExceptionBehavior> Except = std::nullopt);
};
/// This provides a uniform API for creating instructions and inserting
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index 14685abef01e9..918c653d30a74 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -25,6 +25,7 @@
#include "llvm/IR/CallingConv.h"
#include "llvm/IR/DerivedTypes.h"
#include "llvm/IR/FMF.h"
+#include "llvm/IR/FPEnv.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/Instruction.h"
#include "llvm/IR/LLVMContext.h"
@@ -1095,6 +1096,13 @@ template <typename InputTy> class OperandBundleDefT {
using OperandBundleDef = OperandBundleDefT<Value *>;
using ConstOperandBundleDef = OperandBundleDefT<const Value *>;
+void addFPRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding);
+void addFPExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except);
+
//===----------------------------------------------------------------------===//
// CallBase Class
//===----------------------------------------------------------------------===//
@@ -1154,6 +1162,8 @@ class CallBase : public Instruction {
/// number of extra operands.
LLVM_ABI unsigned getNumSubclassExtraOperandsDynamic() const;
+ MemoryEffects getFloatingPointMemoryEffects() const;
+
public:
using Instruction::getContext;
@@ -2164,6 +2174,12 @@ class CallBase : public Instruction {
return false;
}
+ /// Return rounding mode specified by operand bundles.
+ RoundingMode getRoundingMode() const;
+
+ /// Return exception behavior specified by operand bundles.
+ std::optional<fp::ExceptionBehavior> getExceptionBehavior() const;
+
/// Used to keep track of an operand bundle. See the main comment on
/// OperandBundleUser above.
struct BundleOpInfo {
diff --git a/llvm/include/llvm/IR/IntrinsicInst.h b/llvm/include/llvm/IR/IntrinsicInst.h
index 2e1389633d7e4..3c58829ccd8d0 100644
--- a/llvm/include/llvm/IR/IntrinsicInst.h
+++ b/llvm/include/llvm/IR/IntrinsicInst.h
@@ -129,6 +129,14 @@ class IntrinsicInst : public CallInst {
/// course of IR transformations
LLVM_ABI static bool mayLowerToFunctionCall(Intrinsic::ID IID);
+ /// Check if \p ID represents a function that may access FP environment and
+ /// may have FP operand bundles.
+ ///
+ /// Access to FP environment means that in the strict FP environment the
+ /// function has read/write memory effect, which is used to maintain proper
+ /// instructions ordering.
+ static bool isFloatingPointOperation(Intrinsic::ID IID);
+
/// Methods for support type inquiry through isa, cast, and dyn_cast:
static bool classof(const CallInst *I) {
auto *F = dyn_cast_or_null<Function>(I->getCalledOperand());
diff --git a/llvm/include/llvm/IR/LLVMContext.h b/llvm/include/llvm/IR/LLVMContext.h
index 852a3a4e2f638..1ac9dbb197d87 100644
--- a/llvm/include/llvm/IR/LLVMContext.h
+++ b/llvm/include/llvm/IR/LLVMContext.h
@@ -97,6 +97,8 @@ class LLVMContext {
OB_ptrauth = 7, // "ptrauth"
OB_kcfi = 8, // "kcfi"
OB_convergencectrl = 9, // "convergencectrl"
+ OB_fp_control = 10, // "fp.control"
+ OB_fp_except = 11, // "fp.except"
};
/// getMDKindID - Return a unique non-zero ID for the specified metadata kind.
diff --git a/llvm/include/llvm/Support/ModRef.h b/llvm/include/llvm/Support/ModRef.h
index 71f3b5bcb9c2b..578753871a2d4 100644
--- a/llvm/include/llvm/Support/ModRef.h
+++ b/llvm/include/llvm/Support/ModRef.h
@@ -235,6 +235,11 @@ template <typename LocationEnum> class MemoryEffectsBase {
return getWithoutLoc(Location::InaccessibleMem).doesNotAccessMemory();
}
+ /// Whether this function accesses inaccessible memory.
+ bool doesAccessInaccessibleMem() const {
+ return isModOrRefSet(getModRef(Location::InaccessibleMem));
+ }
+
/// Whether this function only (at most) accesses errno memory.
bool onlyAccessesErrnoMem() const {
return getWithoutLoc(Location::ErrnoMem).doesNotAccessMemory();
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index 67f21d3756e93..d48bebeb6d2b7 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -34,6 +34,17 @@ std::optional<RoundingMode> convertStrToRoundingMode(StringRef RoundingArg) {
.Default(std::nullopt);
}
+std::optional<RoundingMode> convertBundleToRoundingMode(StringRef RoundingArg) {
+ return StringSwitch<std::optional<RoundingMode>>(RoundingArg)
+ .Case("dyn", RoundingMode::Dynamic)
+ .Case("rte", RoundingMode::NearestTiesToEven)
+ .Case("rmm", RoundingMode::NearestTiesToAway)
+ .Case("rtn", RoundingMode::TowardNegative)
+ .Case("rtp", RoundingMode::TowardPositive)
+ .Case("rtz", RoundingMode::TowardZero)
+ .Default(std::nullopt);
+}
+
std::optional<StringRef> convertRoundingModeToStr(RoundingMode UseRounding) {
std::optional<StringRef> RoundingStr;
switch (UseRounding) {
@@ -61,6 +72,33 @@ std::optional<StringRef> convertRoundingModeToStr(RoundingMode UseRounding) {
return RoundingStr;
}
+std::optional<StringRef> convertRoundingModeToBundle(RoundingMode UseRounding) {
+ std::optional<StringRef> RoundingStr;
+ switch (UseRounding) {
+ case RoundingMode::Dynamic:
+ RoundingStr = "dyn";
+ break;
+ case RoundingMode::NearestTiesToEven:
+ RoundingStr = "rte";
+ break;
+ case RoundingMode::NearestTiesToAway:
+ RoundingStr = "rmm";
+ break;
+ case RoundingMode::TowardNegative:
+ RoundingStr = "rtn";
+ break;
+ case RoundingMode::TowardPositive:
+ RoundingStr = "rtp";
+ break;
+ case RoundingMode::TowardZero:
+ RoundingStr = "rtz";
+ break;
+ default:
+ break;
+ }
+ return RoundingStr;
+}
+
std::optional<fp::ExceptionBehavior>
convertStrToExceptionBehavior(StringRef ExceptionArg) {
return StringSwitch<std::optional<fp::ExceptionBehavior>>(ExceptionArg)
@@ -70,6 +108,15 @@ convertStrToExceptionBehavior(StringRef ExceptionArg) {
.Default(std::nullopt);
}
+std::optional<fp::ExceptionBehavior>
+convertBundleToExceptionBehavior(StringRef ExceptionArg) {
+ return StringSwitch<std::optional<fp::ExceptionBehavior>>(ExceptionArg)
+ .Case("ignore", fp::ebIgnore)
+ .Case("maytrap", fp::ebMayTrap)
+ .Case("strict", fp::ebStrict)
+ .Default(std::nullopt);
+}
+
std::optional<StringRef>
convertExceptionBehaviorToStr(fp::ExceptionBehavior UseExcept) {
std::optional<StringRef> ExceptStr;
@@ -87,6 +134,23 @@ convertExceptionBehaviorToStr(fp::ExceptionBehavior UseExcept) {
return ExceptStr;
}
+std::optional<StringRef>
+convertExceptionBehaviorToBundle(fp::ExceptionBehavior UseExcept) {
+ std::optional<StringRef> ExceptStr;
+ switch (UseExcept) {
+ case fp::ebStrict:
+ ExceptStr = "strict";
+ break;
+ case fp::ebIgnore:
+ ExceptStr = "ignore";
+ break;
+ case fp::ebMayTrap:
+ ExceptStr = "maytrap";
+ break;
+ }
+ return ExceptStr;
+}
+
Intrinsic::ID getConstrainedIntrinsicID(const Instruction &Instr) {
Intrinsic::ID IID = Intrinsic::not_intrinsic;
switch (Instr.getOpcode()) {
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 49c6dc7f401ad..ddeb7949627c6 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -112,6 +112,67 @@ IRBuilderBase::createCallHelper(Function *Callee, ArrayRef<Value *> Ops,
return CI;
}
+CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
+ ArrayRef<Value *> Args,
+ ArrayRef<OperandBundleDef> OpBundles,
+ const Twine &Name, MDNode *FPMathTag) {
+ assert(std::count_if(OpBundles.begin(), OpBundles.end(),
+ [](const OperandBundleDef &Item) {
+ return Item.getTag() == "fp.control";
+ }) <= 1);
+ assert(std::count_if(OpBundles.begin(), OpBundles.end(),
+ [](const OperandBundleDef &Item) {
+ return Item.getTag() == "fp.except";
+ }) <= 1);
+
+ ArrayRef<OperandBundleDef> ActualBundlesRef = OpBundles;
+ SmallVector<OperandBundleDef, 2> ActualBundles;
+
+ // If the builder is in strictfp mode and has non-default options (like
+ // non-dynamic rounding), add corresponding operand bundle. If such bundle is
+ // already present, assume it overwrites defaults.
+ bool doesTheCallAccessFPEnv = false;
+ if (IsFPConstrained) {
+ if (const auto *Func = dyn_cast<Function>(Callee)) {
+ if (Intrinsic::ID ID = Func->getIntrinsicID()) {
+ if (IntrinsicInst::isFloatingPointOperation(ID)) {
+ doesTheCallAccessFPEnv = true;
+ bool NeedRound = DefaultConstrainedRounding != RoundingMode::Dynamic;
+ bool NeedExcept = DefaultConstrainedExcept != fp::ebStrict;
+ for (const auto &Item : OpBundles) {
+ if (NeedRound && Item.getTag() == "fp.control")
+ NeedRound = false;
+ else if (NeedExcept && Item.getTag() == "fp.except")
+ NeedExcept = false;
+ ActualBundles.push_back(Item);
+ }
+ if (NeedRound)
+ createFPRoundingBundle(ActualBundles, DefaultConstrainedRounding);
+ if (NeedExcept)
+ createFPExceptionBundle(ActualBundles, DefaultConstrainedExcept);
+ ActualBundlesRef = ActualBundles;
+ }
+ }
+ }
+ }
+
+ // If the call accesses FPE, update memory effects accordingly.
+ CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
+ if (doesTheCallAccessFPEnv) {
+ MemoryEffects ME = MemoryEffects::inaccessibleMemOnly();
+ if (CI->getAttributes().hasFnAttr(Attribute::Memory))
+ ME |= CI->getAttributes().getMemoryEffects();
+ auto A = Attribute::getWithMemoryEffects(getContext(), ME);
+ CI->addFnAttr(A);
+ }
+
+ if (IsFPConstrained)
+ setConstrainedFPCallAttr(CI);
+ if (isa<FPMathOperator>(CI))
+ setFPAttrs(CI, FPMathTag, FMF);
+ return Insert(CI, Name);
+}
+
static Value *CreateVScaleMultiple(IRBuilderBase &B, Type *Ty, uint64_t Scale) {
Value *VScale = B.CreateVScale(Ty);
if (Scale == 1)
@@ -1281,6 +1342,20 @@ CallInst *IRBuilderBase::CreateDereferenceableAssumption(Value *PtrValue,
{DereferenceableOpB});
}
+void IRBuilderBase::createFPRoundingBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<RoundingMode> Rounding) {
+ addFPRoundingBundle(Context, Bundles,
+ Rounding.value_or(DefaultConstrainedRounding));
+}
+
+void IRBuilderBase::createFPExceptionBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ std::optional<fp::ExceptionBehavior> Except) {
+ addFPExceptionBundle(Context, Bundles,
+ Except.value_or(DefaultConstrainedExcept));
+}
+
IRBuilderDefaultInserter::~IRBuilderDefaultInserter() = default;
IRBuilderCallbackInserter::~IRBuilderCallbackInserter() = default;
IRBuilderFolder::~IRBuilderFolder() = default;
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index b8963823f1c67..295de574ae4f3 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -618,9 +618,10 @@ bool CallBase::hasReadingOperandBundles() const {
// Implementation note: this is a conservative implementation of operand
// bundle semantics, where *any* non-assume operand bundle (other than
// ptrauth) forces a callsite to be at least readonly.
- return hasOperandBundlesOtherThan({LLVMContext::OB_ptrauth,
- LLVMContext::OB_kcfi,
- LLVMContext::OB_convergencectrl}) &&
+ return hasOperandBundlesOtherThan(
+ {LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
+ LLVMContext::OB_fp_control, LLVMContext::OB_fp_except,
+ LLVMContext::OB_convergencectrl}) &&
getIntrinsicID() != Intrinsic::assume;
}
@@ -628,14 +629,70 @@ bool CallBase::hasClobberingOperandBundles() const {
return hasOperandBundlesOtherThan(
{LLVMContext::OB_deopt, LLVMContext::OB_funclet,
LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
+ LLVMContext::OB_fp_control, LLVMContext::OB_fp_except,
LLVMContext::OB_convergencectrl}) &&
getIntrinsicID() != Intrinsic::assume;
}
+RoundingMode CallBase::getRoundingMode() const {
+ // Try reading rounding mode from FP bundle.
+ std::optional<RoundingMode> RM;
+ if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_control)) {
+ Value *V = RoundingBundle->Inputs.front();
+ Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
+ RM = convertBundleToRoundingMode(cast<MDString>(MD)->getString());
+ }
+ if (RM)
+ return *RM;
+
+ // No FP bundle, try to guess from the current mode.
+ if (getParent())
+ if (auto *F = getFunction(); F)
+ return F->getAttributes().hasFnAttr(Attribute::StrictFP)
+ ? RoundingMode::Dynamic
+ : RoundingMode::NearestTiesToEven;
+
+ // Isolated call. Assume default environment.
+ return RoundingMode::NearestTiesToEven;
+}
+
+std::optional<fp::ExceptionBehavior> CallBase::getExceptionBehavior() const {
+ // Try determining exception behavior from FP bundle.
+ std::optional<fp::ExceptionBehavior> EB;
+ if (auto ExceptionBundle = getOperandBundle(LLVMContext::OB_fp_except)) {
+ Value *V = ExceptionBundle->Inputs.front();
+ Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
+ EB = convertBundleToExceptionBehavior(cast<MDString>(MD)->getString());
+ }
+ if (EB)
+ return *EB;
+
+ // No FP bundle, try to guess from the current mode.
+ if (getParent())
+ if (auto *F = getFunction(); F)
+ return F->getAttributes().hasFnAttr(Attribute::StrictFP) ? fp::ebStrict
+ : fp::ebIgnore;
+
+ // Isolated call. Assume default environment.
+ return fp::ebIgnore;
+}
+
+MemoryEffects CallBase::getFloatingPointMemoryEffects() const {
+ if (Intrinsic::ID IntrID = getIntrinsicID())
+ if (const BasicBlock *BB = getParent())
+ if (const Function *F = BB->getParent())
+ if (F->hasFnAttribute(Attribute::StrictFP))
+ if (IntrinsicInst::isFloatingPointOperation(IntrID)) {
+ return MemoryEffects::inaccessibleMemOnly();
+ }
+ return MemoryEffects::none();
+}
+
MemoryEffects CallBase::getMemoryEffects() const {
MemoryEffects ME = getAttributes().getMemoryEffects();
if (auto *Fn = dyn_cast<Function>(getCalledOperand())) {
MemoryEffects FnME = Fn->getMemoryEffects();
+ FnME |= getFloatingPointMemoryEffects();
if (hasOperandBundles()) {
// TODO: Add a method to get memory effects for operand bundles instead.
if (hasReadingOperandBundles())
@@ -740,6 +797,26 @@ bool CallBase::hasArgumentWithAdditionalReturnCaptureComponents() const {
return false;
}
+void llvm::addFPRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding) {
+ std::optional<StringRef> RndStr = convertRoundingModeToBundle(Rounding);
+ assert(RndStr && "Garbage rounding mode!");
+ auto *RoundingMDS = MDString::get(Ctx, *RndStr);
+ auto *RM = MetadataAsValue::get(Ctx, RoundingMDS);
+ Bundles.emplace_back("fp.control", RM);
+}
+
+void llvm::addFPExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except) {
+ std::optional<StringRef> ExcStr = convertExceptionBehaviorToBundle(Except);
+ assert(ExcStr && "Garbage exception behavior!");
+ auto *ExceptMDS = MDString::get(Ctx, *ExcStr);
+ auto *EB = MetadataAsValue::get(Ctx, ExceptMDS);
+ Bundles.emplace_back("fp.except", EB);
+}
+
//===----------------------------------------------------------------------===//
// CallInst Implementation
//===----------------------------------------------------------------------===//
diff --git a/llvm/lib/IR/IntrinsicInst.cpp b/llvm/lib/IR/IntrinsicInst.cpp
index 23a4d1b5c615e..c00f3bb8fd6d8 100644
--- a/llvm/lib/IR/IntrinsicInst.cpp
+++ b/llvm/lib/IR/IntrinsicInst.cpp
@@ -66,6 +66,16 @@ bool IntrinsicInst::mayLowerToFunctionCall(Intrinsic::ID IID) {
}
}
+bool IntrinsicInst::isFloatingPointOperation(Intrinsic::ID IID) {
+ switch (IID) {
+#define FUNCTION(NAME, D) case Intrinsic::NAME:
+#include "llvm/IR/FloatingPointOps.def"
+ return true;
+ default:
+ return false;
+ }
+}
+
//===----------------------------------------------------------------------===//
/// DbgVariableIntrinsic - This is the common base class for debug info
/// intrinsics for variables.
diff --git a/llvm/lib/IR/LLVMContext.cpp b/llvm/lib/IR/LLVMContext.cpp
index 57532cd491dd6..5008f9a57bc16 100644
--- a/llvm/lib/IR/LLVMContext.cpp
+++ b/llvm/lib/IR/LLVMContext.cpp
@@ -53,6 +53,10 @@ static StringRef knownBundleName(unsigned BundleTagID) {
return "kcfi";
case LLVMContext::OB_convergencectrl:
return "convergencectrl";
+ case LLVMContext::OB_fp_control:
+ return "fp.control";
+ case LLVMContext::OB_fp_except:
+ return "fp.except";
default:
llvm_unreachable("unknown bundle id");
}
@@ -82,6 +86,16 @@ LLVMContext::LLVMContext() : pImpl(new LLVMContextImpl(*this)) {
assert(Entry->second == BundleTagID && "operand bundle id drifted!");
}
+ auto *RoundingEntry = pImpl->getOrInsertBundleTag("fp.control");
+ assert(RoundingEntry->second == LLVMContext::OB_fp_control &&
+ "fp.control operand bundle id drifted!");
+ (void)RoundingEntry;
+
+ auto *ExceptionEntry = pImpl->getOrInsertBundleTag("fp.except");
+ assert(ExceptionEntry->second == LLVMContext::OB_fp_except &&
+ "fp.except operand bundle id drifted!");
+ (void)ExceptionEntry;
+
SyncScope::ID SingleThreadSSID =
pImpl->getOrInsertSyncScopeID("singlethread");
assert(SingleThreadSSID == SyncScope::SingleThread &&
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index ca3f148f881a4..dbda66a932ae0 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -3836,7 +3836,8 @@ void Verifier::visitCallBase(CallBase &Call) {
FoundGCTransitionBundle = false, FoundCFGuardTargetBundle = false,
FoundPreallocatedBundle = false, FoundGCLiveBundle = false,
FoundPtrauthBundle = false, FoundKCFIBundle = false,
- FoundAttachedCallBundle = false;
+ FoundAttachedCallBundle = false, FoundFpeControlBundle = false,
+ FoundFpeExceptBundle = false;
for (unsigned i = 0, e = Call.getNumOperandBundles(); i < e; ++i) {
OperandBundleUse BU = Call.getOperandBundleAt(i);
uint32_t Tag = BU.getTagID();
@@ -3899,6 +3900,34 @@ void Verifier::visitCallBase(CallBase &Call) {
"Multiple \"clang.arc.attachedcall\" operand bundles", Call);
FoundAttachedCallBundle = true;
verifyAttachedCallBundle(Call, BU);
+ } else if (Tag == LLVMContext::OB_fp_control) {
+ Check(!FoundFpeControlBundle, "Multiple fp.control operand bundles",
+ Call);
+ Check(BU.Inputs.size() == 1,
+ "Expected exactly one fp.control bundle operand", Call);
+ auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
+ Check(V, "Value of fp.control bundle operand must be a metadata", Call);
+ auto *MDS = dyn_cast<MDString>(V->getMetadata());
+ Check(MDS, "Value of fp.control bundle operand must be a string", Call);
+ auto RM = convertBundleToRoundingMode(MDS->getString());
+ Check(RM.has_value(),
+ "Value of fp.control bundle operand is not a correct rounding mode",
+ Call);
+ FoundFpeControlBundle = true;
+ } else if (Tag == LLVMContext::OB_fp_except) {
+ Check(!FoundFpeExceptBundle, "Multiple fp.except operand bundles", Call);
+ Check(BU.Inputs.size() == 1,
+ "Expected exactly one fp.except bundle operand", Call);
+ auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
+ Check(V, "Value of fp.except bundle operand must be a metadata", Call);
+ auto *MDS = dyn_cast<MDString>(V->getMetadata());
+ Check(MDS, "Value of fp.except bundle operand must be a string", Call);
+ auto EB = convertBundleToExceptionBehavior(MDS->getString());
+ Check(EB.has_value(),
+ "Value of fp.except bundle operand is not a correct exception "
+ "behavior",
+ Call);
+ FoundFpeExceptBundle = true;
}
}
diff --git a/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll b/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll
index d860104b9cb3d..01e5b3f6673ae 100644
--- a/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll
+++ b/llvm/test/Bitcode/operand-bundles-bc-analyzer.ll
@@ -13,6 +13,8 @@
; CHECK-NEXT: <OPERAND_BUNDLE_TAG
; CHECK-NEXT: <OPERAND_BUNDLE_TAG
; CHECK-NEXT: <OPERAND_BUNDLE_TAG
+; CHECK-NEXT: <OPERAND_BUNDLE_TAG
+; CHECK-NEXT: <OPERAND_BUNDLE_TAG
; CHECK-NEXT: </OPERAND_BUNDLE_TAGS_BLOCK
; CHECK: <FUNCTION_BLOCK
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index 4934843d5a2ed..12b0fca97e5af 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -51,4 +51,95 @@ entry:
ret double %fadd
}
+; Test multiple fp.control bundles.
+; CHECK-NEXT: Multiple fp.control operand bundles
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+define double @f6(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that has more than one operands.
+; CHECK-NEXT: Expected exactly one fp.control bundle operand
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+define double @f7(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that has non-metadata operand.
+; CHECK-NEXT: Value of fp.control bundle operand must be a metadata
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(i32 0) ]
+define double @f8(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(i32 0) ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that has non-string operand.
+; CHECK-NEXT: Value of fp.control bundle operand must be a string
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata i64 3) ]
+define double @f9(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !{i64 3}) ]
+ ret double %ftrunc
+}
+
+; Test fp.control bundle that specifies incorrect value.
+; CHECK-NEXT: Value of fp.control bundle operand is not a correct rounding mode
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"qqq") ]
+define double @f10(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"qqq") ]
+ ret double %ftrunc
+}
+
+; Test multiple fp.except bundles.
+; CHECK-NEXT: Multiple fp.except operand bundles
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict"), "fp.except"(metadata !"strict") ]
+define double @f11(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !"strict"), "fp.except"(metadata !"strict") ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that has more than one operands.
+; CHECK-NEXT: Expected exactly one fp.except bundle operand
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict", metadata !"strict") ]
+define double @f12(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !"strict", metadata !"strict") ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that has non-metadata operand.
+; CHECK-NEXT: Value of fp.except bundle operand must be a metadata
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(i32 0) ]
+define double @f13(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(i32 0) ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that has non-string operand.
+; CHECK-NEXT: Value of fp.except bundle operand must be a string
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata i64 3) ]
+define double @f14(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !{i64 3}) ]
+ ret double %ftrunc
+}
+
+; Test fp.except bundle that specifies incorrect value.
+; CHECK-NEXT: Value of fp.except bundle operand is not a correct exception behavior
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"qqq") ]
+define double @f15(double %a) #0 {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.except"(metadata !"qqq") ]
+ ret double %ftrunc
+}
+
+
attributes #0 = { strictfp }
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index 4f2ede3321080..a0a0d8ced8fe6 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -457,6 +457,203 @@ TEST_F(IRBuilderTest, ConstrainedFPFunctionCall) {
EXPECT_FALSE(verifyModule(*M));
}
+TEST_F(IRBuilderTest, FPBundlesDefault) {
+ IRBuilder<> Builder(BB);
+ GlobalVariable *GVDouble = new GlobalVariable(
+ *M, Type::getDoubleTy(Ctx), true, GlobalValue::ExternalLinkage, nullptr);
+ Value *FnArg = Builder.CreateLoad(GVDouble->getValueType(), GVDouble);
+ Function *Fn = Intrinsic::getOrInsertDeclaration(
+ M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
+
+ // A floating-point operation does not have side effects in default
+ // environment even.
+ {
+ Value *V = Builder.CreateCall(Fn, {FnArg});
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // Check call with FP bundles, rounding is set to default value.
+ // nearbyint(%x) [ "fp.control" (metadata !"rte") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // Check call with FP bundles, exception behavior is set to default value.
+ // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // Check call with FP bundles, both rounding mode and exception behavior are
+ // set.
+ // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+}
+
+TEST_F(IRBuilderTest, FPBundlesStrict) {
+ F->addFnAttr(Attribute::StrictFP);
+
+ IRBuilder<> Builder(BB);
+ Builder.setDefaultConstrainedExcept(fp::ebStrict);
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardZero);
+ Builder.setIsFPConstrained(true);
+
+ GlobalVariable *GVDouble = new GlobalVariable(
+ *M, Type::getDoubleTy(Ctx), true, GlobalValue::ExternalLinkage, nullptr);
+ Value *FnArg = Builder.CreateLoad(GVDouble->getValueType(), GVDouble);
+ Function *Fn = Intrinsic::getOrInsertDeclaration(
+ M.get(), Intrinsic::nearbyint, {Type::getDoubleTy(Ctx)});
+
+ // A floating-point operation has side effects in strictfp environment even
+ // if it has no FP bundles.
+ {
+ Value *V = Builder.CreateCall(Fn, {FnArg});
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, with default (dynamic) rounding mode
+ // nearbyint(%x) [ "fp.control" (metadata !"dyn") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, with specific rounding mode
+ // nearbyint(%x) [ "fp.control" (metadata !"rtz") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, exception behavior is set to default value.
+ // nearbyint(%x) [ "fp.except" (metadata !"strict") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebStrict);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebStrict, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with FP bundles, exception behavior is set to specific value.
+ // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Check call with both FP bundles.
+ // nearbyint(%x) [ "fp.control" (metadata !"rtz"),
+ // "fp.except" (metadata !"ignore") ]
+ {
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesAccessInaccessibleMem());
+ }
+
+ // Function calls, that do not depend on FP options, does not have
+ // memory effects.
+ {
+ Function *Fn = Intrinsic::getOrInsertDeclaration(M.get(), Intrinsic::fabs,
+ {Type::getDoubleTy(Ctx)});
+ Value *V = Builder.CreateCall(Fn, {FnArg});
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+}
+
TEST_F(IRBuilderTest, Lifetime) {
IRBuilder<> Builder(BB);
AllocaInst *Var1 = Builder.CreateAlloca(Builder.getInt8Ty());
>From 0d7af1d370a656951fc8e95ad9f1238201f6b53f Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Wed, 16 Apr 2025 00:03:49 +0700
Subject: [PATCH 2/7] Fix test
---
clang/test/CodeGen/strictfp-elementwise-builtins.cpp | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
index b250512efc5c7..532686512b0d4 100644
--- a/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
+++ b/clang/test/CodeGen/strictfp-elementwise-builtins.cpp
@@ -210,7 +210,7 @@ float4 strict_elementwise_rint(float4 a) {
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z28strict_elementwise_nearbyintDv4_f
// CHECK-SAME: (<4 x float> noundef [[A:%.*]]) local_unnamed_addr #[[ATTR2]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_NEARBYINT:%.*]] = tail call <4 x float> @llvm.nearbyint.v4f32(<4 x float> [[A]]) #[[ATTR4]]
+// CHECK-NEXT: [[ELT_NEARBYINT:%.*]] = tail call <4 x float> @llvm.nearbyint.v4f32(<4 x float> [[A]]) #[[ATTR5:[0-9]+]]
// CHECK-NEXT: ret <4 x float> [[ELT_NEARBYINT]]
//
float4 strict_elementwise_nearbyint(float4 a) {
@@ -300,7 +300,7 @@ float4 strict_elementwise_atan2(float4 a, float4 b) {
// CHECK-LABEL: define dso_local noundef <4 x float> @_Z24strict_elementwise_truncDv4_f
// CHECK-SAME: (<4 x float> noundef [[A:%.*]]) local_unnamed_addr #[[ATTR2]] {
// CHECK-NEXT: entry:
-// CHECK-NEXT: [[ELT_TRUNC:%.*]] = tail call <4 x float> @llvm.trunc.v4f32(<4 x float> [[A]]) #[[ATTR4]]
+// CHECK-NEXT: [[ELT_TRUNC:%.*]] = tail call <4 x float> @llvm.trunc.v4f32(<4 x float> [[A]]) #[[ATTR5]]
// CHECK-NEXT: ret <4 x float> [[ELT_TRUNC]]
//
float4 strict_elementwise_trunc(float4 a) {
>From 7219856c662f8dc01c15bc7496f3dfcbcf29d2fd Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 17 Apr 2025 14:00:55 +0700
Subject: [PATCH 3/7] Set MemoryEffects of call only if called function is
nomem
It make call close to constrained calls, where memory effects of the
calls do not define memory effects. It helps migration to operand
bundles.
---
llvm/lib/IR/IRBuilder.cpp | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index ddeb7949627c6..8727684ee3c82 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -131,12 +131,13 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
// If the builder is in strictfp mode and has non-default options (like
// non-dynamic rounding), add corresponding operand bundle. If such bundle is
// already present, assume it overwrites defaults.
- bool doesTheCallAccessFPEnv = false;
+ bool NeedUpdateMemoryEffects = false;
if (IsFPConstrained) {
if (const auto *Func = dyn_cast<Function>(Callee)) {
if (Intrinsic::ID ID = Func->getIntrinsicID()) {
if (IntrinsicInst::isFloatingPointOperation(ID)) {
- doesTheCallAccessFPEnv = true;
+ MemoryEffects FME = Func->getMemoryEffects();
+ NeedUpdateMemoryEffects = !FME.doesAccessInaccessibleMem();
bool NeedRound = DefaultConstrainedRounding != RoundingMode::Dynamic;
bool NeedExcept = DefaultConstrainedExcept != fp::ebStrict;
for (const auto &Item : OpBundles) {
@@ -158,10 +159,8 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
// If the call accesses FPE, update memory effects accordingly.
CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
- if (doesTheCallAccessFPEnv) {
+ if (NeedUpdateMemoryEffects) {
MemoryEffects ME = MemoryEffects::inaccessibleMemOnly();
- if (CI->getAttributes().hasFnAttr(Attribute::Memory))
- ME |= CI->getAttributes().getMemoryEffects();
auto A = Attribute::getWithMemoryEffects(getContext(), ME);
CI->addFnAttr(A);
}
>From 39b696de95a79774f284d1020924fbf51959915e Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Thu, 17 Apr 2025 15:30:17 +0700
Subject: [PATCH 4/7] Fix method signature
---
llvm/include/llvm/IR/InstrTypes.h | 6 +++---
llvm/lib/IR/Instructions.cpp | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index 918c653d30a74..ac08d1bee5782 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -2174,11 +2174,11 @@ class CallBase : public Instruction {
return false;
}
- /// Return rounding mode specified by operand bundles.
+ /// Return rounding mode specified for this call.
RoundingMode getRoundingMode() const;
- /// Return exception behavior specified by operand bundles.
- std::optional<fp::ExceptionBehavior> getExceptionBehavior() const;
+ /// Return exception behavior specified for this call.
+ fp::ExceptionBehavior getExceptionBehavior() const;
/// Used to keep track of an operand bundle. See the main comment on
/// OperandBundleUser above.
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index 295de574ae4f3..3adce4b45281a 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -656,7 +656,7 @@ RoundingMode CallBase::getRoundingMode() const {
return RoundingMode::NearestTiesToEven;
}
-std::optional<fp::ExceptionBehavior> CallBase::getExceptionBehavior() const {
+fp::ExceptionBehavior CallBase::getExceptionBehavior() const {
// Try determining exception behavior from FP bundle.
std::optional<fp::ExceptionBehavior> EB;
if (auto ExceptionBundle = getOperandBundle(LLVMContext::OB_fp_except)) {
>From 0cd03e23adc03011da7f3806e889d25325e95219 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Fri, 18 Apr 2025 01:55:35 +0700
Subject: [PATCH 5/7] Organize checks in Verifier
---
llvm/lib/IR/Verifier.cpp | 52 ++++++++++++++++++-----------
llvm/test/Verifier/fp-intrinsics.ll | 22 ++++++------
2 files changed, 44 insertions(+), 30 deletions(-)
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index dbda66a932ae0..72c6dbb72f4c5 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -3901,32 +3901,46 @@ void Verifier::visitCallBase(CallBase &Call) {
FoundAttachedCallBundle = true;
verifyAttachedCallBundle(Call, BU);
} else if (Tag == LLVMContext::OB_fp_control) {
- Check(!FoundFpeControlBundle, "Multiple fp.control operand bundles",
- Call);
- Check(BU.Inputs.size() == 1,
- "Expected exactly one fp.control bundle operand", Call);
- auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
- Check(V, "Value of fp.control bundle operand must be a metadata", Call);
- auto *MDS = dyn_cast<MDString>(V->getMetadata());
- Check(MDS, "Value of fp.control bundle operand must be a string", Call);
- auto RM = convertBundleToRoundingMode(MDS->getString());
- Check(RM.has_value(),
- "Value of fp.control bundle operand is not a correct rounding mode",
+ Check(!FoundFpeControlBundle, "Multiple \"fp.control\" operand bundles",
Call);
+ bool FoundRoundingMode = false;
+ for (auto &U : BU.Inputs) {
+ Value *V = U.get();
+ Check(isa<MetadataAsValue>(V),
+ "Value of a \"fp.control\" bundle operand must be a metadata",
+ Call);
+ Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
+ Check(isa<MDString>(MD),
+ "Value of a \"fp.control\" bundle operand must be a string",
+ Call);
+ StringRef Item = cast<MDString>(MD)->getString();
+ if (convertBundleToRoundingMode(Item)) {
+ Check(!FoundRoundingMode, "Rounding mode is specified more that once",
+ Call);
+ FoundRoundingMode = true;
+ } else {
+ CheckFailed("Unrecognized value in \"fp.control\" bundle operand",
+ Call);
+ }
+ }
FoundFpeControlBundle = true;
} else if (Tag == LLVMContext::OB_fp_except) {
- Check(!FoundFpeExceptBundle, "Multiple fp.except operand bundles", Call);
+ Check(!FoundFpeExceptBundle, "Multiple \"fp.except\" operand bundles",
+ Call);
Check(BU.Inputs.size() == 1,
- "Expected exactly one fp.except bundle operand", Call);
+ "Expected exactly one \"fp.except\" bundle operand", Call);
auto *V = dyn_cast<MetadataAsValue>(BU.Inputs.front());
- Check(V, "Value of fp.except bundle operand must be a metadata", Call);
+ Check(V, "Value of a \"fp.except\" bundle operand must be a metadata",
+ Call);
auto *MDS = dyn_cast<MDString>(V->getMetadata());
- Check(MDS, "Value of fp.except bundle operand must be a string", Call);
- auto EB = convertBundleToExceptionBehavior(MDS->getString());
- Check(EB.has_value(),
- "Value of fp.except bundle operand is not a correct exception "
- "behavior",
+ Check(MDS, "Value of a \"fp.except\" bundle operand must be a string",
Call);
+ auto EB = convertBundleToExceptionBehavior(MDS->getString());
+ Check(
+ EB.has_value(),
+ "Value of a \"fp.except\" bundle operand is not a correct exception "
+ "behavior",
+ Call);
FoundFpeExceptBundle = true;
}
}
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index 12b0fca97e5af..1e1df1b0da96a 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -52,7 +52,7 @@ entry:
}
; Test multiple fp.control bundles.
-; CHECK-NEXT: Multiple fp.control operand bundles
+; CHECK-NEXT: Multiple "fp.control" operand bundles
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
define double @f6(double %a) #0 {
entry:
@@ -60,8 +60,8 @@ entry:
ret double %ftrunc
}
-; Test fp.control bundle that has more than one operands.
-; CHECK-NEXT: Expected exactly one fp.control bundle operand
+; Test fp.control bundle that has more than one rounding mode specification.
+; CHECK-NEXT: Rounding mode is specified more that once
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz", metadata !"rte") ]
define double @f7(double %a) #0 {
entry:
@@ -70,7 +70,7 @@ entry:
}
; Test fp.control bundle that has non-metadata operand.
-; CHECK-NEXT: Value of fp.control bundle operand must be a metadata
+; CHECK-NEXT: Value of a "fp.control" bundle operand must be a metadata
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(i32 0) ]
define double @f8(double %a) #0 {
entry:
@@ -79,7 +79,7 @@ entry:
}
; Test fp.control bundle that has non-string operand.
-; CHECK-NEXT: Value of fp.control bundle operand must be a string
+; CHECK-NEXT: Value of a "fp.control" bundle operand must be a string
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata i64 3) ]
define double @f9(double %a) #0 {
entry:
@@ -88,7 +88,7 @@ entry:
}
; Test fp.control bundle that specifies incorrect value.
-; CHECK-NEXT: Value of fp.control bundle operand is not a correct rounding mode
+; CHECK-NEXT: Unrecognized value in "fp.control" bundle operand
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"qqq") ]
define double @f10(double %a) #0 {
entry:
@@ -97,7 +97,7 @@ entry:
}
; Test multiple fp.except bundles.
-; CHECK-NEXT: Multiple fp.except operand bundles
+; CHECK-NEXT: Multiple "fp.except" operand bundles
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict"), "fp.except"(metadata !"strict") ]
define double @f11(double %a) #0 {
entry:
@@ -106,7 +106,7 @@ entry:
}
; Test fp.except bundle that has more than one operands.
-; CHECK-NEXT: Expected exactly one fp.except bundle operand
+; CHECK-NEXT: Expected exactly one "fp.except" bundle operand
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"strict", metadata !"strict") ]
define double @f12(double %a) #0 {
entry:
@@ -115,7 +115,7 @@ entry:
}
; Test fp.except bundle that has non-metadata operand.
-; CHECK-NEXT: Value of fp.except bundle operand must be a metadata
+; CHECK-NEXT: Value of a "fp.except" bundle operand must be a metadata
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(i32 0) ]
define double @f13(double %a) #0 {
entry:
@@ -124,7 +124,7 @@ entry:
}
; Test fp.except bundle that has non-string operand.
-; CHECK-NEXT: Value of fp.except bundle operand must be a string
+; CHECK-NEXT: Value of a "fp.except" bundle operand must be a string
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata i64 3) ]
define double @f14(double %a) #0 {
entry:
@@ -133,7 +133,7 @@ entry:
}
; Test fp.except bundle that specifies incorrect value.
-; CHECK-NEXT: Value of fp.except bundle operand is not a correct exception behavior
+; CHECK-NEXT: Value of a "fp.except" bundle operand is not a correct exception behavior
; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.except"(metadata !"qqq") ]
define double @f15(double %a) #0 {
entry:
>From 617dee0c1ce537cb208ab045f16840a0f0a262e1 Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Sat, 19 Apr 2025 02:05:28 +0700
Subject: [PATCH 6/7] Small changes in documentation
---
llvm/docs/LangRef.rst | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index ba119ef97f765..12fd0c562cb3e 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3104,8 +3104,9 @@ floating-point control modes and the treatment of status bits respectively.
An operand bundle tagged with "fp.control" contains information about the
control modes used for the operation execution. Operands specified in this
bundle represent particular options. Currently, only rounding mode is supported.
-It is represented by a metadata string value, which specifies the rounding mode
-to be used for the operation evaluation. Possible values are:
+
+Rounding mode is represented by a metadata string value, which specifies the
+the mode used for the operation evaluation. Possible values are:
::
@@ -3116,10 +3117,10 @@ to be used for the operation evaluation. Possible values are:
"rmm" - to nearest, ties away from zero
"dyn" - rounding mode is taken from control register
-If "fp.control" is absent, the default rounding rounding mode is taken from the
-control register (dynamic rounding). In the particular case of
-:ref:`default floating-point environment <floatenv>`, it must be rounding to
-nearest, ties to even.
+Only one such value may be specified. If "fp.control" is absent, the default
+rounding rounding mode is taken from the control register (dynamic rounding).
+In the particular case of :ref:`default floating-point environment <floatenv>`,
+the operation uses rounding to nearest, ties to even.
An operand bundle tagged with "fp.except" may be associated with operations
that can read or write floating-point exception flags. It contains a single
>From 131f152c1e10fd58c27602eade3a3750654c615a Mon Sep 17 00:00:00 2001
From: Serge Pavlov <sepavloff at gmail.com>
Date: Wed, 23 Jul 2025 13:16:45 +0700
Subject: [PATCH 7/7] Update according to the obtained feedback
- Bundles are renamed: "fp.control" -> "fp.round",
- Corrected documentation,
- DAG nodes STRICT_* are used to lower the supported intrinsics,
- Constrained properties in IRBuilder are reused for handling FP bundles,
- DAG gets methods for lowering FP operations,
- IR dump prints FP memory effects,
- Fixed some problems in IRBuilderBase::CreateCall,
- Added a new check to Verifier,
- Added tests for checking code generation.
---
clang/lib/CodeGen/CodeGenFunction.cpp | 2 +-
llvm/docs/LangRef.rst | 53 ++---
llvm/include/llvm/IR/FloatingPointOps.def | 16 +-
llvm/include/llvm/IR/IRBuilder.h | 35 ++--
llvm/include/llvm/IR/InstrTypes.h | 22 ++-
llvm/include/llvm/IR/LLVMContext.h | 2 +-
.../SelectionDAG/SelectionDAGBuilder.cpp | 112 ++++++++---
.../SelectionDAG/SelectionDAGBuilder.h | 3 +
llvm/lib/IR/AsmWriter.cpp | 25 +++
llvm/lib/IR/FPEnv.cpp | 24 +--
llvm/lib/IR/IRBuilder.cpp | 87 +++++----
llvm/lib/IR/Instructions.cpp | 67 ++++---
llvm/lib/IR/LLVMContext.cpp | 10 +-
llvm/lib/IR/Verifier.cpp | 24 ++-
.../CodeGen/X86/fp-strict-scalar-round.ll | 184 ++++++++++++++++++
llvm/test/CodeGen/X86/vec-strict-512.ll | 36 ++++
llvm/test/Verifier/fp-intrinsics.ll | 47 +++--
llvm/unittests/IR/IRBuilderTest.cpp | 91 ++++++---
18 files changed, 615 insertions(+), 225 deletions(-)
diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp
index ab345a598c4e8..5f17e29d46718 100644
--- a/clang/lib/CodeGen/CodeGenFunction.cpp
+++ b/clang/lib/CodeGen/CodeGenFunction.cpp
@@ -1092,7 +1092,7 @@ void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
if ((FD && (FD->UsesFPIntrin() || FD->hasAttr<StrictFPAttr>())) ||
(!FD && (FPExceptionBehavior != llvm::fp::ebIgnore ||
RM != llvm::RoundingMode::NearestTiesToEven))) {
- Builder.setIsFPConstrained(true);
+ Builder.setIsFPConstrained(true, false);
Fn->addFnAttr(llvm::Attribute::StrictFP);
}
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 12fd0c562cb3e..aa2fb1327e8b6 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3098,43 +3098,48 @@ Floating-point Operand Bundles
These operand bundles are used for calls that involve floating-point
operations and interact with :ref:`floating-point environment <floatenv>` or
depend on floating-point options, such as rounding mode, denormal modes, etc.
-There are two kinds of such operand bundles, which represent the value of
-floating-point control modes and the treatment of status bits respectively.
-An operand bundle tagged with "fp.control" contains information about the
-control modes used for the operation execution. Operands specified in this
-bundle represent particular options. Currently, only rounding mode is supported.
-
-Rounding mode is represented by a metadata string value, which specifies the
-the mode used for the operation evaluation. Possible values are:
+An operand bundle tagged with "fp.round" contains information about the
+rounding mode used for the operation execution (the effective rounding mode).
+This mode is represented by a metadata string value and specifies the mode used
+for the operation evaluation. Possible values are:
::
- "rtz" - toward zero
- "rte" - to nearest, ties to even
- "rtp" - toward positive infinity
- "rtn" - toward negative infinity
- "rmm" - to nearest, ties away from zero
- "dyn" - rounding mode is taken from control register
+ "towardzero"
+ "tonearest"
+ "upward"
+ "downward"
+ "tonearestaway"
+ "dynamic"
-Only one such value may be specified. If "fp.control" is absent, the default
-rounding rounding mode is taken from the control register (dynamic rounding).
-In the particular case of :ref:`default floating-point environment <floatenv>`,
-the operation uses rounding to nearest, ties to even.
+Only one value may be specified. If the "fp.round" bundle is absent, and
+the operation depends on rounding mode, the default behavior is to use the value
+from a control register (dynamic rounding). In the specific case of
+:ref:`default floating-point environment <floatenv>`, the register is assumed to
+set rounding to nearest, ties to even.
An operand bundle tagged with "fp.except" may be associated with operations
-that can read or write floating-point exception flags. It contains a single
-metadata string value, which can have one of the following values:
+that can raise floating-point exceptions. It contains a single metadata string
+value, which can have one of the following:
::
"ignore"
"strict"
- "maytrap"
-
-It has the same meaning as the corresponding argument in
-:ref:`constrained intrinsics <constrainedfp>`.
+If the argument is "ignore", floating-point exceptions raised by the call are not
+intended to be observed. Optimization transformations may reorder such operations
+or omit them in some cases. For example, if the call result is unused, the call
+may be removed, even if it could raise exceptions. This is the only permitted value
+in the :ref:`default floating-point environment <floatenv>`.
+
+If the argument of "fp.except" is "strict", all transformations must preserve the
+floating-point exception semantics of the original code. Any exception that would
+have been raised by the original code must also be raised by the transformed code
+unless it can be proven unobservable. No new observable floating-point exceptions
+may be introduced. This value may only be used only in functions with
+``strictfp`` attribute.
.. _moduleasm:
diff --git a/llvm/include/llvm/IR/FloatingPointOps.def b/llvm/include/llvm/IR/FloatingPointOps.def
index 8567b5dbac302..465be751a0087 100644
--- a/llvm/include/llvm/IR/FloatingPointOps.def
+++ b/llvm/include/llvm/IR/FloatingPointOps.def
@@ -10,15 +10,21 @@
//
//===----------------------------------------------------------------------===//
+// Describes floating-point operation.
+// Arguments of the entries are:
+// N - intrinsic function name,
+// D - DAG node corresponding to the intrinsic.
#ifndef FUNCTION
#define FUNCTION(N,D)
#endif
-// Arguments of the entries are:
-// - intrinsic function name,
-// - DAG node corresponding to the intrinsic.
+// Describes floating-point operation, which lowers to STRICT_* nodes in DAG.
+#ifndef LEGACY_DAG
+#define LEGACY_DAG(N,D) FUNCTION(N,D)
+#endif
-FUNCTION(nearbyint, FNEARBYINT)
-FUNCTION(trunc, FTRUNC)
+LEGACY_DAG(nearbyint, FNEARBYINT)
+LEGACY_DAG(trunc, FTRUNC)
#undef FUNCTION
+#undef LEGACY_DAG
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index e9c5bdae51cc6..b07e7f38503e9 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -153,8 +153,8 @@ class IRBuilderBase {
FastMathFlags FMF;
bool IsFPConstrained = false;
- fp::ExceptionBehavior DefaultConstrainedExcept = fp::ebStrict;
- RoundingMode DefaultConstrainedRounding = RoundingMode::Dynamic;
+ fp::ExceptionBehavior DefaultConstrainedExcept = fp::ebIgnore;
+ RoundingMode DefaultConstrainedRounding = RoundingMode::NearestTiesToEven;
ArrayRef<OperandBundleDef> DefaultOperandBundles;
@@ -348,7 +348,18 @@ class IRBuilderBase {
/// enabled the CreateF<op>() calls instead create constrained
/// floating point intrinsic calls. Fast math flags are unaffected
/// by this setting.
- void setIsFPConstrained(bool IsCon) { IsFPConstrained = IsCon; }
+ void setIsFPConstrained(bool IsCon, bool AndReset = true) {
+ if (AndReset) {
+ if (IsCon) {
+ setDefaultConstrainedRounding(RoundingMode::Dynamic);
+ setDefaultConstrainedExcept(fp::ebStrict);
+ } else {
+ setDefaultConstrainedRounding(RoundingMode::NearestTiesToEven);
+ setDefaultConstrainedExcept(fp::ebIgnore);
+ }
+ }
+ IsFPConstrained = IsCon;
+ }
/// Query for the use of constrained floating point math
bool getIsFPConstrained() { return IsFPConstrained; }
@@ -2761,23 +2772,21 @@ class IRBuilderBase {
LLVM_ABI CallInst *CreateDereferenceableAssumption(Value *PtrValue,
Value *SizeValue);
- /// Create an operand bundle in the provided bundle set to represent given FP
- /// rounding mode.
+ /// Create an operand bundle in the provided bundle set to represent the given
+ /// floating-point rounding mode.
///
/// If the rounding mode is not defined, adds the default rounding mode,
/// stored in this builder object.
- void
- createFPRoundingBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<RoundingMode> Rounding = std::nullopt);
+ void createRoundingBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode RM);
- /// Create an operand bundle in the provided bundle set to represent FP
- /// exception behavior.
+ /// Create an operand bundle in the provided bundle set to represent the given
+ /// floating-point exception behavior.
///
/// If the exception behavior is not defined, adds the default behavior,
/// stored in this builder object.
- void createFPExceptionBundle(
- SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<fp::ExceptionBehavior> Except = std::nullopt);
+ void createExceptionBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except);
};
/// This provides a uniform API for creating instructions and inserting
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index ac08d1bee5782..b3c9a89e805e3 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -1096,12 +1096,17 @@ template <typename InputTy> class OperandBundleDefT {
using OperandBundleDef = OperandBundleDefT<Value *>;
using ConstOperandBundleDef = OperandBundleDefT<const Value *>;
-void addFPRoundingBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- RoundingMode Rounding);
-void addFPExceptionBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- fp::ExceptionBehavior Except);
+/// Add a bundle with tag "fp.round" and the specified rounding to the given
+/// bundle set.
+void addRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding);
+
+/// Add a bundle with tag "fp.except" and the specified exception behavior to
+/// the given bundle set.
+void addExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except);
//===----------------------------------------------------------------------===//
// CallBase Class
@@ -1162,6 +1167,7 @@ class CallBase : public Instruction {
/// number of extra operands.
LLVM_ABI unsigned getNumSubclassExtraOperandsDynamic() const;
+ /// Get memory effects specific to floating-point operations.
MemoryEffects getFloatingPointMemoryEffects() const;
public:
@@ -2174,10 +2180,10 @@ class CallBase : public Instruction {
return false;
}
- /// Return rounding mode specified for this call.
+ /// Return the effective rounding mode for this call.
RoundingMode getRoundingMode() const;
- /// Return exception behavior specified for this call.
+ /// Return the effective exception behavior for this call.
fp::ExceptionBehavior getExceptionBehavior() const;
/// Used to keep track of an operand bundle. See the main comment on
diff --git a/llvm/include/llvm/IR/LLVMContext.h b/llvm/include/llvm/IR/LLVMContext.h
index 1ac9dbb197d87..b5ba8d392bc26 100644
--- a/llvm/include/llvm/IR/LLVMContext.h
+++ b/llvm/include/llvm/IR/LLVMContext.h
@@ -97,7 +97,7 @@ class LLVMContext {
OB_ptrauth = 7, // "ptrauth"
OB_kcfi = 8, // "kcfi"
OB_convergencectrl = 9, // "convergencectrl"
- OB_fp_control = 10, // "fp.control"
+ OB_fp_round = 10, // "fp.round"
OB_fp_except = 11, // "fp.except"
};
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 868e2f4d7134f..890e8c0c9b783 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -6816,9 +6816,7 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
case Intrinsic::exp10:
case Intrinsic::floor:
case Intrinsic::ceil:
- case Intrinsic::trunc:
case Intrinsic::rint:
- case Intrinsic::nearbyint:
case Intrinsic::round:
case Intrinsic::roundeven:
case Intrinsic::canonicalize: {
@@ -6840,9 +6838,7 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
case Intrinsic::exp10: Opcode = ISD::FEXP10; break;
case Intrinsic::floor: Opcode = ISD::FFLOOR; break;
case Intrinsic::ceil: Opcode = ISD::FCEIL; break;
- case Intrinsic::trunc: Opcode = ISD::FTRUNC; break;
case Intrinsic::rint: Opcode = ISD::FRINT; break;
- case Intrinsic::nearbyint: Opcode = ISD::FNEARBYINT; break;
case Intrinsic::round: Opcode = ISD::FROUND; break;
case Intrinsic::roundeven: Opcode = ISD::FROUNDEVEN; break;
case Intrinsic::canonicalize: Opcode = ISD::FCANONICALIZE; break;
@@ -6977,6 +6973,11 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
#include "llvm/IR/VPIntrinsics.def"
visitVectorPredicationIntrinsic(cast<VPIntrinsic>(I));
return;
+#define FUNCTION(NAME, DAGN) \
+ case Intrinsic::NAME: \
+ visitFPOperation(I, ISD::DAGN); \
+ break;
+#include "llvm/IR/FloatingPointOps.def"
case Intrinsic::fptrunc_round: {
// Get the last argument, the metadata and convert it to an integer in the
// call
@@ -8273,6 +8274,33 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
}
}
+void SelectionDAGBuilder::pushOutChain(SDValue Result,
+ fp::ExceptionBehavior EB) {
+ assert(Result.getNode()->getNumValues() == 2);
+
+ // Push node to the appropriate list so that future instructions can be
+ // chained up correctly.
+ SDValue OutChain = Result.getValue(1);
+ switch (EB) {
+ case fp::ExceptionBehavior::ebIgnore:
+ // The only reason why ebIgnore nodes still need to be chained is that
+ // they might depend on the current rounding mode, and therefore must
+ // not be moved across instruction that may change that mode.
+ [[fallthrough]];
+ case fp::ExceptionBehavior::ebMayTrap:
+ // These must not be moved across calls or instructions that may change
+ // floating-point exception masks.
+ PendingConstrainedFP.push_back(OutChain);
+ break;
+ case fp::ExceptionBehavior::ebStrict:
+ // These must not be moved across calls or instructions that may change
+ // floating-point exception masks or read floating-point exception flags.
+ // In addition, they cannot be optimized out even if unused.
+ PendingConstrainedFPStrict.push_back(OutChain);
+ break;
+ }
+}
+
void SelectionDAGBuilder::visitConstrainedFPIntrinsic(
const ConstrainedFPIntrinsic &FPI) {
SDLoc sdl = getCurSDLoc();
@@ -8286,32 +8314,6 @@ void SelectionDAGBuilder::visitConstrainedFPIntrinsic(
for (unsigned I = 0, E = FPI.getNonMetadataArgCount(); I != E; ++I)
Opers.push_back(getValue(FPI.getArgOperand(I)));
- auto pushOutChain = [this](SDValue Result, fp::ExceptionBehavior EB) {
- assert(Result.getNode()->getNumValues() == 2);
-
- // Push node to the appropriate list so that future instructions can be
- // chained up correctly.
- SDValue OutChain = Result.getValue(1);
- switch (EB) {
- case fp::ExceptionBehavior::ebIgnore:
- // The only reason why ebIgnore nodes still need to be chained is that
- // they might depend on the current rounding mode, and therefore must
- // not be moved across instruction that may change that mode.
- [[fallthrough]];
- case fp::ExceptionBehavior::ebMayTrap:
- // These must not be moved across calls or instructions that may change
- // floating-point exception masks.
- PendingConstrainedFP.push_back(OutChain);
- break;
- case fp::ExceptionBehavior::ebStrict:
- // These must not be moved across calls or instructions that may change
- // floating-point exception masks or read floating-point exception flags.
- // In addition, they cannot be optimized out even if unused.
- PendingConstrainedFPStrict.push_back(OutChain);
- break;
- }
- };
-
const TargetLowering &TLI = DAG.getTargetLoweringInfo();
EVT VT = TLI.getValueType(DAG.getDataLayout(), FPI.getType());
SDVTList VTs = DAG.getVTList(VT, MVT::Other);
@@ -9355,6 +9357,56 @@ bool SelectionDAGBuilder::visitBinaryFloatCall(const CallInst &I,
return true;
}
+bool SelectionDAGBuilder::visitFPOperation(const CallInst &I, unsigned Opcode) {
+ // We already checked this call's prototype; verify it doesn't modify errno.
+ MemoryEffects ME = I.getMemoryEffects();
+ if (!ME.onlyAccessesInaccessibleMem())
+ return false;
+
+ SmallVector<SDValue, 4> Operands;
+ bool HasChain = ME.doesAccessInaccessibleMem();
+ if (HasChain)
+ Operands.push_back(getRoot());
+ for (auto &Arg : I.args())
+ Operands.push_back(getValue(Arg));
+
+ const TargetLowering &TLI = DAG.getTargetLoweringInfo();
+ EVT VT = TLI.getValueType(DAG.getDataLayout(), I.getType(), true);
+ SDVTList NodeVT;
+ if (HasChain)
+ NodeVT = DAG.getVTList(VT, MVT::Other);
+ else
+ NodeVT = DAG.getVTList(VT);
+
+ SDNodeFlags Flags;
+ fp::ExceptionBehavior EB = I.getExceptionBehavior();
+ if (EB == fp::ExceptionBehavior::ebIgnore)
+ Flags.setNoFPExcept(true);
+ if (auto *FPOp = dyn_cast<FPMathOperator>(&I))
+ Flags.copyFMF(*FPOp);
+
+ // Temporary solution: use STRICT_* nodes.
+ if (HasChain)
+ switch (Opcode) {
+ default:
+ break;
+#define LEGACY_DAG(NAME, DAGN) \
+ case ISD::DAGN: \
+ Opcode = ISD::STRICT_##DAGN; \
+ break;
+#include "llvm/IR/FloatingPointOps.def"
+ }
+
+ SDLoc sdl = getCurSDLoc();
+ SDValue Result = DAG.getNode(Opcode, sdl, NodeVT, Operands, Flags);
+ if (HasChain)
+ pushOutChain(Result, EB);
+
+ SDValue FPResult = Result.getValue(0);
+ setValue(&I, FPResult);
+ return true;
+}
+
void SelectionDAGBuilder::visitCall(const CallInst &I) {
// Handle inline assembly differently.
if (I.isInlineAsm()) {
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
index c251755ee7064..85619424054d6 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h
@@ -367,6 +367,8 @@ class SelectionDAGBuilder {
/// Evict any dangling debug information, attempting to salvage it first.
void resolveOrClearDbgInfo();
+ void pushOutChain(SDValue Result, fp::ExceptionBehavior EB);
+
SDValue getValue(const Value *V);
SDValue getNonRegisterValue(const Value *V);
@@ -611,6 +613,7 @@ class SelectionDAGBuilder {
bool visitStrNLenCall(const CallInst &I);
bool visitUnaryFloatCall(const CallInst &I, unsigned Opcode);
bool visitBinaryFloatCall(const CallInst &I, unsigned Opcode);
+ bool visitFPOperation(const CallInst &I, unsigned Opcode);
void visitAtomicLoad(const LoadInst &I);
void visitAtomicStore(const StoreInst &I);
void visitLoadFromSwiftError(const LoadInst &I);
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index dc6d599fa9585..a6b07d17f95c4 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -4388,6 +4388,31 @@ void AssemblyWriter::printInfoComment(const Value &V) {
if (PrintInstAddrs)
Out << " ; " << &V;
+
+ if (auto *CI = dyn_cast<CallInst>(&V))
+ if (Intrinsic::ID IID = CI->getIntrinsicID())
+ if (IntrinsicInst::isFloatingPointOperation(IID))
+ if (const BasicBlock *BB = CI->getParent())
+ if (const Function *F = BB->getParent())
+ if (F->hasFnAttribute(Attribute::StrictFP)) {
+ MemoryEffects ME = CI->getMemoryEffects();
+ ModRefInfo MR = ME.getModRef(IRMemLocation::InaccessibleMem);
+ Out << " ; fpe=[";
+ switch (MR) {
+ case ModRefInfo::NoModRef:
+ break;
+ case ModRefInfo::Ref:
+ Out << "r";
+ break;
+ case ModRefInfo::Mod:
+ Out << "w";
+ break;
+ case ModRefInfo::ModRef:
+ Out << "rw";
+ break;
+ }
+ Out << "]";
+ }
}
static void maybePrintCallAddrSpace(const Value *Operand, const Instruction *I,
diff --git a/llvm/lib/IR/FPEnv.cpp b/llvm/lib/IR/FPEnv.cpp
index d48bebeb6d2b7..3a24c379faf65 100644
--- a/llvm/lib/IR/FPEnv.cpp
+++ b/llvm/lib/IR/FPEnv.cpp
@@ -36,12 +36,12 @@ std::optional<RoundingMode> convertStrToRoundingMode(StringRef RoundingArg) {
std::optional<RoundingMode> convertBundleToRoundingMode(StringRef RoundingArg) {
return StringSwitch<std::optional<RoundingMode>>(RoundingArg)
- .Case("dyn", RoundingMode::Dynamic)
- .Case("rte", RoundingMode::NearestTiesToEven)
- .Case("rmm", RoundingMode::NearestTiesToAway)
- .Case("rtn", RoundingMode::TowardNegative)
- .Case("rtp", RoundingMode::TowardPositive)
- .Case("rtz", RoundingMode::TowardZero)
+ .Case("dynamic", RoundingMode::Dynamic)
+ .Case("tonearest", RoundingMode::NearestTiesToEven)
+ .Case("tonearestaway", RoundingMode::NearestTiesToAway)
+ .Case("downward", RoundingMode::TowardNegative)
+ .Case("upward", RoundingMode::TowardPositive)
+ .Case("towardzero", RoundingMode::TowardZero)
.Default(std::nullopt);
}
@@ -76,22 +76,22 @@ std::optional<StringRef> convertRoundingModeToBundle(RoundingMode UseRounding) {
std::optional<StringRef> RoundingStr;
switch (UseRounding) {
case RoundingMode::Dynamic:
- RoundingStr = "dyn";
+ RoundingStr = "dynamic";
break;
case RoundingMode::NearestTiesToEven:
- RoundingStr = "rte";
+ RoundingStr = "tonearest";
break;
case RoundingMode::NearestTiesToAway:
- RoundingStr = "rmm";
+ RoundingStr = "tonearestaway";
break;
case RoundingMode::TowardNegative:
- RoundingStr = "rtn";
+ RoundingStr = "downward";
break;
case RoundingMode::TowardPositive:
- RoundingStr = "rtp";
+ RoundingStr = "upward";
break;
case RoundingMode::TowardZero:
- RoundingStr = "rtz";
+ RoundingStr = "towardzero";
break;
default:
break;
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index 8727684ee3c82..6e188f12b1834 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -116,49 +116,54 @@ CallInst *IRBuilderBase::CreateCall(FunctionType *FTy, Value *Callee,
ArrayRef<Value *> Args,
ArrayRef<OperandBundleDef> OpBundles,
const Twine &Name, MDNode *FPMathTag) {
- assert(std::count_if(OpBundles.begin(), OpBundles.end(),
- [](const OperandBundleDef &Item) {
- return Item.getTag() == "fp.control";
- }) <= 1);
- assert(std::count_if(OpBundles.begin(), OpBundles.end(),
- [](const OperandBundleDef &Item) {
- return Item.getTag() == "fp.except";
- }) <= 1);
-
+ // Operand bundles can be extended by floating-point bundles. In this case
+ // we need a copy of OpBundles, because ArrayRef is immutable.
ArrayRef<OperandBundleDef> ActualBundlesRef = OpBundles;
SmallVector<OperandBundleDef, 2> ActualBundles;
- // If the builder is in strictfp mode and has non-default options (like
- // non-dynamic rounding), add corresponding operand bundle. If such bundle is
- // already present, assume it overwrites defaults.
bool NeedUpdateMemoryEffects = false;
- if (IsFPConstrained) {
- if (const auto *Func = dyn_cast<Function>(Callee)) {
- if (Intrinsic::ID ID = Func->getIntrinsicID()) {
- if (IntrinsicInst::isFloatingPointOperation(ID)) {
+ if (const auto *Func = dyn_cast<Function>(Callee))
+ if (Intrinsic::ID ID = Func->getIntrinsicID())
+ if (IntrinsicInst::isFloatingPointOperation(ID)) {
+ // If the builder has non-default floating-point options, add
+ // corresponding operand bundle unless a bundle with such tag is already
+ // present.
+ bool NeedRounding;
+ bool NeedExceptions;
+ if (IsFPConstrained) {
+ NeedRounding = DefaultConstrainedRounding != RoundingMode::Dynamic;
+ NeedExceptions = DefaultConstrainedExcept != fp::ebStrict;
+ } else {
+ NeedRounding =
+ DefaultConstrainedRounding != RoundingMode::NearestTiesToEven;
+ NeedExceptions = false;
+ assert(DefaultConstrainedExcept == fp::ebIgnore &&
+ "FP exception in default mode must be ignored");
+ }
+ if (NeedRounding || NeedExceptions) {
+ for (const auto &Bundle : OpBundles) {
+ if (NeedRounding && Bundle.getTag() == "fp.round")
+ NeedRounding = false;
+ if (NeedExceptions && Bundle.getTag() == "fp.except")
+ NeedExceptions = false;
+ }
+ if (NeedRounding || NeedExceptions) {
+ ActualBundles.append(OpBundles.begin(), OpBundles.end());
+ if (NeedRounding)
+ createRoundingBundle(ActualBundles, DefaultConstrainedRounding);
+ if (NeedExceptions)
+ createExceptionBundle(ActualBundles, DefaultConstrainedExcept);
+ ActualBundlesRef = ActualBundles;
+ }
+ }
+ if (IsFPConstrained) {
MemoryEffects FME = Func->getMemoryEffects();
NeedUpdateMemoryEffects = !FME.doesAccessInaccessibleMem();
- bool NeedRound = DefaultConstrainedRounding != RoundingMode::Dynamic;
- bool NeedExcept = DefaultConstrainedExcept != fp::ebStrict;
- for (const auto &Item : OpBundles) {
- if (NeedRound && Item.getTag() == "fp.control")
- NeedRound = false;
- else if (NeedExcept && Item.getTag() == "fp.except")
- NeedExcept = false;
- ActualBundles.push_back(Item);
- }
- if (NeedRound)
- createFPRoundingBundle(ActualBundles, DefaultConstrainedRounding);
- if (NeedExcept)
- createFPExceptionBundle(ActualBundles, DefaultConstrainedExcept);
- ActualBundlesRef = ActualBundles;
}
}
- }
- }
// If the call accesses FPE, update memory effects accordingly.
- CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
+ CallInst *CI = CallInst::Create(FTy, Callee, Args, ActualBundlesRef);
if (NeedUpdateMemoryEffects) {
MemoryEffects ME = MemoryEffects::inaccessibleMemOnly();
auto A = Attribute::getWithMemoryEffects(getContext(), ME);
@@ -1341,18 +1346,14 @@ CallInst *IRBuilderBase::CreateDereferenceableAssumption(Value *PtrValue,
{DereferenceableOpB});
}
-void IRBuilderBase::createFPRoundingBundle(
- SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<RoundingMode> Rounding) {
- addFPRoundingBundle(Context, Bundles,
- Rounding.value_or(DefaultConstrainedRounding));
+void IRBuilderBase::createRoundingBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles, RoundingMode RM) {
+ addRoundingBundle(Context, Bundles, RM);
}
-void IRBuilderBase::createFPExceptionBundle(
- SmallVectorImpl<OperandBundleDef> &Bundles,
- std::optional<fp::ExceptionBehavior> Except) {
- addFPExceptionBundle(Context, Bundles,
- Except.value_or(DefaultConstrainedExcept));
+void IRBuilderBase::createExceptionBundle(
+ SmallVectorImpl<OperandBundleDef> &Bundles, fp::ExceptionBehavior Except) {
+ addExceptionBundle(Context, Bundles, Except);
}
IRBuilderDefaultInserter::~IRBuilderDefaultInserter() = default;
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index 3adce4b45281a..5eb3ba8f776bf 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -620,7 +620,7 @@ bool CallBase::hasReadingOperandBundles() const {
// ptrauth) forces a callsite to be at least readonly.
return hasOperandBundlesOtherThan(
{LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
- LLVMContext::OB_fp_control, LLVMContext::OB_fp_except,
+ LLVMContext::OB_fp_round, LLVMContext::OB_fp_except,
LLVMContext::OB_convergencectrl}) &&
getIntrinsicID() != Intrinsic::assume;
}
@@ -629,25 +629,23 @@ bool CallBase::hasClobberingOperandBundles() const {
return hasOperandBundlesOtherThan(
{LLVMContext::OB_deopt, LLVMContext::OB_funclet,
LLVMContext::OB_ptrauth, LLVMContext::OB_kcfi,
- LLVMContext::OB_fp_control, LLVMContext::OB_fp_except,
+ LLVMContext::OB_fp_round, LLVMContext::OB_fp_except,
LLVMContext::OB_convergencectrl}) &&
getIntrinsicID() != Intrinsic::assume;
}
RoundingMode CallBase::getRoundingMode() const {
- // Try reading rounding mode from FP bundle.
- std::optional<RoundingMode> RM;
- if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_control)) {
+ // Try reading rounding mode from operand bundle.
+ if (auto RoundingBundle = getOperandBundle(LLVMContext::OB_fp_round)) {
Value *V = RoundingBundle->Inputs.front();
Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
- RM = convertBundleToRoundingMode(cast<MDString>(MD)->getString());
+ if (auto RM = convertBundleToRoundingMode(cast<MDString>(MD)->getString()))
+ return *RM;
}
- if (RM)
- return *RM;
- // No FP bundle, try to guess from the current mode.
- if (getParent())
- if (auto *F = getFunction(); F)
+ // No FP bundle, try to guess from attributes of the current function.
+ if (const BasicBlock *BB = getParent())
+ if (const Function *F = BB->getParent())
return F->getAttributes().hasFnAttr(Attribute::StrictFP)
? RoundingMode::Dynamic
: RoundingMode::NearestTiesToEven;
@@ -658,18 +656,17 @@ RoundingMode CallBase::getRoundingMode() const {
fp::ExceptionBehavior CallBase::getExceptionBehavior() const {
// Try determining exception behavior from FP bundle.
- std::optional<fp::ExceptionBehavior> EB;
if (auto ExceptionBundle = getOperandBundle(LLVMContext::OB_fp_except)) {
Value *V = ExceptionBundle->Inputs.front();
Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
- EB = convertBundleToExceptionBehavior(cast<MDString>(MD)->getString());
+ if (auto EB =
+ convertBundleToExceptionBehavior(cast<MDString>(MD)->getString()))
+ return *EB;
}
- if (EB)
- return *EB;
- // No FP bundle, try to guess from the current mode.
- if (getParent())
- if (auto *F = getFunction(); F)
+ // No FP bundle, try to guess from attributes of the current function.
+ if (const BasicBlock *BB = getParent())
+ if (const Function *F = BB->getParent())
return F->getAttributes().hasFnAttr(Attribute::StrictFP) ? fp::ebStrict
: fp::ebIgnore;
@@ -681,10 +678,12 @@ MemoryEffects CallBase::getFloatingPointMemoryEffects() const {
if (Intrinsic::ID IntrID = getIntrinsicID())
if (const BasicBlock *BB = getParent())
if (const Function *F = BB->getParent())
- if (F->hasFnAttribute(Attribute::StrictFP))
- if (IntrinsicInst::isFloatingPointOperation(IntrID)) {
- return MemoryEffects::inaccessibleMemOnly();
- }
+ if (F->hasFnAttribute(Attribute::StrictFP) &&
+ IntrinsicInst::isFloatingPointOperation(IntrID)) {
+ // Floating-point operations in strictfp function always have side
+ // effect at least because they can raise exceptions.
+ return MemoryEffects::inaccessibleMemOnly();
+ }
return MemoryEffects::none();
}
@@ -797,19 +796,29 @@ bool CallBase::hasArgumentWithAdditionalReturnCaptureComponents() const {
return false;
}
-void llvm::addFPRoundingBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- RoundingMode Rounding) {
+void llvm::addRoundingBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ RoundingMode Rounding) {
+ assert(std::find_if(Bundles.begin(), Bundles.end(),
+ [](const OperandBundleDef &B) {
+ return B.getTag() == "fp.round";
+ }) == Bundles.end() &&
+ "Bundle 'fp.round' already exists");
std::optional<StringRef> RndStr = convertRoundingModeToBundle(Rounding);
assert(RndStr && "Garbage rounding mode!");
auto *RoundingMDS = MDString::get(Ctx, *RndStr);
auto *RM = MetadataAsValue::get(Ctx, RoundingMDS);
- Bundles.emplace_back("fp.control", RM);
+ Bundles.emplace_back("fp.round", RM);
}
-void llvm::addFPExceptionBundle(LLVMContext &Ctx,
- SmallVectorImpl<OperandBundleDef> &Bundles,
- fp::ExceptionBehavior Except) {
+void llvm::addExceptionBundle(LLVMContext &Ctx,
+ SmallVectorImpl<OperandBundleDef> &Bundles,
+ fp::ExceptionBehavior Except) {
+ assert(std::find_if(Bundles.begin(), Bundles.end(),
+ [](const OperandBundleDef &B) {
+ return B.getTag() == "fp.except";
+ }) == Bundles.end() &&
+ "Bundle 'fp.except' already exists");
std::optional<StringRef> ExcStr = convertExceptionBehaviorToBundle(Except);
assert(ExcStr && "Garbage exception behavior!");
auto *ExceptMDS = MDString::get(Ctx, *ExcStr);
diff --git a/llvm/lib/IR/LLVMContext.cpp b/llvm/lib/IR/LLVMContext.cpp
index 5008f9a57bc16..bc985078c03f4 100644
--- a/llvm/lib/IR/LLVMContext.cpp
+++ b/llvm/lib/IR/LLVMContext.cpp
@@ -53,8 +53,8 @@ static StringRef knownBundleName(unsigned BundleTagID) {
return "kcfi";
case LLVMContext::OB_convergencectrl:
return "convergencectrl";
- case LLVMContext::OB_fp_control:
- return "fp.control";
+ case LLVMContext::OB_fp_round:
+ return "fp.round";
case LLVMContext::OB_fp_except:
return "fp.except";
default:
@@ -86,9 +86,9 @@ LLVMContext::LLVMContext() : pImpl(new LLVMContextImpl(*this)) {
assert(Entry->second == BundleTagID && "operand bundle id drifted!");
}
- auto *RoundingEntry = pImpl->getOrInsertBundleTag("fp.control");
- assert(RoundingEntry->second == LLVMContext::OB_fp_control &&
- "fp.control operand bundle id drifted!");
+ auto *RoundingEntry = pImpl->getOrInsertBundleTag("fp.round");
+ assert(RoundingEntry->second == LLVMContext::OB_fp_round &&
+ "fp.round operand bundle id drifted!");
(void)RoundingEntry;
auto *ExceptionEntry = pImpl->getOrInsertBundleTag("fp.except");
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 72c6dbb72f4c5..5d96f6d63f25d 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -3836,7 +3836,7 @@ void Verifier::visitCallBase(CallBase &Call) {
FoundGCTransitionBundle = false, FoundCFGuardTargetBundle = false,
FoundPreallocatedBundle = false, FoundGCLiveBundle = false,
FoundPtrauthBundle = false, FoundKCFIBundle = false,
- FoundAttachedCallBundle = false, FoundFpeControlBundle = false,
+ FoundAttachedCallBundle = false, FoundFpeRoundBundle = false,
FoundFpeExceptBundle = false;
for (unsigned i = 0, e = Call.getNumOperandBundles(); i < e; ++i) {
OperandBundleUse BU = Call.getOperandBundleAt(i);
@@ -3900,30 +3900,29 @@ void Verifier::visitCallBase(CallBase &Call) {
"Multiple \"clang.arc.attachedcall\" operand bundles", Call);
FoundAttachedCallBundle = true;
verifyAttachedCallBundle(Call, BU);
- } else if (Tag == LLVMContext::OB_fp_control) {
- Check(!FoundFpeControlBundle, "Multiple \"fp.control\" operand bundles",
+ } else if (Tag == LLVMContext::OB_fp_round) {
+ Check(!FoundFpeRoundBundle, "Multiple \"fp.round\" operand bundles",
Call);
bool FoundRoundingMode = false;
for (auto &U : BU.Inputs) {
Value *V = U.get();
Check(isa<MetadataAsValue>(V),
- "Value of a \"fp.control\" bundle operand must be a metadata",
+ "Value of a \"fp.round\" bundle operand must be a metadata",
Call);
Metadata *MD = cast<MetadataAsValue>(V)->getMetadata();
Check(isa<MDString>(MD),
- "Value of a \"fp.control\" bundle operand must be a string",
- Call);
+ "Value of a \"fp.round\" bundle operand must be a string", Call);
StringRef Item = cast<MDString>(MD)->getString();
if (convertBundleToRoundingMode(Item)) {
Check(!FoundRoundingMode, "Rounding mode is specified more that once",
Call);
FoundRoundingMode = true;
} else {
- CheckFailed("Unrecognized value in \"fp.control\" bundle operand",
+ CheckFailed("Unrecognized value in \"fp.round\" bundle operand",
Call);
}
}
- FoundFpeControlBundle = true;
+ FoundFpeRoundBundle = true;
} else if (Tag == LLVMContext::OB_fp_except) {
Check(!FoundFpeExceptBundle, "Multiple \"fp.except\" operand bundles",
Call);
@@ -3941,6 +3940,15 @@ void Verifier::visitCallBase(CallBase &Call) {
"Value of a \"fp.except\" bundle operand is not a correct exception "
"behavior",
Call);
+ if (EB && *EB != fp::ebIgnore)
+ if (const BasicBlock *BB = Call.getParent())
+ if (const Function *F = BB->getParent()) {
+ bool StrictFP = F->hasFnAttribute(Attribute::StrictFP);
+ Check(StrictFP,
+ "Value of a \"fp.except\" bundle operand in "
+ "default mode must be \"ignore\"",
+ Call);
+ }
FoundFpeExceptBundle = true;
}
}
diff --git a/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll b/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll
index 13f890ae6e191..35c066afb9c13 100644
--- a/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll
+++ b/llvm/test/CodeGen/X86/fp-strict-scalar-round.ll
@@ -250,6 +250,46 @@ define float @ftrunc32(float %f) #0 {
ret float %res
}
+define float @ftrunc32ob(float %f) #0 {
+; SSE41-X86-LABEL: ftrunc32ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; SSE41-X86-NEXT: roundss $11, %xmm0, %xmm0
+; SSE41-X86-NEXT: movss %xmm0, (%esp)
+; SSE41-X86-NEXT: flds (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: popl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: ftrunc32ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundss $11, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: ftrunc32ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX-X86-NEXT: vroundss $11, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovss %xmm0, (%esp)
+; AVX-X86-NEXT: flds (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: popl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: ftrunc32ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundss $11, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call float @llvm.trunc.f32(float %f) [ "fp.except"(metadata !"strict") ]
+ ret float %res
+}
+
define double @ftruncf64(double %f) #0 {
; SSE41-X86-LABEL: ftruncf64:
; SSE41-X86: # %bb.0:
@@ -303,6 +343,58 @@ define double @ftruncf64(double %f) #0 {
ret double %res
}
+define double @ftruncf64ob(double %f) #0 {
+; SSE41-X86-LABEL: ftruncf64ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: .cfi_offset %ebp, -8
+; SSE41-X86-NEXT: movl %esp, %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_register %ebp
+; SSE41-X86-NEXT: andl $-8, %esp
+; SSE41-X86-NEXT: subl $8, %esp
+; SSE41-X86-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
+; SSE41-X86-NEXT: roundsd $11, %xmm0, %xmm0
+; SSE41-X86-NEXT: movsd %xmm0, (%esp)
+; SSE41-X86-NEXT: fldl (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: movl %ebp, %esp
+; SSE41-X86-NEXT: popl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa %esp, 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: ftruncf64ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundsd $11, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: ftruncf64ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: .cfi_offset %ebp, -8
+; AVX-X86-NEXT: movl %esp, %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_register %ebp
+; AVX-X86-NEXT: andl $-8, %esp
+; AVX-X86-NEXT: subl $8, %esp
+; AVX-X86-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX-X86-NEXT: vroundsd $11, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovsd %xmm0, (%esp)
+; AVX-X86-NEXT: fldl (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: movl %ebp, %esp
+; AVX-X86-NEXT: popl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa %esp, 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: ftruncf64ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundsd $11, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call double @llvm.trunc.f64(double %f) [ "fp.except"(metadata !"strict") ]
+ ret double %res
+}
+
define float @frint32(float %f) #0 {
; SSE41-X86-LABEL: frint32:
; SSE41-X86: # %bb.0:
@@ -441,6 +533,46 @@ define float @fnearbyint32(float %f) #0 {
ret float %res
}
+define float @fnearbyint32ob(float %f) #0 {
+; SSE41-X86-LABEL: fnearbyint32ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; SSE41-X86-NEXT: roundss $12, %xmm0, %xmm0
+; SSE41-X86-NEXT: movss %xmm0, (%esp)
+; SSE41-X86-NEXT: flds (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: popl %eax
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: fnearbyint32ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundss $12, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: fnearbyint32ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX-X86-NEXT: vroundss $12, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovss %xmm0, (%esp)
+; AVX-X86-NEXT: flds (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: popl %eax
+; AVX-X86-NEXT: .cfi_def_cfa_offset 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: fnearbyint32ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundss $12, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call float @llvm.nearbyint.f32(float %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret float %res
+}
+
define double @fnearbyintf64(double %f) #0 {
; SSE41-X86-LABEL: fnearbyintf64:
; SSE41-X86: # %bb.0:
@@ -495,6 +627,58 @@ define double @fnearbyintf64(double %f) #0 {
ret double %res
}
+define double @fnearbyint64ob(double %f) #0 {
+; SSE41-X86-LABEL: fnearbyint64ob:
+; SSE41-X86: # %bb.0:
+; SSE41-X86-NEXT: pushl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_offset 8
+; SSE41-X86-NEXT: .cfi_offset %ebp, -8
+; SSE41-X86-NEXT: movl %esp, %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa_register %ebp
+; SSE41-X86-NEXT: andl $-8, %esp
+; SSE41-X86-NEXT: subl $8, %esp
+; SSE41-X86-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
+; SSE41-X86-NEXT: roundsd $12, %xmm0, %xmm0
+; SSE41-X86-NEXT: movsd %xmm0, (%esp)
+; SSE41-X86-NEXT: fldl (%esp)
+; SSE41-X86-NEXT: wait
+; SSE41-X86-NEXT: movl %ebp, %esp
+; SSE41-X86-NEXT: popl %ebp
+; SSE41-X86-NEXT: .cfi_def_cfa %esp, 4
+; SSE41-X86-NEXT: retl
+;
+; SSE41-X64-LABEL: fnearbyint64ob:
+; SSE41-X64: # %bb.0:
+; SSE41-X64-NEXT: roundsd $12, %xmm0, %xmm0
+; SSE41-X64-NEXT: retq
+;
+; AVX-X86-LABEL: fnearbyint64ob:
+; AVX-X86: # %bb.0:
+; AVX-X86-NEXT: pushl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_offset 8
+; AVX-X86-NEXT: .cfi_offset %ebp, -8
+; AVX-X86-NEXT: movl %esp, %ebp
+; AVX-X86-NEXT: .cfi_def_cfa_register %ebp
+; AVX-X86-NEXT: andl $-8, %esp
+; AVX-X86-NEXT: subl $8, %esp
+; AVX-X86-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX-X86-NEXT: vroundsd $12, %xmm0, %xmm0, %xmm0
+; AVX-X86-NEXT: vmovsd %xmm0, (%esp)
+; AVX-X86-NEXT: fldl (%esp)
+; AVX-X86-NEXT: wait
+; AVX-X86-NEXT: movl %ebp, %esp
+; AVX-X86-NEXT: popl %ebp
+; AVX-X86-NEXT: .cfi_def_cfa %esp, 4
+; AVX-X86-NEXT: retl
+;
+; AVX-X64-LABEL: fnearbyint64ob:
+; AVX-X64: # %bb.0:
+; AVX-X64-NEXT: vroundsd $12, %xmm0, %xmm0, %xmm0
+; AVX-X64-NEXT: retq
+ %res = call double @llvm.nearbyint.f64(double %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret double %res
+}
+
define float @fround32(float %f) #0 {
; SSE41-X86-LABEL: fround32:
; SSE41-X86: # %bb.0:
diff --git a/llvm/test/CodeGen/X86/vec-strict-512.ll b/llvm/test/CodeGen/X86/vec-strict-512.ll
index 2cafd74af4953..5bd78d99c078b 100644
--- a/llvm/test/CodeGen/X86/vec-strict-512.ll
+++ b/llvm/test/CodeGen/X86/vec-strict-512.ll
@@ -231,6 +231,15 @@ define <16 x float> @strict_vector_ftrunc_v16f32(<16 x float> %f) #0 {
ret <16 x float> %res
}
+define <16 x float> @strict_vector_ftrunc_v16f32ob(<16 x float> %f) #0 {
+; CHECK-LABEL: strict_vector_ftrunc_v16f32ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscaleps $11, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <16 x float> @llvm.trunc.v16f32(<16 x float> %f) [ "fp.except"(metadata !"strict") ]
+ ret <16 x float> %res
+}
+
define <8 x double> @strict_vector_ftrunc_v8f64(<8 x double> %f) #0 {
; CHECK-LABEL: strict_vector_ftrunc_v8f64:
; CHECK: # %bb.0:
@@ -240,6 +249,15 @@ define <8 x double> @strict_vector_ftrunc_v8f64(<8 x double> %f) #0 {
ret <8 x double> %res
}
+define <8 x double> @strict_vector_ftrunc_v8f64ob(<8 x double> %f) #0 {
+; CHECK-LABEL: strict_vector_ftrunc_v8f64ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscalepd $11, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <8 x double> @llvm.trunc.v8f64(<8 x double> %f) [ "fp.except"(metadata !"strict") ]
+ ret <8 x double> %res
+}
+
define <16 x float> @strict_vector_frint_v16f32(<16 x float> %f) #0 {
; CHECK-LABEL: strict_vector_frint_v16f32:
; CHECK: # %bb.0:
@@ -270,6 +288,15 @@ define <16 x float> @strict_vector_fnearbyint_v16f32(<16 x float> %f) #0 {
ret <16 x float> %res
}
+define <16 x float> @strict_vector_fnearbyint_v16f32ob(<16 x float> %f) #0 {
+; CHECK-LABEL: strict_vector_fnearbyint_v16f32ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscaleps $12, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <16 x float> @llvm.nearbyint.v16f32(<16 x float> %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret <16 x float> %res
+}
+
define <8 x double> @strict_vector_fnearbyint_v8f64(<8 x double> %f) #0 {
; CHECK-LABEL: strict_vector_fnearbyint_v8f64:
; CHECK: # %bb.0:
@@ -280,4 +307,13 @@ define <8 x double> @strict_vector_fnearbyint_v8f64(<8 x double> %f) #0 {
ret <8 x double> %res
}
+define <8 x double> @strict_vector_fnearbyint_v8f64ob(<8 x double> %f) #0 {
+; CHECK-LABEL: strict_vector_fnearbyint_v8f64ob:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vrndscalepd $12, %zmm0, %zmm0
+; CHECK-NEXT: ret{{[l|q]}}
+ %res = call <8 x double> @llvm.nearbyint.v8f64(<8 x double> %f) [ "fp.round"(metadata !"dynamic"), "fp.except"(metadata !"strict") ]
+ ret <8 x double> %res
+}
+
attributes #0 = { strictfp }
diff --git a/llvm/test/Verifier/fp-intrinsics.ll b/llvm/test/Verifier/fp-intrinsics.ll
index 1e1df1b0da96a..f96d519870aaa 100644
--- a/llvm/test/Verifier/fp-intrinsics.ll
+++ b/llvm/test/Verifier/fp-intrinsics.ll
@@ -51,48 +51,48 @@ entry:
ret double %fadd
}
-; Test multiple fp.control bundles.
-; CHECK-NEXT: Multiple "fp.control" operand bundles
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+; Test multiple fp.round bundles.
+; CHECK-NEXT: Multiple "fp.round" operand bundles
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
define double @f6(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz"), "fp.control"(metadata !"rtz") ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"towardzero"), "fp.round"(metadata !"towardzero") ]
ret double %ftrunc
}
-; Test fp.control bundle that has more than one rounding mode specification.
+; Test fp.round bundle that has more than one rounding mode specification.
; CHECK-NEXT: Rounding mode is specified more that once
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
define double @f7(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"rtz", metadata !"rte") ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"towardzero", metadata !"tonearest") ]
ret double %ftrunc
}
-; Test fp.control bundle that has non-metadata operand.
-; CHECK-NEXT: Value of a "fp.control" bundle operand must be a metadata
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(i32 0) ]
+; Test fp.round bundle that has non-metadata operand.
+; CHECK-NEXT: Value of a "fp.round" bundle operand must be a metadata
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(i32 0) ]
define double @f8(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(i32 0) ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(i32 0) ]
ret double %ftrunc
}
-; Test fp.control bundle that has non-string operand.
-; CHECK-NEXT: Value of a "fp.control" bundle operand must be a string
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata i64 3) ]
+; Test fp.round bundle that has non-string operand.
+; CHECK-NEXT: Value of a "fp.round" bundle operand must be a string
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata i64 3) ]
define double @f9(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !{i64 3}) ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !{i64 3}) ]
ret double %ftrunc
}
-; Test fp.control bundle that specifies incorrect value.
-; CHECK-NEXT: Unrecognized value in "fp.control" bundle operand
-; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.control"(metadata !"qqq") ]
+; Test fp.round bundle that specifies incorrect value.
+; CHECK-NEXT: Unrecognized value in "fp.round" bundle operand
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) #{{[0-9]+}} [ "fp.round"(metadata !"qqq") ]
define double @f10(double %a) #0 {
entry:
- %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.control"(metadata !"qqq") ]
+ %ftrunc = call double @llvm.trunc.f64(double %a) #0 [ "fp.round"(metadata !"qqq") ]
ret double %ftrunc
}
@@ -141,5 +141,14 @@ entry:
ret double %ftrunc
}
+; Test fp.except bundle in default mode cannot be other than "ignore".
+; CHECK-NEXT: Value of a "fp.except" bundle operand in default mode must be "ignore"
+; CHECK-NEXT: %ftrunc = call double @llvm.trunc.f64(double %a) [ "fp.except"(metadata !"strict") ]
+define double @f16(double %a) {
+entry:
+ %ftrunc = call double @llvm.trunc.f64(double %a) [ "fp.except"(metadata !"strict") ]
+ ret double %ftrunc
+}
+
attributes #0 = { strictfp }
diff --git a/llvm/unittests/IR/IRBuilderTest.cpp b/llvm/unittests/IR/IRBuilderTest.cpp
index a0a0d8ced8fe6..7b552c643620d 100644
--- a/llvm/unittests/IR/IRBuilderTest.cpp
+++ b/llvm/unittests/IR/IRBuilderTest.cpp
@@ -272,7 +272,9 @@ TEST_F(IRBuilderTest, ConstrainedFP) {
// See if we get constrained intrinsics instead of non-constrained
// instructions.
- Builder.setIsFPConstrained(true);
+ Builder.setDefaultConstrainedRounding(RoundingMode::Dynamic);
+ Builder.setDefaultConstrainedExcept(fp::ebStrict);
+ Builder.setIsFPConstrained(true, false);
auto Parent = BB->getParent();
Parent->addFnAttr(Attribute::StrictFP);
@@ -470,7 +472,7 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
{
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -480,13 +482,13 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
}
// Check call with FP bundles, rounding is set to default value.
- // nearbyint(%x) [ "fp.control" (metadata !"rte") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"tonearest") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -499,10 +501,10 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
// nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -513,14 +515,15 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
// Check call with FP bundles, both rounding mode and exception behavior are
// set.
- // nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"tonearest"),
+ // "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -528,6 +531,40 @@ TEST_F(IRBuilderTest, FPBundlesDefault) {
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
+
+ // If the builder object specifies a rounding mode, the resulting call will
+ // include the corresponding "fp.round" operand bundle.
+ {
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardNegative);
+ SmallVector<OperandBundleDef, 1> Bundles;
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::TowardNegative, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
+
+ // If the builder object specifies a rounding mode but the operand bundles
+ // already contain an "fp.round" bundle, the builder's specified mode
+ // is ignored.
+ {
+ Builder.setDefaultConstrainedRounding(RoundingMode::TowardNegative);
+ SmallVector<OperandBundleDef, 1> Bundles;
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
+ Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
+ auto *I = cast<IntrinsicInst>(V);
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
+ EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
+ EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
+ EXPECT_EQ(fp::ebIgnore, I->getExceptionBehavior());
+ MemoryEffects ME = I->getMemoryEffects();
+ EXPECT_TRUE(ME.doesNotAccessMemory());
+ }
}
TEST_F(IRBuilderTest, FPBundlesStrict) {
@@ -549,7 +586,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
{
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -559,13 +596,13 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
}
// Check call with FP bundles, with default (dynamic) rounding mode
- // nearbyint(%x) [ "fp.control" (metadata !"dyn") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"dynamic") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::Dynamic);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -575,13 +612,13 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
}
// Check call with FP bundles, with specific rounding mode
- // nearbyint(%x) [ "fp.control" (metadata !"rtz") ]
+ // nearbyint(%x) [ "fp.round" (metadata !"towardzero") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::TowardZero);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::TowardZero, I->getRoundingMode());
@@ -594,10 +631,10 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
// nearbyint(%x) [ "fp.except" (metadata !"strict") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebStrict);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebStrict);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -610,10 +647,10 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
// nearbyint(%x) [ "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::Dynamic, I->getRoundingMode());
@@ -623,15 +660,15 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
}
// Check call with both FP bundles.
- // nearbyint(%x) [ "fp.control" (metadata !"rtz"),
+ // nearbyint(%x) [ "fp.round" (metadata !"towardzero"),
// "fp.except" (metadata !"ignore") ]
{
SmallVector<OperandBundleDef, 1> Bundles;
- llvm::addFPRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
- llvm::addFPExceptionBundle(Ctx, Bundles, fp::ebIgnore);
+ llvm::addRoundingBundle(Ctx, Bundles, RoundingMode::NearestTiesToEven);
+ llvm::addExceptionBundle(Ctx, Bundles, fp::ebIgnore);
Value *V = Builder.CreateCall(Fn, {FnArg}, Bundles);
auto *I = cast<IntrinsicInst>(V);
- EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
EXPECT_TRUE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
EXPECT_EQ(Intrinsic::nearbyint, I->getIntrinsicID());
EXPECT_EQ(RoundingMode::NearestTiesToEven, I->getRoundingMode());
@@ -648,7 +685,7 @@ TEST_F(IRBuilderTest, FPBundlesStrict) {
Value *V = Builder.CreateCall(Fn, {FnArg});
auto *I = cast<IntrinsicInst>(V);
EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_except).has_value());
- EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_control).has_value());
+ EXPECT_FALSE(I->getOperandBundle(LLVMContext::OB_fp_round).has_value());
MemoryEffects ME = I->getMemoryEffects();
EXPECT_TRUE(ME.doesNotAccessMemory());
}
More information about the llvm-commits
mailing list