[PATCH] Flag to enable IEEE-754 friendly FP optimizations
Sergey Dmitrouk
sdmitrouk at accesssoftek.com
Thu Feb 5 09:56:56 PST 2015
Hello,
Thanks for reviewing this.
On Wed, Feb 04, 2015 at 10:42:35AM -0800, Mehdi Amini wrote:
> Patch 1:
>
> The comment for:
> + unsigned AllowFPExceptAccess : 1;
> mentions rounding and the comment for:
> + unsigned AllowFPRoundAccess : 1;
> mentions exceptions, they are reversed.
Updated.
> Patch 2:
>
> Is kexc or kround flag allowed to be present at the same time as fast?
Yes, currently they are. There is no particular reason for this except
for me not being able to decide whether "fast" should reset new flags or
not. Left it unrelated to each other as they are unlikely to be used at
the same time, ready to change it if you have any suggestions.
> Patch 3:
>
> isSafeToOptimizeFPOp() is not clear to me, especially how TLI->hasFloatingPointExceptions() disable any check, including rounding, while the description of this flag is "Whether the target supports or cares about preserving floating point exception behavior”, which seems orthogonal to getTarget().Options.AllowFPRoundAccess?
> I assume it is done this way to keep existing behavior? Is it transitional?
That's a mistake, it was fine until recent change for rounding when I
reordered if statements incorrectly. Thanks.
> Patch 4:
>
> isSafeToSpeculativelyExecute(), comment might deserve to be updated , I don’t think it is true anymore with respect to rounding:
> "Return true if the instruction does not have any effects besides calculating the result and does not have undefined behavior”.
Updated.
> canTrap(), again preserving the dynamic “rounding” is included in “trap”? Maybe the comment could be updated because it does not seem obvious to me.
Actually, rounding flag wasn't even used and exceptions flag was
unnecessary, so I just reverted that part. It was required with flags
being detached from instructions, but now that they are part of
fast-math flags checking constant expressions should be no-operation
(operations that go against flags must not be converted to constant
expressions).
> Patch 10: if you really want to test it, maybe the C++ unittests can do the job?
Thanks, added a unit-test and a correction for "frem" instruction.
> Patch 12: haven’t look into detail where the infinite loop occurs, but it seems like a hammer fix for the issue. The example mentioned in the test ((a + b) => (a - (-b)) => (a + b) => etc) makes me thinks that one of the two forms should be canonical and InstCombine should not make one of these two transforms. What do you think?
I chose this way as it should guarantee to work for all cases, but you
might be right in that something is wrong with pattern matching in
InstCombine, see below.
There is an implicit four step loop between InstCombiner::visitFAdd() and
InstCombiner::visitFSub():
1. FAdd: Convert A to -A.
2. FSub: Convert -A to A; switch operands.
3. FAdd: Convert B to -B.
4. FSub: Convert -B to B; switch operands.
5. Go to step 1.
Both functions try to do something about negative values, but they
actually hang in case of positive arguments as well. I couldn't find
any checks whether operands are negative, it seems to be "assumed".
Maybe dyn_castFNegVal() used to perform the check, but it doesn't anymore.
If I add `C->isNegative()` check to dyn_castFNegVal(), the issue
disappears, but I'm not sure if it's the right place. Meaning of the
comment there:
// Constants can be considered to be negated values if they can be
// folded.
if (ConstantFP *C = dyn_cast<ConstantFP>(V))
return ConstantExpr::getFNeg(C);
is unclear to me. Does it suggest that the function doesn't care about
sign of the value? In this case, checks for negative operands should be
performed outside (in visitFAdd() and visitFSub() functions).
Updated patches are attached (0000-recent-changes.diff contains latest
changes only). There is nothing for InstCombine yet and a small new
change (not counting test file) for LICM pass.
Thanks,
Sergey
-------------- next part --------------
include/llvm/Analysis/ValueTracking.h | 5 ++--
include/llvm/IR/Constant.h | 2 +-
include/llvm/IR/IRBuilder.h | 6 ++---
include/llvm/Target/TargetOptions.h | 4 +--
lib/Analysis/ValueTracking.cpp | 2 +-
lib/CodeGen/SelectionDAG/SelectionDAG.cpp | 5 ++--
lib/IR/Constants.cpp | 32 ++++--------------------
lib/Transforms/Scalar/LICM.cpp | 4 +++
test/Transforms/LICM/fpenv.ll | 30 ++++++++++++++++++++++
unittests/IR/IRBuilderTest.cpp | 41 +++++++++++++++++++++++++++++++
10 files changed, 93 insertions(+), 38 deletions(-)
diff --git a/include/llvm/Analysis/ValueTracking.h b/include/llvm/Analysis/ValueTracking.h
index c42ab01..338934b 100644
--- a/include/llvm/Analysis/ValueTracking.h
+++ b/include/llvm/Analysis/ValueTracking.h
@@ -183,8 +183,9 @@ namespace llvm {
bool onlyUsedByLifetimeMarkers(const Value *V);
/// isSafeToSpeculativelyExecute - Return true if the instruction does not
- /// have any effects besides calculating the result and does not have
- /// undefined behavior.
+ /// have any effects besides calculating the result (in a way that doesn't
+ /// lose floating-point exceptions or precision when requested via
+ /// corresponding flags) and does not have undefined behavior.
///
/// This method never returns true for an instruction that returns true for
/// mayHaveSideEffects; however, this method also does some other checks in
diff --git a/include/llvm/IR/Constant.h b/include/llvm/IR/Constant.h
index dfc5710..d26991e 100644
--- a/include/llvm/IR/Constant.h
+++ b/include/llvm/IR/Constant.h
@@ -77,7 +77,7 @@ public:
/// canTrap - Return true if evaluation of this constant could trap. This is
/// true for things like constant expressions that could divide by zero.
- bool canTrap(bool KeepExceptions = false, bool KeepRounding = false) const;
+ bool canTrap() const;
/// isThreadDependent - Return true if the value can vary between threads.
bool isThreadDependent() const;
diff --git a/include/llvm/IR/IRBuilder.h b/include/llvm/IR/IRBuilder.h
index d18b5fa..d8dac17 100644
--- a/include/llvm/IR/IRBuilder.h
+++ b/include/llvm/IR/IRBuilder.h
@@ -811,9 +811,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFRem(LC, RC, KeepExceptions, KeepRounding,
- true),
- Name);
+ if (Constant *C = Folder.CreateFRem(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFRem(LHS, RHS),
FPMathTag, FMF), Name);
}
diff --git a/include/llvm/Target/TargetOptions.h b/include/llvm/Target/TargetOptions.h
index df3f804..61bf616 100644
--- a/include/llvm/Target/TargetOptions.h
+++ b/include/llvm/Target/TargetOptions.h
@@ -130,14 +130,14 @@ namespace llvm {
/// AllowFPExceptAccess - This flag is enabled when the
/// -enable-except-access-fp-math or -enable-fpenv-access-fp-math flag is
/// specified on the command line. When this flag is off (the default), the
- /// code generator is allowed to assume rounding part of floating-point
+ /// code generator is allowed to assume exceptions part of floating-point
/// environment is not accessed by the program.
unsigned AllowFPExceptAccess : 1;
/// AllowFPRoundAccess - This flag is enabled when the
/// -enable-round-access-fp-math or -enable-fpenv-access-fp-math flag is
/// specified on the command line. When this flag is off (the default), the
- /// code generator is allowed to assume exceptions part of floating-point
+ /// code generator is allowed to assume rounding part of floating-point
/// environment is not accessed by the program.
unsigned AllowFPRoundAccess : 1;
diff --git a/lib/Analysis/ValueTracking.cpp b/lib/Analysis/ValueTracking.cpp
index 8ada931..1cb1099 100644
--- a/lib/Analysis/ValueTracking.cpp
+++ b/lib/Analysis/ValueTracking.cpp
@@ -2557,7 +2557,7 @@ bool llvm::isSafeToSpeculativelyExecute(const Value *V,
for (unsigned i = 0, e = Inst->getNumOperands(); i != e; ++i)
if (Constant *C = dyn_cast<Constant>(Inst->getOperand(i)))
- if (C->canTrap(KeepExceptions))
+ if (C->canTrap())
return false;
switch (Inst->getOpcode()) {
diff --git a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index c060832..6e4c16e 100644
--- a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -3608,12 +3608,13 @@ SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT, SDValue N1,
bool SelectionDAG::isSafeToOptimizeFPOp(APFloat::opStatus s,
APFloat::opStatus UnsafeOps) const {
+ if (getTarget().Options.AllowFPRoundAccess && (s & APFloat::opInexact))
+ return false;
+
if (!TLI->hasFloatingPointExceptions())
return true;
if (getTarget().Options.AllowFPExceptAccess)
return s == APFloat::opOK;
- if (getTarget().Options.AllowFPRoundAccess && (s & APFloat::opInexact))
- return false;
return (s & UnsafeOps) == APFloat::opOK;
}
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index cdbe424..caecae5 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -306,8 +306,7 @@ void Constant::destroyConstantImpl() {
}
static bool canTrapImpl(const Constant *C,
- SmallPtrSetImpl<const ConstantExpr *> &NonTrappingOps,
- bool KeepExceptions, bool KeepRounding) {
+ SmallPtrSetImpl<const ConstantExpr *> &NonTrappingOps) {
assert(C->getType()->isFirstClassType() && "Cannot evaluate aggregate vals!");
// The only thing that could possibly trap are constant exprs.
const ConstantExpr *CE = dyn_cast<ConstantExpr>(C);
@@ -317,35 +316,14 @@ static bool canTrapImpl(const Constant *C,
// ConstantExpr traps if any operands can trap.
for (unsigned i = 0, e = C->getNumOperands(); i != e; ++i) {
if (ConstantExpr *Op = dyn_cast<ConstantExpr>(CE->getOperand(i))) {
- if (NonTrappingOps.insert(Op).second && canTrapImpl(Op, NonTrappingOps,
- KeepExceptions,
- KeepRounding)) {
+ if (NonTrappingOps.insert(Op).second && canTrapImpl(Op, NonTrappingOps)) {
return true;
}
}
}
// Otherwise, only specific operations can trap.
- unsigned Opcode = CE->getOpcode();
- if (KeepExceptions || KeepRounding) {
- switch (Opcode) {
- case Instruction::FMul:
- case Instruction::FAdd:
- case Instruction::FSub:
- case Instruction::FDiv:
- case Instruction::FRem: {
- APFloat::opStatus S;
- if (!ConstantExpr::getFPOpExceptions(CE, S))
- return false;
-
- if (KeepExceptions && S != APFloat::opOK)
- return false;
- if (KeepRounding && (S & APFloat::opInexact))
- return false;
- }
- }
- }
- switch (Opcode) {
+ switch (CE->getOpcode()) {
default:
return false;
case Instruction::UDiv:
@@ -363,9 +341,9 @@ static bool canTrapImpl(const Constant *C,
/// canTrap - Return true if evaluation of this constant could trap. This is
/// true for things like constant expressions that could divide by zero.
-bool Constant::canTrap(bool KeepExceptions, bool KeepRounding) const {
+bool Constant::canTrap() const {
SmallPtrSet<const ConstantExpr *, 4> NonTrappingOps;
- return canTrapImpl(this, NonTrappingOps, KeepExceptions, KeepRounding);
+ return canTrapImpl(this, NonTrappingOps);
}
/// Check if C contains a GlobalValue for which Predicate is true.
diff --git a/lib/Transforms/Scalar/LICM.cpp b/lib/Transforms/Scalar/LICM.cpp
index a78d77f..c889c94 100644
--- a/lib/Transforms/Scalar/LICM.cpp
+++ b/lib/Transforms/Scalar/LICM.cpp
@@ -501,6 +501,10 @@ bool LICM::canSinkOrHoistInst(Instruction &I) {
!isa<InsertValueInst>(I))
return false;
+ if (const FPMathOperator *MathOp = dyn_cast<FPMathOperator>(&I))
+ if (MathOp->hasKeepExceptions() || MathOp->hasKeepRounding())
+ return false;
+
return isSafeToExecuteUnconditionally(I);
}
diff --git a/test/Transforms/LICM/fpenv.ll b/test/Transforms/LICM/fpenv.ll
new file mode 100644
index 0000000..6fb459c
--- /dev/null
+++ b/test/Transforms/LICM/fpenv.ll
@@ -0,0 +1,30 @@
+; RUN: opt < %s -O3 -S | FileCheck %s
+
+define void @fdiv-is-not-hoisted-out-of-loop(i32 %n) {
+; CHECK-LABEL: @fdiv-is-not-hoisted-out-of-loop(
+; CHECK: for.body:
+; CHECK: fdiv
+entry:
+ %cmp2 = icmp sgt i32 %n, 0
+ br i1 %cmp2, label %for.body.lr.ph, label %for.end
+
+for.body.lr.ph:
+ %0 = add i32 %n, -1
+ br label %for.body
+
+for.body:
+ %i.03 = phi i32 [ 0, %for.body.lr.ph ], [ %inc, %for.body ]
+ %div = fdiv kexc kround double 0.000000e+00, 0.000000e+00
+ tail call void @process(double %div)
+ %inc = add nuw nsw i32 %i.03, 1
+ %exitcond = icmp eq i32 %i.03, %0
+ br i1 %exitcond, label %for.end.loopexit, label %for.body
+
+for.end.loopexit:
+ br label %for.end
+
+for.end:
+ ret void
+}
+
+declare void @process(double) nounwind
diff --git a/unittests/IR/IRBuilderTest.cpp b/unittests/IR/IRBuilderTest.cpp
index 08a729a..18cdcb3 100644
--- a/unittests/IR/IRBuilderTest.cpp
+++ b/unittests/IR/IRBuilderTest.cpp
@@ -203,6 +203,47 @@ TEST_F(IRBuilderTest, FastMathFlags) {
}
+TEST_F(IRBuilderTest, FPEnvFlags) {
+ IRBuilder<> Builder(BB);
+ Value *F;
+
+ Constant *Huge = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e+308);
+ Constant *Big = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e+300);
+ Constant *Small = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e-300);
+ Constant *Tiny = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e-308);
+ Constant *Zero = ConstantFP::get(Type::getDoubleTy(Ctx), 0.0);
+
+ F = Builder.CreateFAdd(Huge, Huge, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFSub(Tiny, Huge, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFMul(Big, Big, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFDiv(Big, Small, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFDiv(Big, Zero, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFRem(Big, Zero, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFAdd(Huge, Huge, "", nullptr, /* KeepExceptions */ false,
+ /* KeepRounding */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFSub(Tiny, Huge, "", nullptr, /* KeepExceptions */ false,
+ /* KeepRounding */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFMul(Huge, Huge, "", nullptr, /* KeepExceptions */ false,
+ /* KeepRounding */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+}
+
TEST_F(IRBuilderTest, WrapFlags) {
IRBuilder<true, NoFolder> Builder(BB);
-------------- next part --------------
>From 560547def7eefe35ee2bb6b18e04f117f85abfde Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Thu, 13 Nov 2014 16:54:23 +0200
Subject: [PATCH 01/13] Add flags and command-line switches for FPEnv
* Two flags for TargetOptions.
* Three command-line switches to enable flags separately or at the same
time.
---
include/llvm/CodeGen/CommandFlags.h | 19 +++++++++++++++++++
include/llvm/Target/TargetOptions.h | 17 ++++++++++++++++-
2 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/include/llvm/CodeGen/CommandFlags.h b/include/llvm/CodeGen/CommandFlags.h
index 973c595..35497f6 100644
--- a/include/llvm/CodeGen/CommandFlags.h
+++ b/include/llvm/CodeGen/CommandFlags.h
@@ -124,6 +124,21 @@ EnableHonorSignDependentRoundingFPMath("enable-sign-dependent-rounding-fp-math",
cl::init(false));
cl::opt<bool>
+EnableFPEnvAccessFPMath("enable-fpenv-access-fp-math",
+ cl::desc("Disable FP math optimizations that affect FP environment access"),
+ cl::init(false));
+
+cl::opt<bool>
+EnableExceptAccessFPMath("enable-except-access-fp-math",
+ cl::desc("Disable FP math optimizations that affect FP exceptions"),
+ cl::init(false));
+
+cl::opt<bool>
+EnableRoundAccessFPMath("enable-round-access-fp-math",
+ cl::desc("Disable FP math optimizations that affect FP rounding"),
+ cl::init(false));
+
+cl::opt<bool>
GenerateSoftFloatCalls("soft-float",
cl::desc("Generate software floating point library calls"),
cl::init(false));
@@ -272,6 +287,10 @@ static inline TargetOptions InitTargetOptionsFromCodeGenFlags() {
Options.NoNaNsFPMath = EnableNoNaNsFPMath;
Options.HonorSignDependentRoundingFPMathOption =
EnableHonorSignDependentRoundingFPMath;
+ Options.AllowFPExceptAccess = EnableExceptAccessFPMath |
+ EnableFPEnvAccessFPMath;
+ Options.AllowFPRoundAccess = EnableRoundAccessFPMath |
+ EnableFPEnvAccessFPMath;
Options.UseSoftFloat = GenerateSoftFloatCalls;
if (FloatABIForCalls != FloatABI::Default)
Options.FloatABIType = FloatABIForCalls;
diff --git a/include/llvm/Target/TargetOptions.h b/include/llvm/Target/TargetOptions.h
index 9ab8242..61bf616 100644
--- a/include/llvm/Target/TargetOptions.h
+++ b/include/llvm/Target/TargetOptions.h
@@ -70,7 +70,8 @@ namespace llvm {
TargetOptions()
: PrintMachineCode(false), NoFramePointerElim(false),
LessPreciseFPMADOption(false), UnsafeFPMath(false),
- NoInfsFPMath(false), NoNaNsFPMath(false),
+ NoInfsFPMath(false), NoNaNsFPMath(false), AllowFPExceptAccess(false),
+ AllowFPRoundAccess(false),
HonorSignDependentRoundingFPMathOption(false), UseSoftFloat(false),
NoZerosInBSS(false), JITEmitDebugInfo(false),
JITEmitDebugInfoToDisk(false), GuaranteedTailCallOpt(false),
@@ -126,6 +127,20 @@ namespace llvm {
/// assume the FP arithmetic arguments and results are never NaNs.
unsigned NoNaNsFPMath : 1;
+ /// AllowFPExceptAccess - This flag is enabled when the
+ /// -enable-except-access-fp-math or -enable-fpenv-access-fp-math flag is
+ /// specified on the command line. When this flag is off (the default), the
+ /// code generator is allowed to assume exceptions part of floating-point
+ /// environment is not accessed by the program.
+ unsigned AllowFPExceptAccess : 1;
+
+ /// AllowFPRoundAccess - This flag is enabled when the
+ /// -enable-round-access-fp-math or -enable-fpenv-access-fp-math flag is
+ /// specified on the command line. When this flag is off (the default), the
+ /// code generator is allowed to assume rounding part of floating-point
+ /// environment is not accessed by the program.
+ unsigned AllowFPRoundAccess : 1;
+
/// HonorSignDependentRoundingFPMath - This returns true when the
/// -enable-sign-dependent-rounding-fp-math is specified. If this returns
/// false (the default), the code generator is allowed to assume that the
--
1.8.4
-------------- next part --------------
>From b024752efa35d339917525cb7800524d99b25aaf Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Thu, 13 Nov 2014 16:06:06 +0200
Subject: [PATCH 02/13] Add FPEnv access flags to fast-math flags
Will be used to disable optimizations related to floating-point
operations which would affect runtime semantics (e.g. raising
floating-point exceptions).
---
docs/LangRef.rst | 9 ++++++++-
include/llvm/IR/Instruction.h | 16 ++++++++++++++++
include/llvm/IR/Operator.h | 30 +++++++++++++++++++++++++++++-
lib/AsmParser/LLLexer.cpp | 2 ++
lib/AsmParser/LLParser.h | 12 +++++++-----
lib/AsmParser/LLToken.h | 2 ++
lib/Bitcode/Reader/BitcodeReader.cpp | 4 ++++
lib/Bitcode/Writer/BitcodeWriter.cpp | 4 ++++
lib/IR/AsmWriter.cpp | 4 ++++
lib/IR/Instruction.cpp | 28 ++++++++++++++++++++++++++++
test/Assembler/fast-math-flags.ll | 8 ++++++++
11 files changed, 112 insertions(+), 7 deletions(-)
diff --git a/docs/LangRef.rst b/docs/LangRef.rst
index b184749..4e3e2cb 100644
--- a/docs/LangRef.rst
+++ b/docs/LangRef.rst
@@ -1800,10 +1800,17 @@ otherwise unsafe floating point operations
Allow Reciprocal - Allow optimizations to use the reciprocal of an
argument rather than perform division.
+``kexc``
+ Keep Exceptions - Forbid optimizations to affect floating point exceptions
+ that can be thrown at runtime.
+
+``kround``
+ Keep Rounding - Forbid optimizations to assume any predefined rounding mode.
+
``fast``
Fast - Allow algebraically equivalent transformations that may
dramatically change results in floating point (e.g. reassociate). This
- flag implies all the others.
+ flag implies all the others except for the last two.
.. _uselistorder:
diff --git a/include/llvm/IR/Instruction.h b/include/llvm/IR/Instruction.h
index ba7791c..8f589b6 100644
--- a/include/llvm/IR/Instruction.h
+++ b/include/llvm/IR/Instruction.h
@@ -231,6 +231,16 @@ public:
/// this flag.
void setHasAllowReciprocal(bool B);
+ /// Set or clear the keep-exceptions flag on this instruction, which must be
+ /// an operator which supports this flag. See LangRef.html for the meaning of
+ /// this flag.
+ void setHasKeepExceptions(bool B);
+
+ /// Set or clear the keep-rounding flag on this instruction, which must be an
+ /// operator which supports this flag. See LangRef.html for the meaning of
+ /// this flag.
+ void setHasKeepRounding(bool B);
+
/// Convenience function for setting multiple fast-math flags on this
/// instruction, which must be an operator which supports these flags. See
/// LangRef.html for the meaning of these flags.
@@ -256,6 +266,12 @@ public:
/// Determine whether the allow-reciprocal flag is set.
bool hasAllowReciprocal() const;
+ /// Determine whether the keep-exceptions flag is set.
+ bool hasKeepExceptions() const;
+
+ /// Determine whether the keep-rounding flag is set.
+ bool hasKeepRounding() const;
+
/// Convenience function for getting all the fast-math flags, which must be an
/// operator which supports these flags. See LangRef.html for the meaning of
/// these flags.
diff --git a/include/llvm/IR/Operator.h b/include/llvm/IR/Operator.h
index 0933f21..f0cef33 100644
--- a/include/llvm/IR/Operator.h
+++ b/include/llvm/IR/Operator.h
@@ -174,7 +174,9 @@ public:
NoNaNs = (1 << 1),
NoInfs = (1 << 2),
NoSignedZeros = (1 << 3),
- AllowReciprocal = (1 << 4)
+ AllowReciprocal = (1 << 4),
+ KeepExceptions = (1 << 5),
+ KeepRounding = (1 << 6)
};
FastMathFlags() : Flags(0)
@@ -192,6 +194,8 @@ public:
bool noSignedZeros() { return 0 != (Flags & NoSignedZeros); }
bool allowReciprocal() { return 0 != (Flags & AllowReciprocal); }
bool unsafeAlgebra() { return 0 != (Flags & UnsafeAlgebra); }
+ bool keepExceptions() { return 0 != (Flags & KeepExceptions); }
+ bool keepRounding() { return 0 != (Flags & KeepRounding); }
/// Flag setters
void setNoNaNs() { Flags |= NoNaNs; }
@@ -205,6 +209,8 @@ public:
setNoSignedZeros();
setAllowReciprocal();
}
+ void setKeepExceptions() { Flags |= KeepExceptions; }
+ void setKeepRounding() { Flags |= KeepRounding; }
void operator&=(const FastMathFlags &OtherFlags) {
Flags &= OtherFlags.Flags;
@@ -251,6 +257,16 @@ private:
(SubclassOptionalData & ~FastMathFlags::AllowReciprocal) |
(B * FastMathFlags::AllowReciprocal);
}
+ void setHasKeepExceptions(bool B) {
+ SubclassOptionalData =
+ (SubclassOptionalData & ~FastMathFlags::KeepExceptions) |
+ (B * FastMathFlags::KeepExceptions);
+ }
+ void setHasKeepRounding(bool B) {
+ SubclassOptionalData =
+ (SubclassOptionalData & ~FastMathFlags::KeepRounding) |
+ (B * FastMathFlags::KeepRounding);
+ }
/// Convenience function for setting multiple fast-math flags.
/// FMF is a mask of the bits to set.
@@ -295,6 +311,18 @@ public:
return (SubclassOptionalData & FastMathFlags::AllowReciprocal) != 0;
}
+ /// Test whether this operation should not be optimized if its runtime
+ /// semantics regarding floating-point exceptions can't be preserved.
+ bool hasKeepExceptions() const {
+ return (SubclassOptionalData & FastMathFlags::KeepExceptions) != 0;
+ }
+
+ /// Test whether this operation should not be optimized if its runtime
+ /// semantics regarding floating-point rounding modes can't be preserved.
+ bool hasKeepRounding() const {
+ return (SubclassOptionalData & FastMathFlags::KeepRounding) != 0;
+ }
+
/// Convenience function for getting all the fast-math flags
FastMathFlags getFastMathFlags() const {
return FastMathFlags(SubclassOptionalData);
diff --git a/lib/AsmParser/LLLexer.cpp b/lib/AsmParser/LLLexer.cpp
index de5db1a..961c83e 100644
--- a/lib/AsmParser/LLLexer.cpp
+++ b/lib/AsmParser/LLLexer.cpp
@@ -545,6 +545,8 @@ lltok::Kind LLLexer::LexIdentifier() {
KEYWORD(ninf);
KEYWORD(nsz);
KEYWORD(arcp);
+ KEYWORD(kexc);
+ KEYWORD(kround);
KEYWORD(fast);
KEYWORD(nuw);
KEYWORD(nsw);
diff --git a/lib/AsmParser/LLParser.h b/lib/AsmParser/LLParser.h
index 235e264..16057b5 100644
--- a/lib/AsmParser/LLParser.h
+++ b/lib/AsmParser/LLParser.h
@@ -198,11 +198,13 @@ namespace llvm {
FastMathFlags FMF;
while (true)
switch (Lex.getKind()) {
- case lltok::kw_fast: FMF.setUnsafeAlgebra(); Lex.Lex(); continue;
- case lltok::kw_nnan: FMF.setNoNaNs(); Lex.Lex(); continue;
- case lltok::kw_ninf: FMF.setNoInfs(); Lex.Lex(); continue;
- case lltok::kw_nsz: FMF.setNoSignedZeros(); Lex.Lex(); continue;
- case lltok::kw_arcp: FMF.setAllowReciprocal(); Lex.Lex(); continue;
+ case lltok::kw_fast: FMF.setUnsafeAlgebra(); Lex.Lex(); continue;
+ case lltok::kw_nnan: FMF.setNoNaNs(); Lex.Lex(); continue;
+ case lltok::kw_ninf: FMF.setNoInfs(); Lex.Lex(); continue;
+ case lltok::kw_nsz: FMF.setNoSignedZeros(); Lex.Lex(); continue;
+ case lltok::kw_arcp: FMF.setAllowReciprocal(); Lex.Lex(); continue;
+ case lltok::kw_kexc: FMF.setKeepExceptions(); Lex.Lex(); continue;
+ case lltok::kw_kround: FMF.setKeepRounding(); Lex.Lex(); continue;
default: return FMF;
}
return FMF;
diff --git a/lib/AsmParser/LLToken.h b/lib/AsmParser/LLToken.h
index 4fb0182..cbf04b6 100644
--- a/lib/AsmParser/LLToken.h
+++ b/lib/AsmParser/LLToken.h
@@ -67,6 +67,8 @@ namespace lltok {
kw_ninf,
kw_nsz,
kw_arcp,
+ kw_kexc,
+ kw_kround,
kw_fast,
kw_nuw,
kw_nsw,
diff --git a/lib/Bitcode/Reader/BitcodeReader.cpp b/lib/Bitcode/Reader/BitcodeReader.cpp
index 9e27225..484f419 100644
--- a/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -2726,6 +2726,10 @@ std::error_code BitcodeReader::ParseFunctionBody(Function *F) {
FMF.setNoSignedZeros();
if (0 != (Record[OpNum] & FastMathFlags::AllowReciprocal))
FMF.setAllowReciprocal();
+ if (0 != (Record[OpNum] & FastMathFlags::KeepExceptions))
+ FMF.setKeepExceptions();
+ if (0 != (Record[OpNum] & FastMathFlags::KeepRounding))
+ FMF.setKeepRounding();
if (FMF.any())
I->setFastMathFlags(FMF);
}
diff --git a/lib/Bitcode/Writer/BitcodeWriter.cpp b/lib/Bitcode/Writer/BitcodeWriter.cpp
index 118edc1..4975f18 100644
--- a/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -745,6 +745,10 @@ static uint64_t GetOptimizationFlags(const Value *V) {
Flags |= FastMathFlags::NoSignedZeros;
if (FPMO->hasAllowReciprocal())
Flags |= FastMathFlags::AllowReciprocal;
+ if (FPMO->hasKeepExceptions())
+ Flags |= FastMathFlags::KeepExceptions;
+ if (FPMO->hasKeepRounding())
+ Flags |= FastMathFlags::KeepRounding;
}
return Flags;
diff --git a/lib/IR/AsmWriter.cpp b/lib/IR/AsmWriter.cpp
index d52ac87..07393de 100644
--- a/lib/IR/AsmWriter.cpp
+++ b/lib/IR/AsmWriter.cpp
@@ -950,6 +950,10 @@ static void WriteOptimizationInfo(raw_ostream &Out, const User *U) {
Out << " nsz";
if (FPO->hasAllowReciprocal())
Out << " arcp";
+ if (FPO->hasKeepExceptions())
+ Out << " kexc";
+ if (FPO->hasKeepRounding())
+ Out << " kround";
}
}
diff --git a/lib/IR/Instruction.cpp b/lib/IR/Instruction.cpp
index 92c6e9f..532c457 100644
--- a/lib/IR/Instruction.cpp
+++ b/lib/IR/Instruction.cpp
@@ -124,6 +124,22 @@ void Instruction::setHasAllowReciprocal(bool B) {
cast<FPMathOperator>(this)->setHasAllowReciprocal(B);
}
+/// Set or clear the keep-exceptions flag on this instruction, which must be an
+/// operator which supports this flag. See LangRef.html for the meaning of this
+/// flag.
+void Instruction::setHasKeepExceptions(bool B) {
+ assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
+ cast<FPMathOperator>(this)->setHasKeepExceptions(B);
+}
+
+/// Set or clear the keep-rounding flag on this instruction, which must be an
+/// operator which supports this flag. See LangRef.html for the meaning of this
+/// flag.
+void Instruction::setHasKeepRounding(bool B) {
+ assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
+ cast<FPMathOperator>(this)->setHasKeepRounding(B);
+}
+
/// Convenience function for setting all the fast-math flags on this
/// instruction, which must be an operator which supports these flags. See
/// LangRef.html for the meaning of these flats.
@@ -167,6 +183,18 @@ bool Instruction::hasAllowReciprocal() const {
return cast<FPMathOperator>(this)->hasAllowReciprocal();
}
+/// Determine whether the keep-exceptions flag is set.
+bool Instruction::hasKeepExceptions() const {
+ assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
+ return cast<FPMathOperator>(this)->hasKeepExceptions();
+}
+
+/// Determine whether the keep-rounding flag is set.
+bool Instruction::hasKeepRounding() const {
+ assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
+ return cast<FPMathOperator>(this)->hasKeepRounding();
+}
+
/// Convenience function for getting all the fast-math flags, which must be an
/// operator which supports these flags. See LangRef.html for the meaning of
/// these flags.
diff --git a/test/Assembler/fast-math-flags.ll b/test/Assembler/fast-math-flags.ll
index 8e75bdf..63a4f66 100644
--- a/test/Assembler/fast-math-flags.ll
+++ b/test/Assembler/fast-math-flags.ll
@@ -138,6 +138,14 @@ entry:
%e = frem nnan nsz float %x, %y
; CHECK: %e_vec = frem nnan <3 x float> %vec, %vec
%e_vec = frem nnan <3 x float> %vec, %vec
+; CHECK: %f = fmul fast float %x, %y
+ %f = fmul kexc kround fast float %x, %y
+; CHECK: %f_vec = frem fast <3 x float> %vec, %vec
+ %f_vec = frem kexc kround fast <3 x float> %vec, %vec
+; CHECK: %g = fmul kexc kround float %x, %y
+ %g = fmul kexc kround float %x, %y
+; CHECK: %g_vec = frem kexc kround <3 x float> %vec, %vec
+ %g_vec = frem kexc kround <3 x float> %vec, %vec
; CHECK: ret float %e
ret float %e
}
--
1.8.4
-------------- next part --------------
>From 94835012723c30b984dda4040a4ddeddd3fd16d2 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Thu, 13 Nov 2014 20:40:40 +0200
Subject: [PATCH 03/13] Consider FPEnv access in SelectionDAG
* Disable optimizations that erase floating-point exceptions at runtime
if application is allowed to access exception mask.
* Disable optimizations if application is allowed to access rounding and
result was rounded.
---
include/llvm/CodeGen/SelectionDAG.h | 5 ++
lib/CodeGen/SelectionDAG/SelectionDAG.cpp | 31 +++++--
test/CodeGen/Generic/selectiondag-fenv.ll | 140 ++++++++++++++++++++++++++++++
3 files changed, 167 insertions(+), 9 deletions(-)
create mode 100644 test/CodeGen/Generic/selectiondag-fenv.ll
diff --git a/include/llvm/CodeGen/SelectionDAG.h b/include/llvm/CodeGen/SelectionDAG.h
index ee4bc08..da6ebc6 100644
--- a/include/llvm/CodeGen/SelectionDAG.h
+++ b/include/llvm/CodeGen/SelectionDAG.h
@@ -1258,6 +1258,11 @@ private:
SDValue N1, SDValue N2, bool nuw, bool nsw,
bool exact);
+ /// Checks whether it's safe to replace a floating-point operation with its
+ /// result. UnsafeOps is bit mask of unacceptable exceptions.
+ bool isSafeToOptimizeFPOp(APFloat::opStatus s,
+ APFloat::opStatus UnsafeOps) const;
+
/// VTList - List of non-single value types.
FoldingSet<SDVTListNode> VTListMap;
diff --git a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index c354155..6e4c16e 100644
--- a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -3445,7 +3445,6 @@ SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT, SDValue N1,
}
// Constant fold FP operations.
- bool HasFPExceptions = TLI->hasFloatingPointExceptions();
ConstantFPSDNode *N1CFP = dyn_cast<ConstantFPSDNode>(N1.getNode());
ConstantFPSDNode *N2CFP = dyn_cast<ConstantFPSDNode>(N2.getNode());
if (N1CFP) {
@@ -3459,30 +3458,32 @@ SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT, SDValue N1,
switch (Opcode) {
case ISD::FADD:
s = V1.add(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || s != APFloat::opInvalidOp)
+ if (isSafeToOptimizeFPOp(s, APFloat::opInvalidOp))
return getConstantFP(V1, VT);
break;
case ISD::FSUB:
s = V1.subtract(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || s!=APFloat::opInvalidOp)
+ if (isSafeToOptimizeFPOp(s, APFloat::opInvalidOp))
return getConstantFP(V1, VT);
break;
case ISD::FMUL:
s = V1.multiply(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || s!=APFloat::opInvalidOp)
+ if (isSafeToOptimizeFPOp(s, APFloat::opInvalidOp))
return getConstantFP(V1, VT);
break;
case ISD::FDIV:
s = V1.divide(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || (s!=APFloat::opInvalidOp &&
- s!=APFloat::opDivByZero)) {
+ if (isSafeToOptimizeFPOp(s,
+ (APFloat::opStatus)(APFloat::opInvalidOp |
+ APFloat::opDivByZero))) {
return getConstantFP(V1, VT);
}
break;
- case ISD::FREM :
+ case ISD::FREM:
s = V1.mod(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || (s!=APFloat::opInvalidOp &&
- s!=APFloat::opDivByZero)) {
+ if (isSafeToOptimizeFPOp(s,
+ (APFloat::opStatus)(APFloat::opInvalidOp |
+ APFloat::opDivByZero))) {
return getConstantFP(V1, VT);
}
break;
@@ -3605,6 +3606,18 @@ SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT, SDValue N1,
return SDValue(N, 0);
}
+bool SelectionDAG::isSafeToOptimizeFPOp(APFloat::opStatus s,
+ APFloat::opStatus UnsafeOps) const {
+ if (getTarget().Options.AllowFPRoundAccess && (s & APFloat::opInexact))
+ return false;
+
+ if (!TLI->hasFloatingPointExceptions())
+ return true;
+ if (getTarget().Options.AllowFPExceptAccess)
+ return s == APFloat::opOK;
+ return (s & UnsafeOps) == APFloat::opOK;
+}
+
SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT,
SDValue N1, SDValue N2, SDValue N3) {
// Perform various simplifications.
diff --git a/test/CodeGen/Generic/selectiondag-fenv.ll b/test/CodeGen/Generic/selectiondag-fenv.ll
new file mode 100644
index 0000000..ed2e3f6
--- /dev/null
+++ b/test/CodeGen/Generic/selectiondag-fenv.ll
@@ -0,0 +1,140 @@
+; RUN: llc < %s -print-machineinstrs=expand-isel-pseudos -o /dev/null -enable-except-access-fp-math 2>&1 | FileCheck %s -check-prefix=CHECK -check-prefix=EXCEPT
+; RUN: llc < %s -print-machineinstrs=expand-isel-pseudos -o /dev/null -enable-round-access-fp-math 2>&1 | FileCheck %s -check-prefix=CHECK -check-prefix=ROUND
+
+; SelectionDAG should not fold floating-point operations that have constant
+; expression as their operands when FEnv access is enabled and such folding
+; leads to differences in observable effects at runtime.
+
+define double @do-not-fold-fadd-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fadd-that-can-trap:
+; CHECK: cp#0: 1.000000e+308
+entry:
+ %val = fadd double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fsub-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fsub-that-can-trap:
+; CHECK: cp#0: 1.000000e-308
+; CHECK: cp#1: 1.000000e+308
+entry:
+ %val = fsub double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fmul-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fmul-that-can-trap:
+; CHECK: cp#0: 1.000000e+300
+entry:
+ %val = fmul double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fdiv-that-can-trap:
+; CHECK: cp#0: 1.000000e+300
+; CHECK: cp#1: 1.000000e-300
+entry:
+ %val = fdiv double 1.000000e+300, 1.000000e-300
+ ret double %val
+}
+
+define double @do-not-fold-frem-that-can-trap() {
+; CHECK-LABEL: do-not-fold-frem-that-can-trap:
+; CHECK: cp#0: 1.0
+entry:
+ %val = frem double 1.0, 0.0
+ ret double %val
+}
+
+define double @do-not-fold-fadd-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fadd-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fadd double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fsub-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fsub-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fsub double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fmul-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fmul-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fmul double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fdiv-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fdiv double 0.101, 0.93
+ ret double %val
+}
+
+; "frem" instruction falls into the same category, but none of tried test-cases
+; could raised inexact exception.
+
+define double @fold-fadd-without-rounding() {
+; CHECK-LABEL: fold-fadd-without-rounding:
+; EXCEPT: cp#0: 2.0
+; ROUND: cp#0: 2.0
+entry:
+ %val = fadd double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fsub-without-rounding() {
+; CHECK-LABEL: fold-fsub-without-rounding:
+; EXCEPT: cp#0: 1.0
+; ROUND: cp#0: 1.0
+entry:
+ %val = fsub double 3.0, 2.0
+ ret double %val
+}
+
+define double @fold-fmul-without-rounding() {
+; CHECK-LABEL: fold-fmul-without-rounding:
+; EXCEPT: cp#0: 6.0
+; ROUND: cp#0: 6.0
+entry:
+ %val = fmul double 2.0, 3.0
+ ret double %val
+}
+
+define double @fold-fdiv-without-rounding() {
+; CHECK-LABEL: fold-fdiv-without-rounding:
+; EXCEPT: cp#0: 2.0
+; ROUND: cp#0: 2.0
+entry:
+ %val = fdiv double 6.0, 3.0
+ ret double %val
+}
+
+define double @fold-frem-without-rounding() {
+; CHECK-LABEL: fold-frem-without-rounding:
+; EXCEPT: cp#0: 1.0
+; ROUND: cp#0: 1.0
+entry:
+ %val = frem double 7.0, 3.0
+ ret double %val
+}
--
1.8.4
-------------- next part --------------
>From ff4c792c31f874e0614c8671dca834061aa8d27a Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Mon, 17 Nov 2014 23:10:29 +0200
Subject: [PATCH 04/13] Skip constant folding to preserve FPEnv
Consider floating point environment flags when constant folding
expressions.
---
include/llvm/Analysis/ValueTracking.h | 9 +++--
include/llvm/IR/Constants.h | 8 ++++-
lib/Analysis/ValueTracking.cpp | 25 +++++++++++++-
lib/IR/ConstantFold.cpp | 51 +++++++++++++++++++--------
lib/IR/ConstantFold.h | 3 +-
lib/IR/Constants.cpp | 65 +++++++++++++++++++++++++++++++++--
6 files changed, 138 insertions(+), 23 deletions(-)
diff --git a/include/llvm/Analysis/ValueTracking.h b/include/llvm/Analysis/ValueTracking.h
index cc58838..338934b 100644
--- a/include/llvm/Analysis/ValueTracking.h
+++ b/include/llvm/Analysis/ValueTracking.h
@@ -183,8 +183,9 @@ namespace llvm {
bool onlyUsedByLifetimeMarkers(const Value *V);
/// isSafeToSpeculativelyExecute - Return true if the instruction does not
- /// have any effects besides calculating the result and does not have
- /// undefined behavior.
+ /// have any effects besides calculating the result (in a way that doesn't
+ /// lose floating-point exceptions or precision when requested via
+ /// corresponding flags) and does not have undefined behavior.
///
/// This method never returns true for an instruction that returns true for
/// mayHaveSideEffects; however, this method also does some other checks in
@@ -201,7 +202,9 @@ namespace llvm {
/// However, this method can return true for instructions that read memory;
/// for such instructions, moving them may change the resulting value.
bool isSafeToSpeculativelyExecute(const Value *V,
- const DataLayout *TD = nullptr);
+ const DataLayout *TD = nullptr,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// isKnownNonNull - Return true if this pointer couldn't possibly be null by
/// its definition. This returns true for allocas, non-extern-weak globals
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index 1b0e1b7..4b05c2f 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -1016,7 +1016,9 @@ public:
///
/// \param OnlyIfReducedTy see \a getWithOperands() docs.
static Constant *get(unsigned Opcode, Constant *C1, Constant *C2,
- unsigned Flags = 0, Type *OnlyIfReducedTy = nullptr);
+ unsigned Flags = 0, Type *OnlyIfReducedTy = nullptr,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// \brief Return an ICmp or FCmp comparison operator constant expression.
///
@@ -1139,6 +1141,10 @@ public:
return V->getValueID() == ConstantExprVal;
}
+ /// Evaluates floating point operation to examine it, does nothing for non-FP
+ /// operations. Returns false for unsupported values.
+ static bool getFPOpExceptions(const Value *V, APFloat::opStatus &S);
+
private:
// Shadow Value::setValueSubclassData with a private forwarding method so that
// subclasses cannot accidentally use it.
diff --git a/lib/Analysis/ValueTracking.cpp b/lib/Analysis/ValueTracking.cpp
index 5d90917..1cb1099 100644
--- a/lib/Analysis/ValueTracking.cpp
+++ b/lib/Analysis/ValueTracking.cpp
@@ -2543,11 +2543,18 @@ bool llvm::onlyUsedByLifetimeMarkers(const Value *V) {
}
bool llvm::isSafeToSpeculativelyExecute(const Value *V,
- const DataLayout *TD) {
+ const DataLayout *TD,
+ bool KeepExceptions,
+ bool KeepRounding) {
const Operator *Inst = dyn_cast<Operator>(V);
if (!Inst)
return false;
+ if (const FPMathOperator *MathOp = dyn_cast<FPMathOperator>(Inst)) {
+ KeepExceptions |= MathOp->hasKeepExceptions();
+ KeepRounding |= MathOp->hasKeepRounding();
+ }
+
for (unsigned i = 0, e = Inst->getNumOperands(); i != e; ++i)
if (Constant *C = dyn_cast<Constant>(Inst->getOperand(i)))
if (C->canTrap())
@@ -2556,6 +2563,22 @@ bool llvm::isSafeToSpeculativelyExecute(const Value *V,
switch (Inst->getOpcode()) {
default:
return true;
+ case Instruction::FMul:
+ case Instruction::FAdd:
+ case Instruction::FSub:
+ case Instruction::FDiv:
+ case Instruction::FRem:
+ if (KeepExceptions || KeepRounding) {
+ APFloat::opStatus S;
+ if (!ConstantExpr::getFPOpExceptions(Inst, S))
+ return false;
+
+ if (KeepExceptions && S != APFloat::opOK)
+ return false;
+ if (KeepRounding && (S & APFloat::opInexact))
+ return false;
+ }
+ return true;
case Instruction::UDiv:
case Instruction::URem: {
// x / y is undefined if y == 0.
diff --git a/lib/IR/ConstantFold.cpp b/lib/IR/ConstantFold.cpp
index 9176bf2..c84cb86 100644
--- a/lib/IR/ConstantFold.cpp
+++ b/lib/IR/ConstantFold.cpp
@@ -897,7 +897,9 @@ Constant *llvm::ConstantFoldInsertValueInstruction(Constant *Agg,
Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
- Constant *C1, Constant *C2) {
+ Constant *C1, Constant *C2,
+ bool KeepExceptions,
+ bool KeepRounding) {
// Handle UndefValue up front.
if (isa<UndefValue>(C1) || isa<UndefValue>(C2)) {
switch (Opcode) {
@@ -1081,7 +1083,8 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
} else if (isa<ConstantInt>(C1)) {
// If C1 is a ConstantInt and C2 is not, swap the operands.
if (Instruction::isCommutative(Opcode))
- return ConstantExpr::get(Opcode, C2, C1);
+ return ConstantExpr::get(Opcode, C2, C1, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
// At this point we know neither constant is an UndefValue.
@@ -1162,23 +1165,39 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
APFloat C1V = CFP1->getValueAPF();
APFloat C2V = CFP2->getValueAPF();
APFloat C3V = C1V; // copy for modification
+ APFloat::opStatus S;
switch (Opcode) {
- default:
+ default:
break;
case Instruction::FAdd:
- (void)C3V.add(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.add(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FSub:
- (void)C3V.subtract(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.subtract(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FMul:
- (void)C3V.multiply(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.multiply(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FDiv:
- (void)C3V.divide(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.divide(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FRem:
- (void)C3V.mod(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.mod(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
}
}
@@ -1191,10 +1210,11 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
ConstantExpr::getExtractElement(C1, ConstantInt::get(Ty, i));
Constant *RHS =
ConstantExpr::getExtractElement(C2, ConstantInt::get(Ty, i));
-
- Result.push_back(ConstantExpr::get(Opcode, LHS, RHS));
+
+ Result.push_back(ConstantExpr::get(Opcode, LHS, RHS, 0, nullptr,
+ KeepExceptions, KeepRounding));
}
-
+
return ConstantVector::get(Result);
}
@@ -1206,15 +1226,18 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
// Given ((a + b) + c), if (b + c) folds to something interesting, return
// (a + (b + c)).
if (Instruction::isAssociative(Opcode) && CE1->getOpcode() == Opcode) {
- Constant *T = ConstantExpr::get(Opcode, CE1->getOperand(1), C2);
+ Constant *T = ConstantExpr::get(Opcode, CE1->getOperand(1), C2, 0,
+ nullptr, KeepExceptions, KeepRounding);
if (!isa<ConstantExpr>(T) || cast<ConstantExpr>(T)->getOpcode() != Opcode)
- return ConstantExpr::get(Opcode, CE1->getOperand(0), T);
+ return ConstantExpr::get(Opcode, CE1->getOperand(0), T, 0, nullptr,
+ KeepExceptions, KeepRounding);
}
} else if (isa<ConstantExpr>(C2)) {
// If C2 is a constant expr and C1 isn't, flop them around and fold the
// other way if possible.
if (Instruction::isCommutative(Opcode))
- return ConstantFoldBinaryInstruction(Opcode, C2, C1);
+ return ConstantFoldBinaryInstruction(Opcode, C2, C1, KeepExceptions,
+ KeepRounding);
}
// i1 can be simplified in many cases.
diff --git a/lib/IR/ConstantFold.h b/lib/IR/ConstantFold.h
index a516abe..ee10877 100644
--- a/lib/IR/ConstantFold.h
+++ b/lib/IR/ConstantFold.h
@@ -44,7 +44,8 @@ namespace llvm {
Constant *ConstantFoldInsertValueInstruction(Constant *Agg, Constant *Val,
ArrayRef<unsigned> Idxs);
Constant *ConstantFoldBinaryInstruction(unsigned Opcode, Constant *V1,
- Constant *V2);
+ Constant *V2, bool KeepFPExceptions,
+ bool KeepFPRounding);
Constant *ConstantFoldCompareInstruction(unsigned short predicate,
Constant *C1, Constant *C2);
Constant *ConstantFoldGetElementPtr(Constant *C, bool inBounds,
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index 1d2602a..92b6363 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -316,8 +316,9 @@ static bool canTrapImpl(const Constant *C,
// ConstantExpr traps if any operands can trap.
for (unsigned i = 0, e = C->getNumOperands(); i != e; ++i) {
if (ConstantExpr *Op = dyn_cast<ConstantExpr>(CE->getOperand(i))) {
- if (NonTrappingOps.insert(Op).second && canTrapImpl(Op, NonTrappingOps))
+ if (NonTrappingOps.insert(Op).second && canTrapImpl(Op, NonTrappingOps)) {
return true;
+ }
}
}
@@ -1824,7 +1825,8 @@ Constant *ConstantExpr::getAddrSpaceCast(Constant *C, Type *DstTy,
}
Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
- unsigned Flags, Type *OnlyIfReducedTy) {
+ unsigned Flags, Type *OnlyIfReducedTy,
+ bool KeepExceptions, bool KeepRounding) {
// Check the operands for consistency first.
assert(Opcode >= Instruction::BinaryOpsBegin &&
Opcode < Instruction::BinaryOpsEnd &&
@@ -1890,8 +1892,11 @@ Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
}
#endif
- if (Constant *FC = ConstantFoldBinaryInstruction(Opcode, C1, C2))
+ if (Constant *FC = ConstantFoldBinaryInstruction(Opcode, C1, C2,
+ KeepExceptions,
+ KeepRounding)) {
return FC; // Fold a few common cases.
+ }
if (OnlyIfReducedTy == C1->getType())
return nullptr;
@@ -2965,3 +2970,57 @@ Instruction *ConstantExpr::getAsInstruction() {
return BO;
}
}
+
+bool ConstantExpr::getFPOpExceptions(const Value *V, APFloat::opStatus &S) {
+ unsigned Opcode;
+ const User *U;
+ if (const Operator *Op = dyn_cast<Operator>(V)) {
+ U = Op;
+ Opcode = Op->getOpcode();
+ } if (const ConstantExpr *C = dyn_cast<ConstantExpr>(V)) {
+ U = C;
+ Opcode = C->getOpcode();
+ } else {
+ return false;
+ }
+
+ switch (Opcode) {
+ default:
+ return true;
+ case Instruction::FMul:
+ case Instruction::FAdd:
+ case Instruction::FSub:
+ case Instruction::FDiv:
+ case Instruction::FRem: {
+ ConstantFP *LHSC = dyn_cast<ConstantFP>(U->getOperand(0));
+ ConstantFP *RHSC = dyn_cast<ConstantFP>(U->getOperand(1));
+
+ if (!LHSC || !RHSC) {
+ return false;
+ }
+
+ APFloat LHS = LHSC->getValueAPF();
+ APFloat RHS = LHSC->getValueAPF();
+
+ switch (Opcode) {
+ case Instruction::FMul:
+ S = LHS.multiply(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FAdd:
+ S = LHS.add(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FSub:
+ S = LHS.subtract(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FDiv:
+ S = LHS.divide(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FRem:
+ S = LHS.mod(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ }
+
+ return true;
+ }
+ }
+}
--
1.8.4
-------------- next part --------------
>From 5c8cbeb2efd509f71699d0a7b2eadbfe2ea4ff1b Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Wed, 26 Nov 2014 15:22:55 +0200
Subject: [PATCH 05/13] Teach IR builder and folders about new flags
---
include/llvm/Analysis/ConstantFolding.h | 8 +-
include/llvm/Analysis/TargetFolder.h | 37 ++++++---
include/llvm/IR/ConstantFolder.h | 44 +++++++----
include/llvm/IR/Constants.h | 24 ++++--
include/llvm/IR/IRBuilder.h | 30 +++++---
lib/Analysis/ConstantFolding.cpp | 45 ++++++++---
lib/IR/ConstantFold.h | 4 +-
lib/IR/Constants.cpp | 36 ++++++---
test/Other/fpenv-constant-fold.ll | 130 ++++++++++++++++++++++++++++++++
9 files changed, 288 insertions(+), 70 deletions(-)
create mode 100644 test/Other/fpenv-constant-fold.ll
diff --git a/include/llvm/Analysis/ConstantFolding.h b/include/llvm/Analysis/ConstantFolding.h
index 09d45ca..ffd46ef 100644
--- a/include/llvm/Analysis/ConstantFolding.h
+++ b/include/llvm/Analysis/ConstantFolding.h
@@ -45,7 +45,9 @@ Constant *ConstantFoldInstruction(Instruction *I,
/// result is returned, if not, null is returned.
Constant *ConstantFoldConstantExpression(const ConstantExpr *CE,
const DataLayout *TD = nullptr,
- const TargetLibraryInfo *TLI =nullptr);
+ const TargetLibraryInfo *TLI = nullptr,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// ConstantFoldInstOperands - Attempt to constant fold an instruction with the
/// specified operands. If successful, the constant result is returned, if not,
@@ -56,7 +58,9 @@ Constant *ConstantFoldConstantExpression(const ConstantExpr *CE,
Constant *ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
ArrayRef<Constant *> Ops,
const DataLayout *TD = nullptr,
- const TargetLibraryInfo *TLI = nullptr);
+ const TargetLibraryInfo *TLI = nullptr,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// ConstantFoldCompareInstOperands - Attempt to constant fold a compare
/// instruction (icmp/fcmp) with the specified operands. If it fails, it
diff --git a/include/llvm/Analysis/TargetFolder.h b/include/llvm/Analysis/TargetFolder.h
index 587a7ef..52e2721 100644
--- a/include/llvm/Analysis/TargetFolder.h
+++ b/include/llvm/Analysis/TargetFolder.h
@@ -51,22 +51,31 @@ public:
bool HasNUW = false, bool HasNSW = false) const {
return Fold(ConstantExpr::getAdd(LHS, RHS, HasNUW, HasNSW));
}
- Constant *CreateFAdd(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFAdd(LHS, RHS));
+ Constant *CreateFAdd(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
return Fold(ConstantExpr::getSub(LHS, RHS, HasNUW, HasNSW));
}
- Constant *CreateFSub(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFSub(LHS, RHS));
+ Constant *CreateFSub(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
- bool HasNUW = false, bool HasNSW = false) const {
- return Fold(ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW));
+ bool HasNUW = false, bool HasNSW = false,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW,
+ KeepExceptions, KeepRounding));
}
- Constant *CreateFMul(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFMul(LHS, RHS));
+ Constant *CreateFMul(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS, bool isExact = false)const{
return Fold(ConstantExpr::getUDiv(LHS, RHS, isExact));
@@ -74,8 +83,10 @@ public:
Constant *CreateSDiv(Constant *LHS, Constant *RHS, bool isExact = false)const{
return Fold(ConstantExpr::getSDiv(LHS, RHS, isExact));
}
- Constant *CreateFDiv(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFDiv(LHS, RHS));
+ Constant *CreateFDiv(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return Fold(ConstantExpr::getURem(LHS, RHS));
@@ -83,8 +94,10 @@ public:
Constant *CreateSRem(Constant *LHS, Constant *RHS) const {
return Fold(ConstantExpr::getSRem(LHS, RHS));
}
- Constant *CreateFRem(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFRem(LHS, RHS));
+ Constant *CreateFRem(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/IR/ConstantFolder.h b/include/llvm/IR/ConstantFolder.h
index e271a14..93f25dd 100644
--- a/include/llvm/IR/ConstantFolder.h
+++ b/include/llvm/IR/ConstantFolder.h
@@ -35,22 +35,31 @@ public:
bool HasNUW = false, bool HasNSW = false) const {
return ConstantExpr::getAdd(LHS, RHS, HasNUW, HasNSW);
}
- Constant *CreateFAdd(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFAdd(LHS, RHS);
+ Constant *CreateFAdd(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
return ConstantExpr::getSub(LHS, RHS, HasNUW, HasNSW);
}
- Constant *CreateFSub(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFSub(LHS, RHS);
+ Constant *CreateFSub(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
- bool HasNUW = false, bool HasNSW = false) const {
- return ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW);
+ bool HasNUW = false, bool HasNSW = false,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW, KeepExceptions,
+ KeepRounding);
}
- Constant *CreateFMul(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFMul(LHS, RHS);
+ Constant *CreateFMul(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS,
bool isExact = false) const {
@@ -60,8 +69,10 @@ public:
bool isExact = false) const {
return ConstantExpr::getSDiv(LHS, RHS, isExact);
}
- Constant *CreateFDiv(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFDiv(LHS, RHS);
+ Constant *CreateFDiv(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return ConstantExpr::getURem(LHS, RHS);
@@ -69,8 +80,10 @@ public:
Constant *CreateSRem(Constant *LHS, Constant *RHS) const {
return ConstantExpr::getSRem(LHS, RHS);
}
- Constant *CreateFRem(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFRem(LHS, RHS);
+ Constant *CreateFRem(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
@@ -95,8 +108,11 @@ public:
}
Constant *CreateBinOp(Instruction::BinaryOps Opc,
- Constant *LHS, Constant *RHS) const {
- return ConstantExpr::get(Opc, LHS, RHS);
+ Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::get(Opc, LHS, RHS, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
//===--------------------------------------------------------------------===//
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index 4b05c2f..c96136b 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -841,19 +841,31 @@ public:
static Constant *getNot(Constant *C);
static Constant *getAdd(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false);
- static Constant *getFAdd(Constant *C1, Constant *C2);
+ static Constant *getFAdd(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getSub(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false);
- static Constant *getFSub(Constant *C1, Constant *C2);
+ static Constant *getFSub(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getMul(Constant *C1, Constant *C2,
- bool HasNUW = false, bool HasNSW = false);
- static Constant *getFMul(Constant *C1, Constant *C2);
+ bool HasNUW = false, bool HasNSW = false,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
+ static Constant *getFMul(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getUDiv(Constant *C1, Constant *C2, bool isExact = false);
static Constant *getSDiv(Constant *C1, Constant *C2, bool isExact = false);
- static Constant *getFDiv(Constant *C1, Constant *C2);
+ static Constant *getFDiv(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getURem(Constant *C1, Constant *C2);
static Constant *getSRem(Constant *C1, Constant *C2);
- static Constant *getFRem(Constant *C1, Constant *C2);
+ static Constant *getFRem(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getAnd(Constant *C1, Constant *C2);
static Constant *getOr(Constant *C1, Constant *C2);
static Constant *getXor(Constant *C1, Constant *C2);
diff --git a/include/llvm/IR/IRBuilder.h b/include/llvm/IR/IRBuilder.h
index e5f62fb..3bf8316 100644
--- a/include/llvm/IR/IRBuilder.h
+++ b/include/llvm/IR/IRBuilder.h
@@ -699,10 +699,12 @@ public:
return CreateAdd(LHS, RHS, Name, true, false);
}
Value *CreateFAdd(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFAdd(LC, RC), Name);
+ return Insert(Folder.CreateFAdd(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFAdd(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -721,10 +723,12 @@ public:
return CreateSub(LHS, RHS, Name, true, false);
}
Value *CreateFSub(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFSub(LC, RC), Name);
+ return Insert(Folder.CreateFSub(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFSub(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -743,10 +747,12 @@ public:
return CreateMul(LHS, RHS, Name, true, false);
}
Value *CreateFMul(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFMul(LC, RC), Name);
+ return Insert(Folder.CreateFMul(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFMul(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -775,10 +781,12 @@ public:
return CreateSDiv(LHS, RHS, Name, true);
}
Value *CreateFDiv(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFDiv(LC, RC), Name);
+ return Insert(Folder.CreateFDiv(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFDiv(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -795,10 +803,12 @@ public:
return Insert(BinaryOperator::CreateSRem(LHS, RHS), Name);
}
Value *CreateFRem(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFRem(LC, RC), Name);
+ return Insert(Folder.CreateFRem(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFRem(LHS, RHS),
FPMathTag, FMF), Name);
}
diff --git a/lib/Analysis/ConstantFolding.cpp b/lib/Analysis/ConstantFolding.cpp
index fcafb41..494b678 100644
--- a/lib/Analysis/ConstantFolding.cpp
+++ b/lib/Analysis/ConstantFolding.cpp
@@ -622,8 +622,9 @@ static Constant *ConstantFoldLoadInst(const LoadInst *LI, const DataLayout *TD){
/// Attempt to symbolically evaluate the result of a binary operator merging
/// these together. If target data info is available, it is provided as DL,
/// otherwise DL is null.
-static Constant *SymbolicallyEvaluateBinop(unsigned Opc, Constant *Op0,
- Constant *Op1, const DataLayout *DL){
+static Constant *SymbolicallyEvaluateBinop(unsigned Opc,
+ Constant *Op0, Constant *Op1,
+ const DataLayout *DL) {
// SROA
// Fold (and 0xffffffff00000000, (shl x, 32)) -> shl.
@@ -901,6 +902,14 @@ static Constant *SymbolicallyEvaluateGEP(ArrayRef<Constant *> Ops,
Constant *llvm::ConstantFoldInstruction(Instruction *I,
const DataLayout *TD,
const TargetLibraryInfo *TLI) {
+ bool KeepExceptions = false;
+ bool KeepRounding = false;
+
+ if (isa<FPMathOperator>(I)) {
+ KeepExceptions = I->hasKeepExceptions();
+ KeepRounding = I->hasKeepRounding();
+ }
+
// Handle PHI nodes quickly here...
if (PHINode *PN = dyn_cast<PHINode>(I)) {
Constant *CommonValue = nullptr;
@@ -919,7 +928,8 @@ Constant *llvm::ConstantFoldInstruction(Instruction *I,
return nullptr;
// Fold the PHI's operands.
if (ConstantExpr *NewC = dyn_cast<ConstantExpr>(C))
- C = ConstantFoldConstantExpression(NewC, TD, TLI);
+ C = ConstantFoldConstantExpression(NewC, TD, TLI,
+ KeepExceptions, KeepRounding);
// If the incoming value is a different constant to
// the one we saw previously, then give up.
if (CommonValue && C != CommonValue)
@@ -942,7 +952,8 @@ Constant *llvm::ConstantFoldInstruction(Instruction *I,
// Fold the Instruction's operands.
if (ConstantExpr *NewCE = dyn_cast<ConstantExpr>(Op))
- Op = ConstantFoldConstantExpression(NewCE, TD, TLI);
+ Op = ConstantFoldConstantExpression(NewCE, TD, TLI,
+ KeepExceptions, KeepRounding);
Ops.push_back(Op);
}
@@ -967,13 +978,15 @@ Constant *llvm::ConstantFoldInstruction(Instruction *I,
EVI->getIndices());
}
- return ConstantFoldInstOperands(I->getOpcode(), I->getType(), Ops, TD, TLI);
+ return ConstantFoldInstOperands(I->getOpcode(), I->getType(), Ops, TD, TLI,
+ KeepExceptions, KeepRounding);
}
static Constant *
ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
const TargetLibraryInfo *TLI,
- SmallPtrSetImpl<ConstantExpr *> &FoldedOps) {
+ SmallPtrSetImpl<ConstantExpr *> &FoldedOps,
+ bool KeepExceptions, bool KeepRounding) {
SmallVector<Constant *, 8> Ops;
for (User::const_op_iterator i = CE->op_begin(), e = CE->op_end(); i != e;
++i) {
@@ -982,7 +995,8 @@ ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
// a ConstantExpr, we don't have to process it again.
if (ConstantExpr *NewCE = dyn_cast<ConstantExpr>(NewC)) {
if (FoldedOps.insert(NewCE).second)
- NewC = ConstantFoldConstantExpressionImpl(NewCE, TD, TLI, FoldedOps);
+ NewC = ConstantFoldConstantExpressionImpl(NewCE, TD, TLI, FoldedOps,
+ KeepExceptions, KeepRounding);
}
Ops.push_back(NewC);
}
@@ -990,7 +1004,8 @@ ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
if (CE->isCompare())
return ConstantFoldCompareInstOperands(CE->getPredicate(), Ops[0], Ops[1],
TD, TLI);
- return ConstantFoldInstOperands(CE->getOpcode(), CE->getType(), Ops, TD, TLI);
+ return ConstantFoldInstOperands(CE->getOpcode(), CE->getType(), Ops, TD, TLI,
+ KeepExceptions, KeepRounding);
}
/// Attempt to fold the constant expression
@@ -998,9 +1013,12 @@ ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
/// result is returned, if not, null is returned.
Constant *llvm::ConstantFoldConstantExpression(const ConstantExpr *CE,
const DataLayout *TD,
- const TargetLibraryInfo *TLI) {
+ const TargetLibraryInfo *TLI,
+ bool KeepExceptions,
+ bool KeepRounding) {
SmallPtrSet<ConstantExpr *, 4> FoldedOps;
- return ConstantFoldConstantExpressionImpl(CE, TD, TLI, FoldedOps);
+ return ConstantFoldConstantExpressionImpl(CE, TD, TLI, FoldedOps,
+ KeepExceptions, KeepRounding);
}
/// Attempt to constant fold an instruction with the
@@ -1016,7 +1034,9 @@ Constant *llvm::ConstantFoldConstantExpression(const ConstantExpr *CE,
Constant *llvm::ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
ArrayRef<Constant *> Ops,
const DataLayout *TD,
- const TargetLibraryInfo *TLI) {
+ const TargetLibraryInfo *TLI,
+ bool KeepExceptions,
+ bool KeepRounding) {
// Handle easy binops first.
if (Instruction::isBinaryOp(Opcode)) {
if (isa<ConstantExpr>(Ops[0]) || isa<ConstantExpr>(Ops[1])) {
@@ -1024,7 +1044,8 @@ Constant *llvm::ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
return C;
}
- return ConstantExpr::get(Opcode, Ops[0], Ops[1]);
+ return ConstantExpr::get(Opcode, Ops[0], Ops[1], 0, nullptr,
+ KeepExceptions, KeepRounding);
}
switch (Opcode) {
diff --git a/lib/IR/ConstantFold.h b/lib/IR/ConstantFold.h
index ee10877..94f83bb 100644
--- a/lib/IR/ConstantFold.h
+++ b/lib/IR/ConstantFold.h
@@ -44,8 +44,8 @@ namespace llvm {
Constant *ConstantFoldInsertValueInstruction(Constant *Agg, Constant *Val,
ArrayRef<unsigned> Idxs);
Constant *ConstantFoldBinaryInstruction(unsigned Opcode, Constant *V1,
- Constant *V2, bool KeepFPExceptions,
- bool KeepFPRounding);
+ Constant *V2, bool KeepExceptions,
+ bool KeepRounding);
Constant *ConstantFoldCompareInstruction(unsigned short predicate,
Constant *C1, Constant *C2);
Constant *ConstantFoldGetElementPtr(Constant *C, bool inBounds,
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index 92b6363..3ba0637 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -2219,8 +2219,10 @@ Constant *ConstantExpr::getAdd(Constant *C1, Constant *C2,
return get(Instruction::Add, C1, C2, Flags);
}
-Constant *ConstantExpr::getFAdd(Constant *C1, Constant *C2) {
- return get(Instruction::FAdd, C1, C2);
+Constant *ConstantExpr::getFAdd(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FAdd, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
@@ -2230,19 +2232,25 @@ Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
return get(Instruction::Sub, C1, C2, Flags);
}
-Constant *ConstantExpr::getFSub(Constant *C1, Constant *C2) {
- return get(Instruction::FSub, C1, C2);
+Constant *ConstantExpr::getFSub(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FSub, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getMul(Constant *C1, Constant *C2,
- bool HasNUW, bool HasNSW) {
+ bool HasNUW, bool HasNSW,
+ bool KeepExceptions, bool KeepRounding) {
unsigned Flags = (HasNUW ? OverflowingBinaryOperator::NoUnsignedWrap : 0) |
(HasNSW ? OverflowingBinaryOperator::NoSignedWrap : 0);
- return get(Instruction::Mul, C1, C2, Flags);
+ return get(Instruction::Mul, C1, C2, Flags, nullptr, KeepExceptions,
+ KeepRounding);
}
-Constant *ConstantExpr::getFMul(Constant *C1, Constant *C2) {
- return get(Instruction::FMul, C1, C2);
+Constant *ConstantExpr::getFMul(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FMul, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getUDiv(Constant *C1, Constant *C2, bool isExact) {
@@ -2255,8 +2263,10 @@ Constant *ConstantExpr::getSDiv(Constant *C1, Constant *C2, bool isExact) {
isExact ? PossiblyExactOperator::IsExact : 0);
}
-Constant *ConstantExpr::getFDiv(Constant *C1, Constant *C2) {
- return get(Instruction::FDiv, C1, C2);
+Constant *ConstantExpr::getFDiv(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FDiv, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getURem(Constant *C1, Constant *C2) {
@@ -2267,8 +2277,10 @@ Constant *ConstantExpr::getSRem(Constant *C1, Constant *C2) {
return get(Instruction::SRem, C1, C2);
}
-Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2) {
- return get(Instruction::FRem, C1, C2);
+Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FRem, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getAnd(Constant *C1, Constant *C2) {
diff --git a/test/Other/fpenv-constant-fold.ll b/test/Other/fpenv-constant-fold.ll
new file mode 100644
index 0000000..01dbfa0
--- /dev/null
+++ b/test/Other/fpenv-constant-fold.ll
@@ -0,0 +1,130 @@
+; InstCombine is used just for its calls to constant folder.
+; RUN: opt -S -instcombine -o - < %s | FileCheck %s
+
+; Target independent constant folder should not fold floating-point operations
+; that have constant expression as their operands when FEnv access is requested
+; and such folding leads to differences in observable effects at runtime.
+
+define double @do-not-fold-fadd-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fadd-that-can-trap
+; CHECK: fadd
+entry:
+ %val = fadd kexc double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fsub-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fsub-that-can-trap
+; CHECK: fsub
+entry:
+ %val = fsub kexc double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fmul-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fmul-that-can-trap
+; CHECK: fmul
+entry:
+ %val = fmul kexc double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fdiv-that-can-trap
+; CHECK: fdiv
+entry:
+ %val = fdiv kexc double 1.000000e+300, 1.000000e-300
+ ret double %val
+}
+
+define double @do-not-fold-frem-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-frem-that-can-trap
+; CHECK: frem
+entry:
+ %val = frem kexc double 1.000000e+300, 1.000000e-300
+ ret double %val
+}
+
+define double @do-not-fold-fadd-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fadd-because-of-rounding
+; CHECK: fadd
+entry:
+ %val = fadd kround double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fsub-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fsub-because-of-rounding
+; CHECK: fsub
+entry:
+ %val = fsub kround double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fmul-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fmul-because-of-rounding
+; CHECK: fmul
+entry:
+ %val = fmul kround double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fdiv-because-of-rounding
+; CHECK: fdiv
+entry:
+ %val = fdiv kround double 0.101, 0.93
+ ret double %val
+}
+
+; "frem" instruction falls into the same category, but none of tried test-cases
+; could raised inexact exception.
+
+define double @fold-fadd-that-cant-trap() {
+; CHECK-LABEL: @fold-fadd-that-cant-trap
+; CHECK-NOT: fadd
+entry:
+ %val = fadd kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fsub-that-cant-trap() {
+; CHECK-LABEL: @fold-fsub-that-cant-trap
+; CHECK-NOT: fsub
+entry:
+ %val = fsub kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fmul-that-cant-trap() {
+; CHECK-LABEL: @fold-fmul-that-cant-trap
+; CHECK-NOT: fmul
+entry:
+ %val = fmul kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fdiv-that-cant-trap() {
+; CHECK-LABEL: @fold-fdiv-that-cant-trap
+; CHECK-NOT: fdiv
+entry:
+ %val = fdiv kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-frem-that-cant-trap() {
+; CHECK-LABEL: @fold-frem-that-cant-trap
+; CHECK-NOT: frem
+entry:
+ %val = frem kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @correcly-handle-nested-expressions() {
+; CHECK-LABEL: @correcly-handle-nested-expressions
+; CHECK: fmul
+entry:
+ %val1 = fmul kexc double 1.000000e+300, 1.000000e+300
+ %val2 = fdiv kexc double %val1, 1.000000e+300
+ ret double %val2
+}
--
1.8.4
-------------- next part --------------
>From 56a66c6c662b5d0c72052be7aeea47df2ee7db00 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Wed, 26 Nov 2014 19:50:50 +0200
Subject: [PATCH 06/13] Do not fold constants on reading in IR asm/bitcode
---
lib/AsmParser/LLParser.cpp | 3 ++-
lib/Bitcode/Reader/BitcodeReader.cpp | 2 +-
test/Assembler/do-not-fold-fp-consts.ll | 36 +++++++++++++++++++++++++++++++++
3 files changed, 39 insertions(+), 2 deletions(-)
create mode 100644 test/Assembler/do-not-fold-fp-consts.ll
diff --git a/lib/AsmParser/LLParser.cpp b/lib/AsmParser/LLParser.cpp
index 0ad921a..322eccf 100644
--- a/lib/AsmParser/LLParser.cpp
+++ b/lib/AsmParser/LLParser.cpp
@@ -2733,7 +2733,8 @@ bool LLParser::ParseValID(ValID &ID, PerFunctionState *PFS) {
if (NUW) Flags |= OverflowingBinaryOperator::NoUnsignedWrap;
if (NSW) Flags |= OverflowingBinaryOperator::NoSignedWrap;
if (Exact) Flags |= PossiblyExactOperator::IsExact;
- Constant *C = ConstantExpr::get(Opc, Val0, Val1, Flags);
+ Constant *C = ConstantExpr::get(Opc, Val0, Val1, Flags, nullptr, true,
+ true);
ID.ConstantVal = C;
ID.Kind = ValID::t_Constant;
return false;
diff --git a/lib/Bitcode/Reader/BitcodeReader.cpp b/lib/Bitcode/Reader/BitcodeReader.cpp
index 484f419..8344101 100644
--- a/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -1630,7 +1630,7 @@ std::error_code BitcodeReader::ParseConstants() {
Flags |= SDivOperator::IsExact;
}
}
- V = ConstantExpr::get(Opc, LHS, RHS, Flags);
+ V = ConstantExpr::get(Opc, LHS, RHS, Flags, nullptr, true, true);
}
break;
}
diff --git a/test/Assembler/do-not-fold-fp-consts.ll b/test/Assembler/do-not-fold-fp-consts.ll
new file mode 100644
index 0000000..6b2ddcb
--- /dev/null
+++ b/test/Assembler/do-not-fold-fp-consts.ll
@@ -0,0 +1,36 @@
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s
+
+define double @do-not-fold-fadd() {
+; CHECK-LABEL: @do-not-fold-fadd
+; CHECK: fadd
+entry:
+ ret double fadd (double 1.000000e+308, double 1.000000e+308)
+}
+
+define double @do-not-fold-fsub() {
+; CHECK-LABEL: @do-not-fold-fsub
+; CHECK: fsub
+entry:
+ ret double fsub (double 1.000000e-308, double 1.000000e+308)
+}
+
+define double @do-not-fold-fmul() {
+; CHECK-LABEL: @do-not-fold-fmul
+; CHECK: fmul
+entry:
+ ret double fmul (double 1.000000e+300, double 1.000000e+300)
+}
+
+define double @do-not-fold-fdiv() {
+; CHECK-LABEL: @do-not-fold-fdiv
+; CHECK: fdiv
+entry:
+ ret double fdiv (double 1.000000e+300, double 1.000000e-300)
+}
+
+define double @do-not-fold-frem() {
+; CHECK-LABEL: @do-not-fold-frem
+; CHECK: frem
+entry:
+ ret double frem (double 1.000000e+300, double 1.000000e-300)
+}
--
1.8.4
-------------- next part --------------
>From 3093faae1fb5408312e672e95961ddfacd56d809 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 28 Nov 2014 12:58:45 +0200
Subject: [PATCH 07/13] Prevent undesired folding by InstSimplify
---
lib/Analysis/InstructionSimplify.cpp | 9 ++++--
test/Transforms/InstSimplify/fast-math.ll | 48 +++++++++++++++++++++++++++++++
2 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/lib/Analysis/InstructionSimplify.cpp b/lib/Analysis/InstructionSimplify.cpp
index 3fbbd7c..357f76b 100644
--- a/lib/Analysis/InstructionSimplify.cpp
+++ b/lib/Analysis/InstructionSimplify.cpp
@@ -796,7 +796,8 @@ static Value *SimplifyFAddInst(Value *Op0, Value *Op1, FastMathFlags FMF,
if (Constant *CRHS = dyn_cast<Constant>(Op1)) {
Constant *Ops[] = { CLHS, CRHS };
return ConstantFoldInstOperands(Instruction::FAdd, CLHS->getType(),
- Ops, Q.DL, Q.TLI);
+ Ops, Q.DL, Q.TLI,
+ FMF.keepExceptions(), FMF.keepRounding());
}
// Canonicalize the constant to the RHS.
@@ -838,7 +839,8 @@ static Value *SimplifyFSubInst(Value *Op0, Value *Op1, FastMathFlags FMF,
if (Constant *CRHS = dyn_cast<Constant>(Op1)) {
Constant *Ops[] = { CLHS, CRHS };
return ConstantFoldInstOperands(Instruction::FSub, CLHS->getType(),
- Ops, Q.DL, Q.TLI);
+ Ops, Q.DL, Q.TLI,
+ FMF.keepExceptions(), FMF.keepRounding());
}
}
@@ -876,7 +878,8 @@ static Value *SimplifyFMulInst(Value *Op0, Value *Op1,
if (Constant *CRHS = dyn_cast<Constant>(Op1)) {
Constant *Ops[] = { CLHS, CRHS };
return ConstantFoldInstOperands(Instruction::FMul, CLHS->getType(),
- Ops, Q.DL, Q.TLI);
+ Ops, Q.DL, Q.TLI,
+ FMF.keepExceptions(), FMF.keepRounding());
}
// Canonicalize the constant to the RHS.
diff --git a/test/Transforms/InstSimplify/fast-math.ll b/test/Transforms/InstSimplify/fast-math.ll
index 71d1ed8..255a283 100644
--- a/test/Transforms/InstSimplify/fast-math.ll
+++ b/test/Transforms/InstSimplify/fast-math.ll
@@ -105,3 +105,51 @@ define float @nofold_fadd_x_0(float %a) {
; CHECK: ret float %no_zero
ret float %no_zero
}
+
+; CHECK-LABEL: @nofold_fadd_kexc(
+define double @nofold_fadd_kexc() {
+; CHECK: fadd
+entry:
+ %val = fadd kexc double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fsub_kexc(
+define double @nofold_fsub_kexc() {
+; CHECK: fsub
+entry:
+ %val = fsub kexc double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fmul_kexc(
+define double @nofold_fmul_kexc() {
+; CHECK: fmul
+entry:
+ %val = fmul kexc double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fadd_kround(
+define double @nofold_fadd_kround() {
+; CHECK: fadd
+entry:
+ %val = fadd kround double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fsub_kround(
+define double @nofold_fsub_kround() {
+; CHECK: fsub
+entry:
+ %val = fsub kround double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fmul_kround(
+define double @nofold_fmul_kround() {
+; CHECK: fmul
+entry:
+ %val = fmul kround double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
--
1.8.4
-------------- next part --------------
>From ec92ec47ace8175112801393d3b9e068710ebf5a Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 28 Nov 2014 15:53:44 +0200
Subject: [PATCH 08/13] Do not simplify expressions with FPEnv access
ConstantExpression can't store these flags, which can lead to
going against them.
---
include/llvm/IR/Constants.h | 4 ++--
lib/Analysis/ConstantFolding.cpp | 2 +-
lib/IR/Constants.cpp | 8 +++++++-
test/Transforms/EarlyCSE/fpenv.ll | 24 ++++++++++++++++++++++++
4 files changed, 34 insertions(+), 4 deletions(-)
create mode 100644 test/Transforms/EarlyCSE/fpenv.ll
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index c96136b..6bb0dc8 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -1029,8 +1029,8 @@ public:
/// \param OnlyIfReducedTy see \a getWithOperands() docs.
static Constant *get(unsigned Opcode, Constant *C1, Constant *C2,
unsigned Flags = 0, Type *OnlyIfReducedTy = nullptr,
- bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepExceptions = false, bool KeepRounding = false,
+ bool Strict = false);
/// \brief Return an ICmp or FCmp comparison operator constant expression.
///
diff --git a/lib/Analysis/ConstantFolding.cpp b/lib/Analysis/ConstantFolding.cpp
index 494b678..5b20ce2 100644
--- a/lib/Analysis/ConstantFolding.cpp
+++ b/lib/Analysis/ConstantFolding.cpp
@@ -1045,7 +1045,7 @@ Constant *llvm::ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
}
return ConstantExpr::get(Opcode, Ops[0], Ops[1], 0, nullptr,
- KeepExceptions, KeepRounding);
+ KeepExceptions, KeepRounding, /* Strict = */ true);
}
switch (Opcode) {
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index 3ba0637..600b23d 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -1826,7 +1826,8 @@ Constant *ConstantExpr::getAddrSpaceCast(Constant *C, Type *DstTy,
Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
unsigned Flags, Type *OnlyIfReducedTy,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
// Check the operands for consistency first.
assert(Opcode >= Instruction::BinaryOpsBegin &&
Opcode < Instruction::BinaryOpsEnd &&
@@ -1901,6 +1902,11 @@ Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
if (OnlyIfReducedTy == C1->getType())
return nullptr;
+ // ConstantExpression can't store these flags, which can lead to going against
+ // them.
+ if (Strict && (KeepExceptions || KeepRounding))
+ return nullptr;
+
Constant *ArgVec[] = { C1, C2 };
ConstantExprKeyType Key(Opcode, ArgVec, 0, Flags);
diff --git a/test/Transforms/EarlyCSE/fpenv.ll b/test/Transforms/EarlyCSE/fpenv.ll
new file mode 100644
index 0000000..aba632c
--- /dev/null
+++ b/test/Transforms/EarlyCSE/fpenv.ll
@@ -0,0 +1,24 @@
+; %val.true and %val.false should not be moved inside phi-node as constant
+; expressions as such transformation looses fast-math flags.
+; RUN: opt < %s -S -early-cse | FileCheck %s
+
+define double @do-not-turn-into-constexpr(double %x) {
+; CHECK-LABEL: @do-not-turn-into-constexpr(
+; CHECK: %val.true = fmul
+; CHECK: %val.false = fmul
+entry:
+ %cmp = fcmp oeq double %x, 0.000000e+00
+ br i1 %cmp, label %cond.true, label %cond.false
+
+cond.true: ; preds = %entry
+ %val.true = fmul kexc double 1.000000e+300, 1.000000e+300
+ br label %cond.end
+
+cond.false: ; preds = %entry
+ %val.false = fmul kexc double 1.000000e-300, 1.000000e-300
+ br label %cond.end
+
+cond.end: ; preds = %cond.false, %cond.true
+ %cond = phi double [ %val.true, %cond.true ], [ %val.false, %cond.false ]
+ ret double %cond
+}
--
1.8.4
-------------- next part --------------
>From 0e5332a609ddb5fbe864d81c803cba8051d7e756 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 12 Dec 2014 16:35:01 +0200
Subject: [PATCH 09/13] Make Strict flag available for more clients
---
include/llvm/Analysis/TargetFolder.h | 35 +++++++++++++++++++++++------------
include/llvm/IR/ConstantFolder.h | 35 +++++++++++++++++++++++------------
include/llvm/IR/Constants.h | 18 ++++++++++++------
lib/IR/Constants.cpp | 30 ++++++++++++++++++------------
4 files changed, 76 insertions(+), 42 deletions(-)
diff --git a/include/llvm/Analysis/TargetFolder.h b/include/llvm/Analysis/TargetFolder.h
index 52e2721..a20c499 100644
--- a/include/llvm/Analysis/TargetFolder.h
+++ b/include/llvm/Analysis/TargetFolder.h
@@ -53,8 +53,10 @@ public:
}
Constant *CreateFAdd(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
@@ -62,20 +64,25 @@ public:
}
Constant *CreateFSub(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
+ bool KeepRounding = false,
+ bool Strict = false) const {
return Fold(ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW,
- KeepExceptions, KeepRounding));
+ KeepExceptions, KeepRounding, Strict));
}
Constant *CreateFMul(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS, bool isExact = false)const{
return Fold(ConstantExpr::getUDiv(LHS, RHS, isExact));
@@ -85,8 +92,10 @@ public:
}
Constant *CreateFDiv(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return Fold(ConstantExpr::getURem(LHS, RHS));
@@ -96,8 +105,10 @@ public:
}
Constant *CreateFRem(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/IR/ConstantFolder.h b/include/llvm/IR/ConstantFolder.h
index 93f25dd..daa88f5 100644
--- a/include/llvm/IR/ConstantFolder.h
+++ b/include/llvm/IR/ConstantFolder.h
@@ -37,8 +37,10 @@ public:
}
Constant *CreateFAdd(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
@@ -46,20 +48,25 @@ public:
}
Constant *CreateFSub(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
+ bool KeepRounding = false,
+ bool Strict = false) const {
return ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *CreateFMul(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS,
bool isExact = false) const {
@@ -71,8 +78,10 @@ public:
}
Constant *CreateFDiv(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return ConstantExpr::getURem(LHS, RHS);
@@ -82,8 +91,10 @@ public:
}
Constant *CreateFRem(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index 6bb0dc8..6a4744f 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -843,29 +843,35 @@ public:
bool HasNUW = false, bool HasNSW = false);
static Constant *getFAdd(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getSub(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false);
static Constant *getFSub(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getMul(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getFMul(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getUDiv(Constant *C1, Constant *C2, bool isExact = false);
static Constant *getSDiv(Constant *C1, Constant *C2, bool isExact = false);
static Constant *getFDiv(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getURem(Constant *C1, Constant *C2);
static Constant *getSRem(Constant *C1, Constant *C2);
static Constant *getFRem(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getAnd(Constant *C1, Constant *C2);
static Constant *getOr(Constant *C1, Constant *C2);
static Constant *getXor(Constant *C1, Constant *C2);
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index 600b23d..caecae5 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -2226,9 +2226,10 @@ Constant *ConstantExpr::getAdd(Constant *C1, Constant *C2,
}
Constant *ConstantExpr::getFAdd(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FAdd, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
@@ -2239,24 +2240,27 @@ Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
}
Constant *ConstantExpr::getFSub(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FSub, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getMul(Constant *C1, Constant *C2,
bool HasNUW, bool HasNSW,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
unsigned Flags = (HasNUW ? OverflowingBinaryOperator::NoUnsignedWrap : 0) |
(HasNSW ? OverflowingBinaryOperator::NoSignedWrap : 0);
return get(Instruction::Mul, C1, C2, Flags, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getFMul(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FMul, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getUDiv(Constant *C1, Constant *C2, bool isExact) {
@@ -2270,9 +2274,10 @@ Constant *ConstantExpr::getSDiv(Constant *C1, Constant *C2, bool isExact) {
}
Constant *ConstantExpr::getFDiv(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FDiv, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getURem(Constant *C1, Constant *C2) {
@@ -2284,9 +2289,10 @@ Constant *ConstantExpr::getSRem(Constant *C1, Constant *C2) {
}
Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FRem, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getAnd(Constant *C1, Constant *C2) {
--
1.8.4
-------------- next part --------------
>From d15a870b4c26321658261c50b9a0f72d238cc84d Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 12 Dec 2014 16:40:08 +0200
Subject: [PATCH 10/13] Use Strict in IRBuilder
Try to create constant expression, but skip it in case of conflicts
with FPEnv access flags.
---
include/llvm/IR/IRBuilder.h | 25 +++++++++++++++----------
unittests/IR/IRBuilderTest.cpp | 41 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 56 insertions(+), 10 deletions(-)
diff --git a/include/llvm/IR/IRBuilder.h b/include/llvm/IR/IRBuilder.h
index 3bf8316..d8dac17 100644
--- a/include/llvm/IR/IRBuilder.h
+++ b/include/llvm/IR/IRBuilder.h
@@ -703,8 +703,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFAdd(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFAdd(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFAdd(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -727,8 +728,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFSub(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFSub(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFSub(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -751,8 +753,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFMul(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFMul(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFMul(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -785,8 +788,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFDiv(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFDiv(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFDiv(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -807,8 +811,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFRem(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFRem(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFRem(LHS, RHS),
FPMathTag, FMF), Name);
}
diff --git a/unittests/IR/IRBuilderTest.cpp b/unittests/IR/IRBuilderTest.cpp
index 08a729a..18cdcb3 100644
--- a/unittests/IR/IRBuilderTest.cpp
+++ b/unittests/IR/IRBuilderTest.cpp
@@ -203,6 +203,47 @@ TEST_F(IRBuilderTest, FastMathFlags) {
}
+TEST_F(IRBuilderTest, FPEnvFlags) {
+ IRBuilder<> Builder(BB);
+ Value *F;
+
+ Constant *Huge = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e+308);
+ Constant *Big = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e+300);
+ Constant *Small = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e-300);
+ Constant *Tiny = ConstantFP::get(Type::getDoubleTy(Ctx), 1.000000e-308);
+ Constant *Zero = ConstantFP::get(Type::getDoubleTy(Ctx), 0.0);
+
+ F = Builder.CreateFAdd(Huge, Huge, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFSub(Tiny, Huge, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFMul(Big, Big, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFDiv(Big, Small, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFDiv(Big, Zero, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFRem(Big, Zero, "", nullptr, /* KeepExceptions */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFAdd(Huge, Huge, "", nullptr, /* KeepExceptions */ false,
+ /* KeepRounding */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFSub(Tiny, Huge, "", nullptr, /* KeepExceptions */ false,
+ /* KeepRounding */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+
+ F = Builder.CreateFMul(Huge, Huge, "", nullptr, /* KeepExceptions */ false,
+ /* KeepRounding */ true);
+ EXPECT_FALSE(isa<Constant>(F));
+}
+
TEST_F(IRBuilderTest, WrapFlags) {
IRBuilder<true, NoFolder> Builder(BB);
--
1.8.4
-------------- next part --------------
>From 5906c0b4c1e4462d261d053794da4dbff0696bd0 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Mon, 15 Dec 2014 14:18:03 +0200
Subject: [PATCH 11/13] Don't convert fpops to constexprs in SCCP
---
lib/Transforms/Scalar/SCCP.cpp | 18 ++++++++++++----
.../SCCP/do-not-convert-fpop-to-constexpr.ll | 24 ++++++++++++++++++++++
2 files changed, 38 insertions(+), 4 deletions(-)
create mode 100644 test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll
diff --git a/lib/Transforms/Scalar/SCCP.cpp b/lib/Transforms/Scalar/SCCP.cpp
index 05b9608..a8bd9eb 100644
--- a/lib/Transforms/Scalar/SCCP.cpp
+++ b/lib/Transforms/Scalar/SCCP.cpp
@@ -851,10 +851,20 @@ void SCCPSolver::visitBinaryOperator(Instruction &I) {
LatticeVal &IV = ValueState[&I];
if (IV.isOverdefined()) return;
- if (V1State.isConstant() && V2State.isConstant())
- return markConstant(IV, &I,
- ConstantExpr::get(I.getOpcode(), V1State.getConstant(),
- V2State.getConstant()));
+ if (V1State.isConstant() && V2State.isConstant()) {
+ bool KeepExceptions = false;
+ bool KeepRounding = false;
+
+ if (isa<FPMathOperator>(&I)) {
+ KeepExceptions = I.hasKeepExceptions();
+ KeepRounding = I.hasKeepRounding();
+ }
+
+ if (Constant *C = ConstantExpr::get(I.getOpcode(), V1State.getConstant(),
+ V2State.getConstant(), 0, nullptr,
+ KeepExceptions, KeepRounding, true))
+ return markConstant(IV, &I, C);
+ }
// If something is undef, wait for it to resolve.
if (!V1State.isOverdefined() && !V2State.isOverdefined())
diff --git a/test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll b/test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll
new file mode 100644
index 0000000..df3275a
--- /dev/null
+++ b/test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll
@@ -0,0 +1,24 @@
+; fmuls shouldn't be converted to constant expressions.
+; RUN: opt < %s -sccp -S -enable-except-access-fp-math | FileCheck %s
+
+define double @do-not-convert-fpop-to-constexpr(double %x) {
+; CHECK-LABEL: @do-not-convert-fpop-to-constexpr(
+; CHECK: fmul
+; CHECK: fmul
+; CHECK-NOT: select
+entry:
+ %cmp = fcmp oeq double %x, 0.000000e+00
+ br i1 %cmp, label %cond.true, label %cond.false
+
+cond.true:
+ %val.true = fmul kexc double 1.000000e+300, 1.000000e+300
+ br label %cond.end
+
+cond.false:
+ %val.false = fmul kexc double 1.000000e-300, 1.000000e-300
+ br label %cond.end
+
+cond.end:
+ %cond = phi double [ %val.true, %cond.true ], [ %val.false, %cond.false ]
+ ret double %cond
+}
--
1.8.4
-------------- next part --------------
>From ee905a8c4af756d93ab1a0ecdf2d3a105fae2f0c Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 19 Dec 2014 15:08:59 +0200
Subject: [PATCH 12/13] Fix hanging of InstCombine
InstCombine shouldn't hang in non-terminating sequence of conversions like:
(a + b) => (a - (-b)) => (a + b) => etc.
---
.../InstCombine/InstructionCombining.cpp | 6 ++++
.../do-not-hand-on-fpops-with-fpenv-access.ll | 42 ++++++++++++++++++++++
2 files changed, 48 insertions(+)
create mode 100644 test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll
diff --git a/lib/Transforms/InstCombine/InstructionCombining.cpp b/lib/Transforms/InstCombine/InstructionCombining.cpp
index d27801b..1022321 100644
--- a/lib/Transforms/InstCombine/InstructionCombining.cpp
+++ b/lib/Transforms/InstCombine/InstructionCombining.cpp
@@ -2634,6 +2634,12 @@ bool InstCombiner::run() {
Instruction *I = Worklist.RemoveOne();
if (I == nullptr) continue; // skip null values.
+ // Skip floating-point operations with FP environment access flags as when
+ // they can't be folded to constants simplification never ends.
+ if (isa<FPMathOperator>(I) &&
+ (I->hasKeepExceptions() || I->hasKeepRounding()))
+ continue;
+
// Check to see if we can DCE the instruction.
if (isInstructionTriviallyDead(I, TLI)) {
DEBUG(dbgs() << "IC: DCE: " << *I << '\n');
diff --git a/test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll b/test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll
new file mode 100644
index 0000000..8ea33f0
--- /dev/null
+++ b/test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll
@@ -0,0 +1,42 @@
+; RUN: opt < %s -instcombine -S
+
+; InstCombine shouldn't hang in non-terminating sequence of conversions like:
+; (a + b) => (a - (-b)) => (a + b) => etc.
+
+define double @test1() {
+entry:
+ %add = fadd kexc double 0x3FF921FB54442D18, 0x3870000000000000
+ ret double %add
+}
+
+define double @test2(double %x) {
+entry:
+ %sub = fsub kexc kround double -0.000000e+00, 1.000000e+00
+ %div = fdiv kexc kround double %sub, %x
+ ret double %div
+}
+
+define double @test3() {
+entry:
+ %sub = fsub kexc kround double -0.000000e+00, 0x400921FB54442D18
+ ret double %sub
+}
+
+define double @test4(i32 %y) {
+entry:
+ %tobool = icmp ne i32 %y, 0
+ br i1 %tobool, label %cond.true, label %cond.false
+
+cond.true:
+ %sub = fsub kexc kround double -0.000000e+00, 0x400921FB54442D18
+ %div = fdiv kexc kround double %sub, 2.000000e+00
+ br label %cond.end
+
+cond.false:
+ %div1 = fdiv kexc kround double 0x400921FB54442D18, 2.000000e+00
+ br label %cond.end
+
+cond.end:
+ %cond = phi double [ %div, %cond.true ], [ %div1, %cond.false ]
+ ret double %cond
+}
--
1.8.4
-------------- next part --------------
>From 51fc8db814b7db01840886a058afb0fb747f782b Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Mon, 2 Feb 2015 18:53:43 +0200
Subject: [PATCH 13/13] Don't hoist FP-ops with side-effects in LICM
---
lib/Transforms/Scalar/LICM.cpp | 4 ++++
test/Transforms/LICM/fpenv.ll | 30 ++++++++++++++++++++++++++++++
2 files changed, 34 insertions(+)
create mode 100644 test/Transforms/LICM/fpenv.ll
diff --git a/lib/Transforms/Scalar/LICM.cpp b/lib/Transforms/Scalar/LICM.cpp
index a78d77f..c889c94 100644
--- a/lib/Transforms/Scalar/LICM.cpp
+++ b/lib/Transforms/Scalar/LICM.cpp
@@ -501,6 +501,10 @@ bool LICM::canSinkOrHoistInst(Instruction &I) {
!isa<InsertValueInst>(I))
return false;
+ if (const FPMathOperator *MathOp = dyn_cast<FPMathOperator>(&I))
+ if (MathOp->hasKeepExceptions() || MathOp->hasKeepRounding())
+ return false;
+
return isSafeToExecuteUnconditionally(I);
}
diff --git a/test/Transforms/LICM/fpenv.ll b/test/Transforms/LICM/fpenv.ll
new file mode 100644
index 0000000..6fb459c
--- /dev/null
+++ b/test/Transforms/LICM/fpenv.ll
@@ -0,0 +1,30 @@
+; RUN: opt < %s -O3 -S | FileCheck %s
+
+define void @fdiv-is-not-hoisted-out-of-loop(i32 %n) {
+; CHECK-LABEL: @fdiv-is-not-hoisted-out-of-loop(
+; CHECK: for.body:
+; CHECK: fdiv
+entry:
+ %cmp2 = icmp sgt i32 %n, 0
+ br i1 %cmp2, label %for.body.lr.ph, label %for.end
+
+for.body.lr.ph:
+ %0 = add i32 %n, -1
+ br label %for.body
+
+for.body:
+ %i.03 = phi i32 [ 0, %for.body.lr.ph ], [ %inc, %for.body ]
+ %div = fdiv kexc kround double 0.000000e+00, 0.000000e+00
+ tail call void @process(double %div)
+ %inc = add nuw nsw i32 %i.03, 1
+ %exitcond = icmp eq i32 %i.03, %0
+ br i1 %exitcond, label %for.end.loopexit, label %for.body
+
+for.end.loopexit:
+ br label %for.end
+
+for.end:
+ ret void
+}
+
+declare void @process(double) nounwind
--
1.8.4
More information about the llvm-commits
mailing list