[PATCH] Flag to enable IEEE-754 friendly FP optimizations
Sergey Dmitrouk
sdmitrouk at accesssoftek.com
Fri Jan 23 07:51:07 PST 2015
Please find updated patches in the attachment. Changes:
- rounding is checked via inexact exception;
- added changes for "frem" instruction, which was absent in several places;
- updated some of tests (to account changes in rounding and fix wrong
names).
For the purpose of possibly saving some time there is also changes.diff with
the difference from the previous version of the patch set.
Best regards,
Sergey
On Thu, Jan 22, 2015 at 08:23:19PM +0200, Sergey Dmitrouk wrote:
> On Thu, Jan 22, 2015 at 08:40:15AM -0800, Stephen Canon wrote:
> >
> > > On Dec 22, 2014, at 4:36 AM, Sergey Dmitrouk <sdmitrouk at accesssoftek.com> wrote:
> > >
> > > At the moment rounding flag just
> > > disables folding, I'm not sure how to actually check that result of
> > > operation doesn't depend on rounding mode. This prevents early
> > > optimization for something like "1.0 + 2.0", so might need adjustments
> > > if there is a way to check whether rounding is important for particular
> > > operation.
> >
> > This piece is actually straightforward. A result is affected by the rounding mode if and only if inexact is raised (otherwise, the result is exact, and therefore no rounding occurred, which means that the rounding mode can’t have any effect).
>
> Thank you, Steve. Will change that. Now that you explained it seems
> obvious, but I somehow skipped exceptions while thinking about tests for
> rounding before.
>
> > – Steve
>
> --
> Sergey
-------------- next part --------------
>From f88a6e98fa017991eb8a69b541a940b1e2b7c968 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Thu, 13 Nov 2014 16:54:23 +0200
Subject: [PATCH 01/12] Add flags and command-line switches for FPEnv
* Two flags for TargetOptions.
* Three command-line switches to enable flags separately or at the same
time.
---
include/llvm/CodeGen/CommandFlags.h | 19 +++++++++++++++++++
include/llvm/Target/TargetOptions.h | 17 ++++++++++++++++-
2 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/include/llvm/CodeGen/CommandFlags.h b/include/llvm/CodeGen/CommandFlags.h
index 973c595..35497f6 100644
--- a/include/llvm/CodeGen/CommandFlags.h
+++ b/include/llvm/CodeGen/CommandFlags.h
@@ -124,6 +124,21 @@ EnableHonorSignDependentRoundingFPMath("enable-sign-dependent-rounding-fp-math",
cl::init(false));
cl::opt<bool>
+EnableFPEnvAccessFPMath("enable-fpenv-access-fp-math",
+ cl::desc("Disable FP math optimizations that affect FP environment access"),
+ cl::init(false));
+
+cl::opt<bool>
+EnableExceptAccessFPMath("enable-except-access-fp-math",
+ cl::desc("Disable FP math optimizations that affect FP exceptions"),
+ cl::init(false));
+
+cl::opt<bool>
+EnableRoundAccessFPMath("enable-round-access-fp-math",
+ cl::desc("Disable FP math optimizations that affect FP rounding"),
+ cl::init(false));
+
+cl::opt<bool>
GenerateSoftFloatCalls("soft-float",
cl::desc("Generate software floating point library calls"),
cl::init(false));
@@ -272,6 +287,10 @@ static inline TargetOptions InitTargetOptionsFromCodeGenFlags() {
Options.NoNaNsFPMath = EnableNoNaNsFPMath;
Options.HonorSignDependentRoundingFPMathOption =
EnableHonorSignDependentRoundingFPMath;
+ Options.AllowFPExceptAccess = EnableExceptAccessFPMath |
+ EnableFPEnvAccessFPMath;
+ Options.AllowFPRoundAccess = EnableRoundAccessFPMath |
+ EnableFPEnvAccessFPMath;
Options.UseSoftFloat = GenerateSoftFloatCalls;
if (FloatABIForCalls != FloatABI::Default)
Options.FloatABIType = FloatABIForCalls;
diff --git a/include/llvm/Target/TargetOptions.h b/include/llvm/Target/TargetOptions.h
index 9ab8242..df3f804 100644
--- a/include/llvm/Target/TargetOptions.h
+++ b/include/llvm/Target/TargetOptions.h
@@ -70,7 +70,8 @@ namespace llvm {
TargetOptions()
: PrintMachineCode(false), NoFramePointerElim(false),
LessPreciseFPMADOption(false), UnsafeFPMath(false),
- NoInfsFPMath(false), NoNaNsFPMath(false),
+ NoInfsFPMath(false), NoNaNsFPMath(false), AllowFPExceptAccess(false),
+ AllowFPRoundAccess(false),
HonorSignDependentRoundingFPMathOption(false), UseSoftFloat(false),
NoZerosInBSS(false), JITEmitDebugInfo(false),
JITEmitDebugInfoToDisk(false), GuaranteedTailCallOpt(false),
@@ -126,6 +127,20 @@ namespace llvm {
/// assume the FP arithmetic arguments and results are never NaNs.
unsigned NoNaNsFPMath : 1;
+ /// AllowFPExceptAccess - This flag is enabled when the
+ /// -enable-except-access-fp-math or -enable-fpenv-access-fp-math flag is
+ /// specified on the command line. When this flag is off (the default), the
+ /// code generator is allowed to assume rounding part of floating-point
+ /// environment is not accessed by the program.
+ unsigned AllowFPExceptAccess : 1;
+
+ /// AllowFPRoundAccess - This flag is enabled when the
+ /// -enable-round-access-fp-math or -enable-fpenv-access-fp-math flag is
+ /// specified on the command line. When this flag is off (the default), the
+ /// code generator is allowed to assume exceptions part of floating-point
+ /// environment is not accessed by the program.
+ unsigned AllowFPRoundAccess : 1;
+
/// HonorSignDependentRoundingFPMath - This returns true when the
/// -enable-sign-dependent-rounding-fp-math is specified. If this returns
/// false (the default), the code generator is allowed to assume that the
--
1.8.4
-------------- next part --------------
>From 997f505ae572461b67bad3d7a8432b69b2148343 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Thu, 13 Nov 2014 16:06:06 +0200
Subject: [PATCH 02/12] Add FPEnv access flags to fast-math flags
Will be used to disable optimizations related to floating-point
operations which would affect runtime semantics (e.g. raising
floating-point exceptions).
---
docs/LangRef.rst | 9 ++++++++-
include/llvm/IR/Instruction.h | 16 ++++++++++++++++
include/llvm/IR/Operator.h | 30 +++++++++++++++++++++++++++++-
lib/AsmParser/LLLexer.cpp | 2 ++
lib/AsmParser/LLParser.h | 12 +++++++-----
lib/AsmParser/LLToken.h | 2 ++
lib/Bitcode/Reader/BitcodeReader.cpp | 4 ++++
lib/Bitcode/Writer/BitcodeWriter.cpp | 4 ++++
lib/IR/AsmWriter.cpp | 4 ++++
lib/IR/Instruction.cpp | 28 ++++++++++++++++++++++++++++
test/Assembler/fast-math-flags.ll | 8 ++++++++
11 files changed, 112 insertions(+), 7 deletions(-)
diff --git a/docs/LangRef.rst b/docs/LangRef.rst
index b184749..4e3e2cb 100644
--- a/docs/LangRef.rst
+++ b/docs/LangRef.rst
@@ -1800,10 +1800,17 @@ otherwise unsafe floating point operations
Allow Reciprocal - Allow optimizations to use the reciprocal of an
argument rather than perform division.
+``kexc``
+ Keep Exceptions - Forbid optimizations to affect floating point exceptions
+ that can be thrown at runtime.
+
+``kround``
+ Keep Rounding - Forbid optimizations to assume any predefined rounding mode.
+
``fast``
Fast - Allow algebraically equivalent transformations that may
dramatically change results in floating point (e.g. reassociate). This
- flag implies all the others.
+ flag implies all the others except for the last two.
.. _uselistorder:
diff --git a/include/llvm/IR/Instruction.h b/include/llvm/IR/Instruction.h
index ba7791c..8f589b6 100644
--- a/include/llvm/IR/Instruction.h
+++ b/include/llvm/IR/Instruction.h
@@ -231,6 +231,16 @@ public:
/// this flag.
void setHasAllowReciprocal(bool B);
+ /// Set or clear the keep-exceptions flag on this instruction, which must be
+ /// an operator which supports this flag. See LangRef.html for the meaning of
+ /// this flag.
+ void setHasKeepExceptions(bool B);
+
+ /// Set or clear the keep-rounding flag on this instruction, which must be an
+ /// operator which supports this flag. See LangRef.html for the meaning of
+ /// this flag.
+ void setHasKeepRounding(bool B);
+
/// Convenience function for setting multiple fast-math flags on this
/// instruction, which must be an operator which supports these flags. See
/// LangRef.html for the meaning of these flags.
@@ -256,6 +266,12 @@ public:
/// Determine whether the allow-reciprocal flag is set.
bool hasAllowReciprocal() const;
+ /// Determine whether the keep-exceptions flag is set.
+ bool hasKeepExceptions() const;
+
+ /// Determine whether the keep-rounding flag is set.
+ bool hasKeepRounding() const;
+
/// Convenience function for getting all the fast-math flags, which must be an
/// operator which supports these flags. See LangRef.html for the meaning of
/// these flags.
diff --git a/include/llvm/IR/Operator.h b/include/llvm/IR/Operator.h
index 0933f21..f0cef33 100644
--- a/include/llvm/IR/Operator.h
+++ b/include/llvm/IR/Operator.h
@@ -174,7 +174,9 @@ public:
NoNaNs = (1 << 1),
NoInfs = (1 << 2),
NoSignedZeros = (1 << 3),
- AllowReciprocal = (1 << 4)
+ AllowReciprocal = (1 << 4),
+ KeepExceptions = (1 << 5),
+ KeepRounding = (1 << 6)
};
FastMathFlags() : Flags(0)
@@ -192,6 +194,8 @@ public:
bool noSignedZeros() { return 0 != (Flags & NoSignedZeros); }
bool allowReciprocal() { return 0 != (Flags & AllowReciprocal); }
bool unsafeAlgebra() { return 0 != (Flags & UnsafeAlgebra); }
+ bool keepExceptions() { return 0 != (Flags & KeepExceptions); }
+ bool keepRounding() { return 0 != (Flags & KeepRounding); }
/// Flag setters
void setNoNaNs() { Flags |= NoNaNs; }
@@ -205,6 +209,8 @@ public:
setNoSignedZeros();
setAllowReciprocal();
}
+ void setKeepExceptions() { Flags |= KeepExceptions; }
+ void setKeepRounding() { Flags |= KeepRounding; }
void operator&=(const FastMathFlags &OtherFlags) {
Flags &= OtherFlags.Flags;
@@ -251,6 +257,16 @@ private:
(SubclassOptionalData & ~FastMathFlags::AllowReciprocal) |
(B * FastMathFlags::AllowReciprocal);
}
+ void setHasKeepExceptions(bool B) {
+ SubclassOptionalData =
+ (SubclassOptionalData & ~FastMathFlags::KeepExceptions) |
+ (B * FastMathFlags::KeepExceptions);
+ }
+ void setHasKeepRounding(bool B) {
+ SubclassOptionalData =
+ (SubclassOptionalData & ~FastMathFlags::KeepRounding) |
+ (B * FastMathFlags::KeepRounding);
+ }
/// Convenience function for setting multiple fast-math flags.
/// FMF is a mask of the bits to set.
@@ -295,6 +311,18 @@ public:
return (SubclassOptionalData & FastMathFlags::AllowReciprocal) != 0;
}
+ /// Test whether this operation should not be optimized if its runtime
+ /// semantics regarding floating-point exceptions can't be preserved.
+ bool hasKeepExceptions() const {
+ return (SubclassOptionalData & FastMathFlags::KeepExceptions) != 0;
+ }
+
+ /// Test whether this operation should not be optimized if its runtime
+ /// semantics regarding floating-point rounding modes can't be preserved.
+ bool hasKeepRounding() const {
+ return (SubclassOptionalData & FastMathFlags::KeepRounding) != 0;
+ }
+
/// Convenience function for getting all the fast-math flags
FastMathFlags getFastMathFlags() const {
return FastMathFlags(SubclassOptionalData);
diff --git a/lib/AsmParser/LLLexer.cpp b/lib/AsmParser/LLLexer.cpp
index de5db1a..961c83e 100644
--- a/lib/AsmParser/LLLexer.cpp
+++ b/lib/AsmParser/LLLexer.cpp
@@ -545,6 +545,8 @@ lltok::Kind LLLexer::LexIdentifier() {
KEYWORD(ninf);
KEYWORD(nsz);
KEYWORD(arcp);
+ KEYWORD(kexc);
+ KEYWORD(kround);
KEYWORD(fast);
KEYWORD(nuw);
KEYWORD(nsw);
diff --git a/lib/AsmParser/LLParser.h b/lib/AsmParser/LLParser.h
index 235e264..16057b5 100644
--- a/lib/AsmParser/LLParser.h
+++ b/lib/AsmParser/LLParser.h
@@ -198,11 +198,13 @@ namespace llvm {
FastMathFlags FMF;
while (true)
switch (Lex.getKind()) {
- case lltok::kw_fast: FMF.setUnsafeAlgebra(); Lex.Lex(); continue;
- case lltok::kw_nnan: FMF.setNoNaNs(); Lex.Lex(); continue;
- case lltok::kw_ninf: FMF.setNoInfs(); Lex.Lex(); continue;
- case lltok::kw_nsz: FMF.setNoSignedZeros(); Lex.Lex(); continue;
- case lltok::kw_arcp: FMF.setAllowReciprocal(); Lex.Lex(); continue;
+ case lltok::kw_fast: FMF.setUnsafeAlgebra(); Lex.Lex(); continue;
+ case lltok::kw_nnan: FMF.setNoNaNs(); Lex.Lex(); continue;
+ case lltok::kw_ninf: FMF.setNoInfs(); Lex.Lex(); continue;
+ case lltok::kw_nsz: FMF.setNoSignedZeros(); Lex.Lex(); continue;
+ case lltok::kw_arcp: FMF.setAllowReciprocal(); Lex.Lex(); continue;
+ case lltok::kw_kexc: FMF.setKeepExceptions(); Lex.Lex(); continue;
+ case lltok::kw_kround: FMF.setKeepRounding(); Lex.Lex(); continue;
default: return FMF;
}
return FMF;
diff --git a/lib/AsmParser/LLToken.h b/lib/AsmParser/LLToken.h
index 4fb0182..cbf04b6 100644
--- a/lib/AsmParser/LLToken.h
+++ b/lib/AsmParser/LLToken.h
@@ -67,6 +67,8 @@ namespace lltok {
kw_ninf,
kw_nsz,
kw_arcp,
+ kw_kexc,
+ kw_kround,
kw_fast,
kw_nuw,
kw_nsw,
diff --git a/lib/Bitcode/Reader/BitcodeReader.cpp b/lib/Bitcode/Reader/BitcodeReader.cpp
index 9e27225..484f419 100644
--- a/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -2726,6 +2726,10 @@ std::error_code BitcodeReader::ParseFunctionBody(Function *F) {
FMF.setNoSignedZeros();
if (0 != (Record[OpNum] & FastMathFlags::AllowReciprocal))
FMF.setAllowReciprocal();
+ if (0 != (Record[OpNum] & FastMathFlags::KeepExceptions))
+ FMF.setKeepExceptions();
+ if (0 != (Record[OpNum] & FastMathFlags::KeepRounding))
+ FMF.setKeepRounding();
if (FMF.any())
I->setFastMathFlags(FMF);
}
diff --git a/lib/Bitcode/Writer/BitcodeWriter.cpp b/lib/Bitcode/Writer/BitcodeWriter.cpp
index 118edc1..4975f18 100644
--- a/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -745,6 +745,10 @@ static uint64_t GetOptimizationFlags(const Value *V) {
Flags |= FastMathFlags::NoSignedZeros;
if (FPMO->hasAllowReciprocal())
Flags |= FastMathFlags::AllowReciprocal;
+ if (FPMO->hasKeepExceptions())
+ Flags |= FastMathFlags::KeepExceptions;
+ if (FPMO->hasKeepRounding())
+ Flags |= FastMathFlags::KeepRounding;
}
return Flags;
diff --git a/lib/IR/AsmWriter.cpp b/lib/IR/AsmWriter.cpp
index d52ac87..07393de 100644
--- a/lib/IR/AsmWriter.cpp
+++ b/lib/IR/AsmWriter.cpp
@@ -950,6 +950,10 @@ static void WriteOptimizationInfo(raw_ostream &Out, const User *U) {
Out << " nsz";
if (FPO->hasAllowReciprocal())
Out << " arcp";
+ if (FPO->hasKeepExceptions())
+ Out << " kexc";
+ if (FPO->hasKeepRounding())
+ Out << " kround";
}
}
diff --git a/lib/IR/Instruction.cpp b/lib/IR/Instruction.cpp
index 92c6e9f..532c457 100644
--- a/lib/IR/Instruction.cpp
+++ b/lib/IR/Instruction.cpp
@@ -124,6 +124,22 @@ void Instruction::setHasAllowReciprocal(bool B) {
cast<FPMathOperator>(this)->setHasAllowReciprocal(B);
}
+/// Set or clear the keep-exceptions flag on this instruction, which must be an
+/// operator which supports this flag. See LangRef.html for the meaning of this
+/// flag.
+void Instruction::setHasKeepExceptions(bool B) {
+ assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
+ cast<FPMathOperator>(this)->setHasKeepExceptions(B);
+}
+
+/// Set or clear the keep-rounding flag on this instruction, which must be an
+/// operator which supports this flag. See LangRef.html for the meaning of this
+/// flag.
+void Instruction::setHasKeepRounding(bool B) {
+ assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
+ cast<FPMathOperator>(this)->setHasKeepRounding(B);
+}
+
/// Convenience function for setting all the fast-math flags on this
/// instruction, which must be an operator which supports these flags. See
/// LangRef.html for the meaning of these flats.
@@ -167,6 +183,18 @@ bool Instruction::hasAllowReciprocal() const {
return cast<FPMathOperator>(this)->hasAllowReciprocal();
}
+/// Determine whether the keep-exceptions flag is set.
+bool Instruction::hasKeepExceptions() const {
+ assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
+ return cast<FPMathOperator>(this)->hasKeepExceptions();
+}
+
+/// Determine whether the keep-rounding flag is set.
+bool Instruction::hasKeepRounding() const {
+ assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
+ return cast<FPMathOperator>(this)->hasKeepRounding();
+}
+
/// Convenience function for getting all the fast-math flags, which must be an
/// operator which supports these flags. See LangRef.html for the meaning of
/// these flags.
diff --git a/test/Assembler/fast-math-flags.ll b/test/Assembler/fast-math-flags.ll
index 8e75bdf..63a4f66 100644
--- a/test/Assembler/fast-math-flags.ll
+++ b/test/Assembler/fast-math-flags.ll
@@ -138,6 +138,14 @@ entry:
%e = frem nnan nsz float %x, %y
; CHECK: %e_vec = frem nnan <3 x float> %vec, %vec
%e_vec = frem nnan <3 x float> %vec, %vec
+; CHECK: %f = fmul fast float %x, %y
+ %f = fmul kexc kround fast float %x, %y
+; CHECK: %f_vec = frem fast <3 x float> %vec, %vec
+ %f_vec = frem kexc kround fast <3 x float> %vec, %vec
+; CHECK: %g = fmul kexc kround float %x, %y
+ %g = fmul kexc kround float %x, %y
+; CHECK: %g_vec = frem kexc kround <3 x float> %vec, %vec
+ %g_vec = frem kexc kround <3 x float> %vec, %vec
; CHECK: ret float %e
ret float %e
}
--
1.8.4
-------------- next part --------------
>From 8f42f9e507e0a3e3240fdbbc1b68dda071cf676c Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Thu, 13 Nov 2014 20:40:40 +0200
Subject: [PATCH 03/12] Consider FPEnv access in SelectionDAG
* Disable optimizations that erase floating-point exceptions at runtime
if application is allowed to access exception mask.
* Disable optimizations if application is allowed to access rounding and
result was rounded.
---
include/llvm/CodeGen/SelectionDAG.h | 5 ++
lib/CodeGen/SelectionDAG/SelectionDAG.cpp | 30 +++++--
test/CodeGen/Generic/selectiondag-fenv.ll | 140 ++++++++++++++++++++++++++++++
3 files changed, 166 insertions(+), 9 deletions(-)
create mode 100644 test/CodeGen/Generic/selectiondag-fenv.ll
diff --git a/include/llvm/CodeGen/SelectionDAG.h b/include/llvm/CodeGen/SelectionDAG.h
index ee4bc08..da6ebc6 100644
--- a/include/llvm/CodeGen/SelectionDAG.h
+++ b/include/llvm/CodeGen/SelectionDAG.h
@@ -1258,6 +1258,11 @@ private:
SDValue N1, SDValue N2, bool nuw, bool nsw,
bool exact);
+ /// Checks whether it's safe to replace a floating-point operation with its
+ /// result. UnsafeOps is bit mask of unacceptable exceptions.
+ bool isSafeToOptimizeFPOp(APFloat::opStatus s,
+ APFloat::opStatus UnsafeOps) const;
+
/// VTList - List of non-single value types.
FoldingSet<SDVTListNode> VTListMap;
diff --git a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index c354155..c060832 100644
--- a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -3445,7 +3445,6 @@ SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT, SDValue N1,
}
// Constant fold FP operations.
- bool HasFPExceptions = TLI->hasFloatingPointExceptions();
ConstantFPSDNode *N1CFP = dyn_cast<ConstantFPSDNode>(N1.getNode());
ConstantFPSDNode *N2CFP = dyn_cast<ConstantFPSDNode>(N2.getNode());
if (N1CFP) {
@@ -3459,30 +3458,32 @@ SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT, SDValue N1,
switch (Opcode) {
case ISD::FADD:
s = V1.add(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || s != APFloat::opInvalidOp)
+ if (isSafeToOptimizeFPOp(s, APFloat::opInvalidOp))
return getConstantFP(V1, VT);
break;
case ISD::FSUB:
s = V1.subtract(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || s!=APFloat::opInvalidOp)
+ if (isSafeToOptimizeFPOp(s, APFloat::opInvalidOp))
return getConstantFP(V1, VT);
break;
case ISD::FMUL:
s = V1.multiply(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || s!=APFloat::opInvalidOp)
+ if (isSafeToOptimizeFPOp(s, APFloat::opInvalidOp))
return getConstantFP(V1, VT);
break;
case ISD::FDIV:
s = V1.divide(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || (s!=APFloat::opInvalidOp &&
- s!=APFloat::opDivByZero)) {
+ if (isSafeToOptimizeFPOp(s,
+ (APFloat::opStatus)(APFloat::opInvalidOp |
+ APFloat::opDivByZero))) {
return getConstantFP(V1, VT);
}
break;
- case ISD::FREM :
+ case ISD::FREM:
s = V1.mod(V2, APFloat::rmNearestTiesToEven);
- if (!HasFPExceptions || (s!=APFloat::opInvalidOp &&
- s!=APFloat::opDivByZero)) {
+ if (isSafeToOptimizeFPOp(s,
+ (APFloat::opStatus)(APFloat::opInvalidOp |
+ APFloat::opDivByZero))) {
return getConstantFP(V1, VT);
}
break;
@@ -3605,6 +3606,17 @@ SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT, SDValue N1,
return SDValue(N, 0);
}
+bool SelectionDAG::isSafeToOptimizeFPOp(APFloat::opStatus s,
+ APFloat::opStatus UnsafeOps) const {
+ if (!TLI->hasFloatingPointExceptions())
+ return true;
+ if (getTarget().Options.AllowFPExceptAccess)
+ return s == APFloat::opOK;
+ if (getTarget().Options.AllowFPRoundAccess && (s & APFloat::opInexact))
+ return false;
+ return (s & UnsafeOps) == APFloat::opOK;
+}
+
SDValue SelectionDAG::getNode(unsigned Opcode, SDLoc DL, EVT VT,
SDValue N1, SDValue N2, SDValue N3) {
// Perform various simplifications.
diff --git a/test/CodeGen/Generic/selectiondag-fenv.ll b/test/CodeGen/Generic/selectiondag-fenv.ll
new file mode 100644
index 0000000..ed2e3f6
--- /dev/null
+++ b/test/CodeGen/Generic/selectiondag-fenv.ll
@@ -0,0 +1,140 @@
+; RUN: llc < %s -print-machineinstrs=expand-isel-pseudos -o /dev/null -enable-except-access-fp-math 2>&1 | FileCheck %s -check-prefix=CHECK -check-prefix=EXCEPT
+; RUN: llc < %s -print-machineinstrs=expand-isel-pseudos -o /dev/null -enable-round-access-fp-math 2>&1 | FileCheck %s -check-prefix=CHECK -check-prefix=ROUND
+
+; SelectionDAG should not fold floating-point operations that have constant
+; expression as their operands when FEnv access is enabled and such folding
+; leads to differences in observable effects at runtime.
+
+define double @do-not-fold-fadd-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fadd-that-can-trap:
+; CHECK: cp#0: 1.000000e+308
+entry:
+ %val = fadd double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fsub-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fsub-that-can-trap:
+; CHECK: cp#0: 1.000000e-308
+; CHECK: cp#1: 1.000000e+308
+entry:
+ %val = fsub double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fmul-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fmul-that-can-trap:
+; CHECK: cp#0: 1.000000e+300
+entry:
+ %val = fmul double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-that-can-trap() {
+; CHECK-LABEL: do-not-fold-fdiv-that-can-trap:
+; CHECK: cp#0: 1.000000e+300
+; CHECK: cp#1: 1.000000e-300
+entry:
+ %val = fdiv double 1.000000e+300, 1.000000e-300
+ ret double %val
+}
+
+define double @do-not-fold-frem-that-can-trap() {
+; CHECK-LABEL: do-not-fold-frem-that-can-trap:
+; CHECK: cp#0: 1.0
+entry:
+ %val = frem double 1.0, 0.0
+ ret double %val
+}
+
+define double @do-not-fold-fadd-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fadd-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fadd double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fsub-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fsub-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fsub double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fmul-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fmul-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fmul double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-because-of-rounding() {
+; CHECK-LABEL: do-not-fold-fdiv-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fdiv double 0.101, 0.93
+ ret double %val
+}
+
+; "frem" instruction falls into the same category, but none of tried test-cases
+; could raised inexact exception.
+
+define double @fold-fadd-without-rounding() {
+; CHECK-LABEL: fold-fadd-without-rounding:
+; EXCEPT: cp#0: 2.0
+; ROUND: cp#0: 2.0
+entry:
+ %val = fadd double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fsub-without-rounding() {
+; CHECK-LABEL: fold-fsub-without-rounding:
+; EXCEPT: cp#0: 1.0
+; ROUND: cp#0: 1.0
+entry:
+ %val = fsub double 3.0, 2.0
+ ret double %val
+}
+
+define double @fold-fmul-without-rounding() {
+; CHECK-LABEL: fold-fmul-without-rounding:
+; EXCEPT: cp#0: 6.0
+; ROUND: cp#0: 6.0
+entry:
+ %val = fmul double 2.0, 3.0
+ ret double %val
+}
+
+define double @fold-fdiv-without-rounding() {
+; CHECK-LABEL: fold-fdiv-without-rounding:
+; EXCEPT: cp#0: 2.0
+; ROUND: cp#0: 2.0
+entry:
+ %val = fdiv double 6.0, 3.0
+ ret double %val
+}
+
+define double @fold-frem-without-rounding() {
+; CHECK-LABEL: fold-frem-without-rounding:
+; EXCEPT: cp#0: 1.0
+; ROUND: cp#0: 1.0
+entry:
+ %val = frem double 7.0, 3.0
+ ret double %val
+}
--
1.8.4
-------------- next part --------------
>From d4f9254e20944503cd67ee4989931e8271f241a4 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Mon, 17 Nov 2014 23:10:29 +0200
Subject: [PATCH 04/12] Skip constant folding to preserve FPEnv
Consider floating point environment flags when constant folding
expressions.
---
include/llvm/Analysis/ValueTracking.h | 4 +-
include/llvm/IR/Constant.h | 2 +-
include/llvm/IR/Constants.h | 8 ++-
lib/Analysis/ValueTracking.cpp | 27 +++++++++-
lib/IR/ConstantFold.cpp | 51 +++++++++++++------
lib/IR/ConstantFold.h | 3 +-
lib/IR/Constants.cpp | 95 ++++++++++++++++++++++++++++++++---
7 files changed, 163 insertions(+), 27 deletions(-)
diff --git a/include/llvm/Analysis/ValueTracking.h b/include/llvm/Analysis/ValueTracking.h
index cc58838..877ffb3 100644
--- a/include/llvm/Analysis/ValueTracking.h
+++ b/include/llvm/Analysis/ValueTracking.h
@@ -201,7 +201,9 @@ namespace llvm {
/// However, this method can return true for instructions that read memory;
/// for such instructions, moving them may change the resulting value.
bool isSafeToSpeculativelyExecute(const Value *V,
- const DataLayout *TD = nullptr);
+ const DataLayout *TD = nullptr,
+ bool KeepFPExceptions = false,
+ bool KeepFPRounding = false);
/// isKnownNonNull - Return true if this pointer couldn't possibly be null by
/// its definition. This returns true for allocas, non-extern-weak globals
diff --git a/include/llvm/IR/Constant.h b/include/llvm/IR/Constant.h
index d26991e..dfc5710 100644
--- a/include/llvm/IR/Constant.h
+++ b/include/llvm/IR/Constant.h
@@ -77,7 +77,7 @@ public:
/// canTrap - Return true if evaluation of this constant could trap. This is
/// true for things like constant expressions that could divide by zero.
- bool canTrap() const;
+ bool canTrap(bool KeepExceptions = false, bool KeepRounding = false) const;
/// isThreadDependent - Return true if the value can vary between threads.
bool isThreadDependent() const;
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index 1b0e1b7..4b05c2f 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -1016,7 +1016,9 @@ public:
///
/// \param OnlyIfReducedTy see \a getWithOperands() docs.
static Constant *get(unsigned Opcode, Constant *C1, Constant *C2,
- unsigned Flags = 0, Type *OnlyIfReducedTy = nullptr);
+ unsigned Flags = 0, Type *OnlyIfReducedTy = nullptr,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// \brief Return an ICmp or FCmp comparison operator constant expression.
///
@@ -1139,6 +1141,10 @@ public:
return V->getValueID() == ConstantExprVal;
}
+ /// Evaluates floating point operation to examine it, does nothing for non-FP
+ /// operations. Returns false for unsupported values.
+ static bool getFPOpExceptions(const Value *V, APFloat::opStatus &S);
+
private:
// Shadow Value::setValueSubclassData with a private forwarding method so that
// subclasses cannot accidentally use it.
diff --git a/lib/Analysis/ValueTracking.cpp b/lib/Analysis/ValueTracking.cpp
index 5d90917..8ada931 100644
--- a/lib/Analysis/ValueTracking.cpp
+++ b/lib/Analysis/ValueTracking.cpp
@@ -2543,19 +2543,42 @@ bool llvm::onlyUsedByLifetimeMarkers(const Value *V) {
}
bool llvm::isSafeToSpeculativelyExecute(const Value *V,
- const DataLayout *TD) {
+ const DataLayout *TD,
+ bool KeepExceptions,
+ bool KeepRounding) {
const Operator *Inst = dyn_cast<Operator>(V);
if (!Inst)
return false;
+ if (const FPMathOperator *MathOp = dyn_cast<FPMathOperator>(Inst)) {
+ KeepExceptions |= MathOp->hasKeepExceptions();
+ KeepRounding |= MathOp->hasKeepRounding();
+ }
+
for (unsigned i = 0, e = Inst->getNumOperands(); i != e; ++i)
if (Constant *C = dyn_cast<Constant>(Inst->getOperand(i)))
- if (C->canTrap())
+ if (C->canTrap(KeepExceptions))
return false;
switch (Inst->getOpcode()) {
default:
return true;
+ case Instruction::FMul:
+ case Instruction::FAdd:
+ case Instruction::FSub:
+ case Instruction::FDiv:
+ case Instruction::FRem:
+ if (KeepExceptions || KeepRounding) {
+ APFloat::opStatus S;
+ if (!ConstantExpr::getFPOpExceptions(Inst, S))
+ return false;
+
+ if (KeepExceptions && S != APFloat::opOK)
+ return false;
+ if (KeepRounding && (S & APFloat::opInexact))
+ return false;
+ }
+ return true;
case Instruction::UDiv:
case Instruction::URem: {
// x / y is undefined if y == 0.
diff --git a/lib/IR/ConstantFold.cpp b/lib/IR/ConstantFold.cpp
index 9176bf2..c84cb86 100644
--- a/lib/IR/ConstantFold.cpp
+++ b/lib/IR/ConstantFold.cpp
@@ -897,7 +897,9 @@ Constant *llvm::ConstantFoldInsertValueInstruction(Constant *Agg,
Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
- Constant *C1, Constant *C2) {
+ Constant *C1, Constant *C2,
+ bool KeepExceptions,
+ bool KeepRounding) {
// Handle UndefValue up front.
if (isa<UndefValue>(C1) || isa<UndefValue>(C2)) {
switch (Opcode) {
@@ -1081,7 +1083,8 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
} else if (isa<ConstantInt>(C1)) {
// If C1 is a ConstantInt and C2 is not, swap the operands.
if (Instruction::isCommutative(Opcode))
- return ConstantExpr::get(Opcode, C2, C1);
+ return ConstantExpr::get(Opcode, C2, C1, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
// At this point we know neither constant is an UndefValue.
@@ -1162,23 +1165,39 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
APFloat C1V = CFP1->getValueAPF();
APFloat C2V = CFP2->getValueAPF();
APFloat C3V = C1V; // copy for modification
+ APFloat::opStatus S;
switch (Opcode) {
- default:
+ default:
break;
case Instruction::FAdd:
- (void)C3V.add(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.add(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FSub:
- (void)C3V.subtract(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.subtract(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FMul:
- (void)C3V.multiply(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.multiply(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FDiv:
- (void)C3V.divide(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.divide(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
case Instruction::FRem:
- (void)C3V.mod(C2V, APFloat::rmNearestTiesToEven);
+ S = C3V.mod(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
return ConstantFP::get(C1->getContext(), C3V);
}
}
@@ -1191,10 +1210,11 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
ConstantExpr::getExtractElement(C1, ConstantInt::get(Ty, i));
Constant *RHS =
ConstantExpr::getExtractElement(C2, ConstantInt::get(Ty, i));
-
- Result.push_back(ConstantExpr::get(Opcode, LHS, RHS));
+
+ Result.push_back(ConstantExpr::get(Opcode, LHS, RHS, 0, nullptr,
+ KeepExceptions, KeepRounding));
}
-
+
return ConstantVector::get(Result);
}
@@ -1206,15 +1226,18 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
// Given ((a + b) + c), if (b + c) folds to something interesting, return
// (a + (b + c)).
if (Instruction::isAssociative(Opcode) && CE1->getOpcode() == Opcode) {
- Constant *T = ConstantExpr::get(Opcode, CE1->getOperand(1), C2);
+ Constant *T = ConstantExpr::get(Opcode, CE1->getOperand(1), C2, 0,
+ nullptr, KeepExceptions, KeepRounding);
if (!isa<ConstantExpr>(T) || cast<ConstantExpr>(T)->getOpcode() != Opcode)
- return ConstantExpr::get(Opcode, CE1->getOperand(0), T);
+ return ConstantExpr::get(Opcode, CE1->getOperand(0), T, 0, nullptr,
+ KeepExceptions, KeepRounding);
}
} else if (isa<ConstantExpr>(C2)) {
// If C2 is a constant expr and C1 isn't, flop them around and fold the
// other way if possible.
if (Instruction::isCommutative(Opcode))
- return ConstantFoldBinaryInstruction(Opcode, C2, C1);
+ return ConstantFoldBinaryInstruction(Opcode, C2, C1, KeepExceptions,
+ KeepRounding);
}
// i1 can be simplified in many cases.
diff --git a/lib/IR/ConstantFold.h b/lib/IR/ConstantFold.h
index a516abe..ee10877 100644
--- a/lib/IR/ConstantFold.h
+++ b/lib/IR/ConstantFold.h
@@ -44,7 +44,8 @@ namespace llvm {
Constant *ConstantFoldInsertValueInstruction(Constant *Agg, Constant *Val,
ArrayRef<unsigned> Idxs);
Constant *ConstantFoldBinaryInstruction(unsigned Opcode, Constant *V1,
- Constant *V2);
+ Constant *V2, bool KeepFPExceptions,
+ bool KeepFPRounding);
Constant *ConstantFoldCompareInstruction(unsigned short predicate,
Constant *C1, Constant *C2);
Constant *ConstantFoldGetElementPtr(Constant *C, bool inBounds,
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index 1d2602a..b443368 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -306,7 +306,8 @@ void Constant::destroyConstantImpl() {
}
static bool canTrapImpl(const Constant *C,
- SmallPtrSetImpl<const ConstantExpr *> &NonTrappingOps) {
+ SmallPtrSetImpl<const ConstantExpr *> &NonTrappingOps,
+ bool KeepExceptions, bool KeepRounding) {
assert(C->getType()->isFirstClassType() && "Cannot evaluate aggregate vals!");
// The only thing that could possibly trap are constant exprs.
const ConstantExpr *CE = dyn_cast<ConstantExpr>(C);
@@ -316,13 +317,35 @@ static bool canTrapImpl(const Constant *C,
// ConstantExpr traps if any operands can trap.
for (unsigned i = 0, e = C->getNumOperands(); i != e; ++i) {
if (ConstantExpr *Op = dyn_cast<ConstantExpr>(CE->getOperand(i))) {
- if (NonTrappingOps.insert(Op).second && canTrapImpl(Op, NonTrappingOps))
+ if (NonTrappingOps.insert(Op).second && canTrapImpl(Op, NonTrappingOps,
+ KeepExceptions,
+ KeepRounding)) {
return true;
+ }
}
}
// Otherwise, only specific operations can trap.
- switch (CE->getOpcode()) {
+ unsigned Opcode = CE->getOpcode();
+ if (KeepExceptions || KeepRounding) {
+ switch (Opcode) {
+ case Instruction::FMul:
+ case Instruction::FAdd:
+ case Instruction::FSub:
+ case Instruction::FDiv:
+ case Instruction::FRem: {
+ APFloat::opStatus S;
+ if (!ConstantExpr::getFPOpExceptions(CE, S))
+ return false;
+
+ if (KeepExceptions && S != APFloat::opOK)
+ return false;
+ if (KeepRounding && (S & APFloat::opInexact))
+ return false;
+ }
+ }
+ }
+ switch (Opcode) {
default:
return false;
case Instruction::UDiv:
@@ -340,9 +363,9 @@ static bool canTrapImpl(const Constant *C,
/// canTrap - Return true if evaluation of this constant could trap. This is
/// true for things like constant expressions that could divide by zero.
-bool Constant::canTrap() const {
+bool Constant::canTrap(bool KeepExceptions, bool KeepRounding) const {
SmallPtrSet<const ConstantExpr *, 4> NonTrappingOps;
- return canTrapImpl(this, NonTrappingOps);
+ return canTrapImpl(this, NonTrappingOps, KeepExceptions, KeepRounding);
}
/// Check if C contains a GlobalValue for which Predicate is true.
@@ -1824,7 +1847,8 @@ Constant *ConstantExpr::getAddrSpaceCast(Constant *C, Type *DstTy,
}
Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
- unsigned Flags, Type *OnlyIfReducedTy) {
+ unsigned Flags, Type *OnlyIfReducedTy,
+ bool KeepExceptions, bool KeepRounding) {
// Check the operands for consistency first.
assert(Opcode >= Instruction::BinaryOpsBegin &&
Opcode < Instruction::BinaryOpsEnd &&
@@ -1890,8 +1914,11 @@ Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
}
#endif
- if (Constant *FC = ConstantFoldBinaryInstruction(Opcode, C1, C2))
+ if (Constant *FC = ConstantFoldBinaryInstruction(Opcode, C1, C2,
+ KeepExceptions,
+ KeepRounding)) {
return FC; // Fold a few common cases.
+ }
if (OnlyIfReducedTy == C1->getType())
return nullptr;
@@ -2965,3 +2992,57 @@ Instruction *ConstantExpr::getAsInstruction() {
return BO;
}
}
+
+bool ConstantExpr::getFPOpExceptions(const Value *V, APFloat::opStatus &S) {
+ unsigned Opcode;
+ const User *U;
+ if (const Operator *Op = dyn_cast<Operator>(V)) {
+ U = Op;
+ Opcode = Op->getOpcode();
+ } if (const ConstantExpr *C = dyn_cast<ConstantExpr>(V)) {
+ U = C;
+ Opcode = C->getOpcode();
+ } else {
+ return false;
+ }
+
+ switch (Opcode) {
+ default:
+ return true;
+ case Instruction::FMul:
+ case Instruction::FAdd:
+ case Instruction::FSub:
+ case Instruction::FDiv:
+ case Instruction::FRem: {
+ ConstantFP *LHSC = dyn_cast<ConstantFP>(U->getOperand(0));
+ ConstantFP *RHSC = dyn_cast<ConstantFP>(U->getOperand(1));
+
+ if (!LHSC || !RHSC) {
+ return false;
+ }
+
+ APFloat LHS = LHSC->getValueAPF();
+ APFloat RHS = LHSC->getValueAPF();
+
+ switch (Opcode) {
+ case Instruction::FMul:
+ S = LHS.multiply(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FAdd:
+ S = LHS.add(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FSub:
+ S = LHS.subtract(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FDiv:
+ S = LHS.divide(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FRem:
+ S = LHS.mod(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ }
+
+ return true;
+ }
+ }
+}
--
1.8.4
-------------- next part --------------
>From b1068ebbb244021353c0d103eacc6b1ee721ef67 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Wed, 26 Nov 2014 15:22:55 +0200
Subject: [PATCH 05/12] Teach IR builder and folders about new flags
---
include/llvm/Analysis/ConstantFolding.h | 8 +-
include/llvm/Analysis/TargetFolder.h | 37 ++++++---
include/llvm/Analysis/ValueTracking.h | 4 +-
include/llvm/IR/ConstantFolder.h | 44 +++++++----
include/llvm/IR/Constants.h | 24 ++++--
include/llvm/IR/IRBuilder.h | 30 +++++---
lib/Analysis/ConstantFolding.cpp | 45 ++++++++---
lib/IR/ConstantFold.h | 4 +-
lib/IR/Constants.cpp | 36 ++++++---
test/Other/fpenv-constant-fold.ll | 130 ++++++++++++++++++++++++++++++++
10 files changed, 290 insertions(+), 72 deletions(-)
create mode 100644 test/Other/fpenv-constant-fold.ll
diff --git a/include/llvm/Analysis/ConstantFolding.h b/include/llvm/Analysis/ConstantFolding.h
index 09d45ca..ffd46ef 100644
--- a/include/llvm/Analysis/ConstantFolding.h
+++ b/include/llvm/Analysis/ConstantFolding.h
@@ -45,7 +45,9 @@ Constant *ConstantFoldInstruction(Instruction *I,
/// result is returned, if not, null is returned.
Constant *ConstantFoldConstantExpression(const ConstantExpr *CE,
const DataLayout *TD = nullptr,
- const TargetLibraryInfo *TLI =nullptr);
+ const TargetLibraryInfo *TLI = nullptr,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// ConstantFoldInstOperands - Attempt to constant fold an instruction with the
/// specified operands. If successful, the constant result is returned, if not,
@@ -56,7 +58,9 @@ Constant *ConstantFoldConstantExpression(const ConstantExpr *CE,
Constant *ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
ArrayRef<Constant *> Ops,
const DataLayout *TD = nullptr,
- const TargetLibraryInfo *TLI = nullptr);
+ const TargetLibraryInfo *TLI = nullptr,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// ConstantFoldCompareInstOperands - Attempt to constant fold a compare
/// instruction (icmp/fcmp) with the specified operands. If it fails, it
diff --git a/include/llvm/Analysis/TargetFolder.h b/include/llvm/Analysis/TargetFolder.h
index 587a7ef..52e2721 100644
--- a/include/llvm/Analysis/TargetFolder.h
+++ b/include/llvm/Analysis/TargetFolder.h
@@ -51,22 +51,31 @@ public:
bool HasNUW = false, bool HasNSW = false) const {
return Fold(ConstantExpr::getAdd(LHS, RHS, HasNUW, HasNSW));
}
- Constant *CreateFAdd(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFAdd(LHS, RHS));
+ Constant *CreateFAdd(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
return Fold(ConstantExpr::getSub(LHS, RHS, HasNUW, HasNSW));
}
- Constant *CreateFSub(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFSub(LHS, RHS));
+ Constant *CreateFSub(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
- bool HasNUW = false, bool HasNSW = false) const {
- return Fold(ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW));
+ bool HasNUW = false, bool HasNSW = false,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW,
+ KeepExceptions, KeepRounding));
}
- Constant *CreateFMul(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFMul(LHS, RHS));
+ Constant *CreateFMul(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS, bool isExact = false)const{
return Fold(ConstantExpr::getUDiv(LHS, RHS, isExact));
@@ -74,8 +83,10 @@ public:
Constant *CreateSDiv(Constant *LHS, Constant *RHS, bool isExact = false)const{
return Fold(ConstantExpr::getSDiv(LHS, RHS, isExact));
}
- Constant *CreateFDiv(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFDiv(LHS, RHS));
+ Constant *CreateFDiv(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return Fold(ConstantExpr::getURem(LHS, RHS));
@@ -83,8 +94,10 @@ public:
Constant *CreateSRem(Constant *LHS, Constant *RHS) const {
return Fold(ConstantExpr::getSRem(LHS, RHS));
}
- Constant *CreateFRem(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFRem(LHS, RHS));
+ Constant *CreateFRem(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return Fold(ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding));
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/Analysis/ValueTracking.h b/include/llvm/Analysis/ValueTracking.h
index 877ffb3..c42ab01 100644
--- a/include/llvm/Analysis/ValueTracking.h
+++ b/include/llvm/Analysis/ValueTracking.h
@@ -202,8 +202,8 @@ namespace llvm {
/// for such instructions, moving them may change the resulting value.
bool isSafeToSpeculativelyExecute(const Value *V,
const DataLayout *TD = nullptr,
- bool KeepFPExceptions = false,
- bool KeepFPRounding = false);
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
/// isKnownNonNull - Return true if this pointer couldn't possibly be null by
/// its definition. This returns true for allocas, non-extern-weak globals
diff --git a/include/llvm/IR/ConstantFolder.h b/include/llvm/IR/ConstantFolder.h
index e271a14..93f25dd 100644
--- a/include/llvm/IR/ConstantFolder.h
+++ b/include/llvm/IR/ConstantFolder.h
@@ -35,22 +35,31 @@ public:
bool HasNUW = false, bool HasNSW = false) const {
return ConstantExpr::getAdd(LHS, RHS, HasNUW, HasNSW);
}
- Constant *CreateFAdd(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFAdd(LHS, RHS);
+ Constant *CreateFAdd(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
return ConstantExpr::getSub(LHS, RHS, HasNUW, HasNSW);
}
- Constant *CreateFSub(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFSub(LHS, RHS);
+ Constant *CreateFSub(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
- bool HasNUW = false, bool HasNSW = false) const {
- return ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW);
+ bool HasNUW = false, bool HasNSW = false,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW, KeepExceptions,
+ KeepRounding);
}
- Constant *CreateFMul(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFMul(LHS, RHS);
+ Constant *CreateFMul(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS,
bool isExact = false) const {
@@ -60,8 +69,10 @@ public:
bool isExact = false) const {
return ConstantExpr::getSDiv(LHS, RHS, isExact);
}
- Constant *CreateFDiv(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFDiv(LHS, RHS);
+ Constant *CreateFDiv(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return ConstantExpr::getURem(LHS, RHS);
@@ -69,8 +80,10 @@ public:
Constant *CreateSRem(Constant *LHS, Constant *RHS) const {
return ConstantExpr::getSRem(LHS, RHS);
}
- Constant *CreateFRem(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFRem(LHS, RHS);
+ Constant *CreateFRem(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding);
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
@@ -95,8 +108,11 @@ public:
}
Constant *CreateBinOp(Instruction::BinaryOps Opc,
- Constant *LHS, Constant *RHS) const {
- return ConstantExpr::get(Opc, LHS, RHS);
+ Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false) const {
+ return ConstantExpr::get(Opc, LHS, RHS, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
//===--------------------------------------------------------------------===//
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index 4b05c2f..c96136b 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -841,19 +841,31 @@ public:
static Constant *getNot(Constant *C);
static Constant *getAdd(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false);
- static Constant *getFAdd(Constant *C1, Constant *C2);
+ static Constant *getFAdd(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getSub(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false);
- static Constant *getFSub(Constant *C1, Constant *C2);
+ static Constant *getFSub(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getMul(Constant *C1, Constant *C2,
- bool HasNUW = false, bool HasNSW = false);
- static Constant *getFMul(Constant *C1, Constant *C2);
+ bool HasNUW = false, bool HasNSW = false,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
+ static Constant *getFMul(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getUDiv(Constant *C1, Constant *C2, bool isExact = false);
static Constant *getSDiv(Constant *C1, Constant *C2, bool isExact = false);
- static Constant *getFDiv(Constant *C1, Constant *C2);
+ static Constant *getFDiv(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getURem(Constant *C1, Constant *C2);
static Constant *getSRem(Constant *C1, Constant *C2);
- static Constant *getFRem(Constant *C1, Constant *C2);
+ static Constant *getFRem(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false);
static Constant *getAnd(Constant *C1, Constant *C2);
static Constant *getOr(Constant *C1, Constant *C2);
static Constant *getXor(Constant *C1, Constant *C2);
diff --git a/include/llvm/IR/IRBuilder.h b/include/llvm/IR/IRBuilder.h
index e5f62fb..3bf8316 100644
--- a/include/llvm/IR/IRBuilder.h
+++ b/include/llvm/IR/IRBuilder.h
@@ -699,10 +699,12 @@ public:
return CreateAdd(LHS, RHS, Name, true, false);
}
Value *CreateFAdd(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFAdd(LC, RC), Name);
+ return Insert(Folder.CreateFAdd(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFAdd(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -721,10 +723,12 @@ public:
return CreateSub(LHS, RHS, Name, true, false);
}
Value *CreateFSub(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFSub(LC, RC), Name);
+ return Insert(Folder.CreateFSub(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFSub(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -743,10 +747,12 @@ public:
return CreateMul(LHS, RHS, Name, true, false);
}
Value *CreateFMul(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFMul(LC, RC), Name);
+ return Insert(Folder.CreateFMul(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFMul(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -775,10 +781,12 @@ public:
return CreateSDiv(LHS, RHS, Name, true);
}
Value *CreateFDiv(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFDiv(LC, RC), Name);
+ return Insert(Folder.CreateFDiv(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFDiv(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -795,10 +803,12 @@ public:
return Insert(BinaryOperator::CreateSRem(LHS, RHS), Name);
}
Value *CreateFRem(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFRem(LC, RC), Name);
+ return Insert(Folder.CreateFRem(LC, RC, KeepExceptions, KeepRounding),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFRem(LHS, RHS),
FPMathTag, FMF), Name);
}
diff --git a/lib/Analysis/ConstantFolding.cpp b/lib/Analysis/ConstantFolding.cpp
index fcafb41..494b678 100644
--- a/lib/Analysis/ConstantFolding.cpp
+++ b/lib/Analysis/ConstantFolding.cpp
@@ -622,8 +622,9 @@ static Constant *ConstantFoldLoadInst(const LoadInst *LI, const DataLayout *TD){
/// Attempt to symbolically evaluate the result of a binary operator merging
/// these together. If target data info is available, it is provided as DL,
/// otherwise DL is null.
-static Constant *SymbolicallyEvaluateBinop(unsigned Opc, Constant *Op0,
- Constant *Op1, const DataLayout *DL){
+static Constant *SymbolicallyEvaluateBinop(unsigned Opc,
+ Constant *Op0, Constant *Op1,
+ const DataLayout *DL) {
// SROA
// Fold (and 0xffffffff00000000, (shl x, 32)) -> shl.
@@ -901,6 +902,14 @@ static Constant *SymbolicallyEvaluateGEP(ArrayRef<Constant *> Ops,
Constant *llvm::ConstantFoldInstruction(Instruction *I,
const DataLayout *TD,
const TargetLibraryInfo *TLI) {
+ bool KeepExceptions = false;
+ bool KeepRounding = false;
+
+ if (isa<FPMathOperator>(I)) {
+ KeepExceptions = I->hasKeepExceptions();
+ KeepRounding = I->hasKeepRounding();
+ }
+
// Handle PHI nodes quickly here...
if (PHINode *PN = dyn_cast<PHINode>(I)) {
Constant *CommonValue = nullptr;
@@ -919,7 +928,8 @@ Constant *llvm::ConstantFoldInstruction(Instruction *I,
return nullptr;
// Fold the PHI's operands.
if (ConstantExpr *NewC = dyn_cast<ConstantExpr>(C))
- C = ConstantFoldConstantExpression(NewC, TD, TLI);
+ C = ConstantFoldConstantExpression(NewC, TD, TLI,
+ KeepExceptions, KeepRounding);
// If the incoming value is a different constant to
// the one we saw previously, then give up.
if (CommonValue && C != CommonValue)
@@ -942,7 +952,8 @@ Constant *llvm::ConstantFoldInstruction(Instruction *I,
// Fold the Instruction's operands.
if (ConstantExpr *NewCE = dyn_cast<ConstantExpr>(Op))
- Op = ConstantFoldConstantExpression(NewCE, TD, TLI);
+ Op = ConstantFoldConstantExpression(NewCE, TD, TLI,
+ KeepExceptions, KeepRounding);
Ops.push_back(Op);
}
@@ -967,13 +978,15 @@ Constant *llvm::ConstantFoldInstruction(Instruction *I,
EVI->getIndices());
}
- return ConstantFoldInstOperands(I->getOpcode(), I->getType(), Ops, TD, TLI);
+ return ConstantFoldInstOperands(I->getOpcode(), I->getType(), Ops, TD, TLI,
+ KeepExceptions, KeepRounding);
}
static Constant *
ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
const TargetLibraryInfo *TLI,
- SmallPtrSetImpl<ConstantExpr *> &FoldedOps) {
+ SmallPtrSetImpl<ConstantExpr *> &FoldedOps,
+ bool KeepExceptions, bool KeepRounding) {
SmallVector<Constant *, 8> Ops;
for (User::const_op_iterator i = CE->op_begin(), e = CE->op_end(); i != e;
++i) {
@@ -982,7 +995,8 @@ ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
// a ConstantExpr, we don't have to process it again.
if (ConstantExpr *NewCE = dyn_cast<ConstantExpr>(NewC)) {
if (FoldedOps.insert(NewCE).second)
- NewC = ConstantFoldConstantExpressionImpl(NewCE, TD, TLI, FoldedOps);
+ NewC = ConstantFoldConstantExpressionImpl(NewCE, TD, TLI, FoldedOps,
+ KeepExceptions, KeepRounding);
}
Ops.push_back(NewC);
}
@@ -990,7 +1004,8 @@ ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
if (CE->isCompare())
return ConstantFoldCompareInstOperands(CE->getPredicate(), Ops[0], Ops[1],
TD, TLI);
- return ConstantFoldInstOperands(CE->getOpcode(), CE->getType(), Ops, TD, TLI);
+ return ConstantFoldInstOperands(CE->getOpcode(), CE->getType(), Ops, TD, TLI,
+ KeepExceptions, KeepRounding);
}
/// Attempt to fold the constant expression
@@ -998,9 +1013,12 @@ ConstantFoldConstantExpressionImpl(const ConstantExpr *CE, const DataLayout *TD,
/// result is returned, if not, null is returned.
Constant *llvm::ConstantFoldConstantExpression(const ConstantExpr *CE,
const DataLayout *TD,
- const TargetLibraryInfo *TLI) {
+ const TargetLibraryInfo *TLI,
+ bool KeepExceptions,
+ bool KeepRounding) {
SmallPtrSet<ConstantExpr *, 4> FoldedOps;
- return ConstantFoldConstantExpressionImpl(CE, TD, TLI, FoldedOps);
+ return ConstantFoldConstantExpressionImpl(CE, TD, TLI, FoldedOps,
+ KeepExceptions, KeepRounding);
}
/// Attempt to constant fold an instruction with the
@@ -1016,7 +1034,9 @@ Constant *llvm::ConstantFoldConstantExpression(const ConstantExpr *CE,
Constant *llvm::ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
ArrayRef<Constant *> Ops,
const DataLayout *TD,
- const TargetLibraryInfo *TLI) {
+ const TargetLibraryInfo *TLI,
+ bool KeepExceptions,
+ bool KeepRounding) {
// Handle easy binops first.
if (Instruction::isBinaryOp(Opcode)) {
if (isa<ConstantExpr>(Ops[0]) || isa<ConstantExpr>(Ops[1])) {
@@ -1024,7 +1044,8 @@ Constant *llvm::ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
return C;
}
- return ConstantExpr::get(Opcode, Ops[0], Ops[1]);
+ return ConstantExpr::get(Opcode, Ops[0], Ops[1], 0, nullptr,
+ KeepExceptions, KeepRounding);
}
switch (Opcode) {
diff --git a/lib/IR/ConstantFold.h b/lib/IR/ConstantFold.h
index ee10877..94f83bb 100644
--- a/lib/IR/ConstantFold.h
+++ b/lib/IR/ConstantFold.h
@@ -44,8 +44,8 @@ namespace llvm {
Constant *ConstantFoldInsertValueInstruction(Constant *Agg, Constant *Val,
ArrayRef<unsigned> Idxs);
Constant *ConstantFoldBinaryInstruction(unsigned Opcode, Constant *V1,
- Constant *V2, bool KeepFPExceptions,
- bool KeepFPRounding);
+ Constant *V2, bool KeepExceptions,
+ bool KeepRounding);
Constant *ConstantFoldCompareInstruction(unsigned short predicate,
Constant *C1, Constant *C2);
Constant *ConstantFoldGetElementPtr(Constant *C, bool inBounds,
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index b443368..50a6e1d 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -2241,8 +2241,10 @@ Constant *ConstantExpr::getAdd(Constant *C1, Constant *C2,
return get(Instruction::Add, C1, C2, Flags);
}
-Constant *ConstantExpr::getFAdd(Constant *C1, Constant *C2) {
- return get(Instruction::FAdd, C1, C2);
+Constant *ConstantExpr::getFAdd(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FAdd, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
@@ -2252,19 +2254,25 @@ Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
return get(Instruction::Sub, C1, C2, Flags);
}
-Constant *ConstantExpr::getFSub(Constant *C1, Constant *C2) {
- return get(Instruction::FSub, C1, C2);
+Constant *ConstantExpr::getFSub(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FSub, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getMul(Constant *C1, Constant *C2,
- bool HasNUW, bool HasNSW) {
+ bool HasNUW, bool HasNSW,
+ bool KeepExceptions, bool KeepRounding) {
unsigned Flags = (HasNUW ? OverflowingBinaryOperator::NoUnsignedWrap : 0) |
(HasNSW ? OverflowingBinaryOperator::NoSignedWrap : 0);
- return get(Instruction::Mul, C1, C2, Flags);
+ return get(Instruction::Mul, C1, C2, Flags, nullptr, KeepExceptions,
+ KeepRounding);
}
-Constant *ConstantExpr::getFMul(Constant *C1, Constant *C2) {
- return get(Instruction::FMul, C1, C2);
+Constant *ConstantExpr::getFMul(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FMul, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getUDiv(Constant *C1, Constant *C2, bool isExact) {
@@ -2277,8 +2285,10 @@ Constant *ConstantExpr::getSDiv(Constant *C1, Constant *C2, bool isExact) {
isExact ? PossiblyExactOperator::IsExact : 0);
}
-Constant *ConstantExpr::getFDiv(Constant *C1, Constant *C2) {
- return get(Instruction::FDiv, C1, C2);
+Constant *ConstantExpr::getFDiv(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FDiv, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getURem(Constant *C1, Constant *C2) {
@@ -2289,8 +2299,10 @@ Constant *ConstantExpr::getSRem(Constant *C1, Constant *C2) {
return get(Instruction::SRem, C1, C2);
}
-Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2) {
- return get(Instruction::FRem, C1, C2);
+Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding) {
+ return get(Instruction::FRem, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding);
}
Constant *ConstantExpr::getAnd(Constant *C1, Constant *C2) {
diff --git a/test/Other/fpenv-constant-fold.ll b/test/Other/fpenv-constant-fold.ll
new file mode 100644
index 0000000..01dbfa0
--- /dev/null
+++ b/test/Other/fpenv-constant-fold.ll
@@ -0,0 +1,130 @@
+; InstCombine is used just for its calls to constant folder.
+; RUN: opt -S -instcombine -o - < %s | FileCheck %s
+
+; Target independent constant folder should not fold floating-point operations
+; that have constant expression as their operands when FEnv access is requested
+; and such folding leads to differences in observable effects at runtime.
+
+define double @do-not-fold-fadd-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fadd-that-can-trap
+; CHECK: fadd
+entry:
+ %val = fadd kexc double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fsub-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fsub-that-can-trap
+; CHECK: fsub
+entry:
+ %val = fsub kexc double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+define double @do-not-fold-fmul-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fmul-that-can-trap
+; CHECK: fmul
+entry:
+ %val = fmul kexc double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-fdiv-that-can-trap
+; CHECK: fdiv
+entry:
+ %val = fdiv kexc double 1.000000e+300, 1.000000e-300
+ ret double %val
+}
+
+define double @do-not-fold-frem-that-can-trap() {
+; CHECK-LABEL: @do-not-fold-frem-that-can-trap
+; CHECK: frem
+entry:
+ %val = frem kexc double 1.000000e+300, 1.000000e-300
+ ret double %val
+}
+
+define double @do-not-fold-fadd-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fadd-because-of-rounding
+; CHECK: fadd
+entry:
+ %val = fadd kround double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fsub-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fsub-because-of-rounding
+; CHECK: fsub
+entry:
+ %val = fsub kround double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fmul-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fmul-because-of-rounding
+; CHECK: fmul
+entry:
+ %val = fmul kround double 0.101, 0.93
+ ret double %val
+}
+
+define double @do-not-fold-fdiv-because-of-rounding() {
+; CHECK-LABEL: @do-not-fold-fdiv-because-of-rounding
+; CHECK: fdiv
+entry:
+ %val = fdiv kround double 0.101, 0.93
+ ret double %val
+}
+
+; "frem" instruction falls into the same category, but none of tried test-cases
+; could raised inexact exception.
+
+define double @fold-fadd-that-cant-trap() {
+; CHECK-LABEL: @fold-fadd-that-cant-trap
+; CHECK-NOT: fadd
+entry:
+ %val = fadd kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fsub-that-cant-trap() {
+; CHECK-LABEL: @fold-fsub-that-cant-trap
+; CHECK-NOT: fsub
+entry:
+ %val = fsub kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fmul-that-cant-trap() {
+; CHECK-LABEL: @fold-fmul-that-cant-trap
+; CHECK-NOT: fmul
+entry:
+ %val = fmul kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fdiv-that-cant-trap() {
+; CHECK-LABEL: @fold-fdiv-that-cant-trap
+; CHECK-NOT: fdiv
+entry:
+ %val = fdiv kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-frem-that-cant-trap() {
+; CHECK-LABEL: @fold-frem-that-cant-trap
+; CHECK-NOT: frem
+entry:
+ %val = frem kexc kround double 1.0, 1.0
+ ret double %val
+}
+
+define double @correcly-handle-nested-expressions() {
+; CHECK-LABEL: @correcly-handle-nested-expressions
+; CHECK: fmul
+entry:
+ %val1 = fmul kexc double 1.000000e+300, 1.000000e+300
+ %val2 = fdiv kexc double %val1, 1.000000e+300
+ ret double %val2
+}
--
1.8.4
-------------- next part --------------
>From cfc965006dbf13176a24fbd39346b63c876e105c Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Wed, 26 Nov 2014 19:50:50 +0200
Subject: [PATCH 06/12] Do not fold constants on reading in IR asm/bitcode
---
lib/AsmParser/LLParser.cpp | 3 ++-
lib/Bitcode/Reader/BitcodeReader.cpp | 2 +-
test/Assembler/do-not-fold-fp-consts.ll | 36 +++++++++++++++++++++++++++++++++
3 files changed, 39 insertions(+), 2 deletions(-)
create mode 100644 test/Assembler/do-not-fold-fp-consts.ll
diff --git a/lib/AsmParser/LLParser.cpp b/lib/AsmParser/LLParser.cpp
index 0ad921a..322eccf 100644
--- a/lib/AsmParser/LLParser.cpp
+++ b/lib/AsmParser/LLParser.cpp
@@ -2733,7 +2733,8 @@ bool LLParser::ParseValID(ValID &ID, PerFunctionState *PFS) {
if (NUW) Flags |= OverflowingBinaryOperator::NoUnsignedWrap;
if (NSW) Flags |= OverflowingBinaryOperator::NoSignedWrap;
if (Exact) Flags |= PossiblyExactOperator::IsExact;
- Constant *C = ConstantExpr::get(Opc, Val0, Val1, Flags);
+ Constant *C = ConstantExpr::get(Opc, Val0, Val1, Flags, nullptr, true,
+ true);
ID.ConstantVal = C;
ID.Kind = ValID::t_Constant;
return false;
diff --git a/lib/Bitcode/Reader/BitcodeReader.cpp b/lib/Bitcode/Reader/BitcodeReader.cpp
index 484f419..8344101 100644
--- a/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -1630,7 +1630,7 @@ std::error_code BitcodeReader::ParseConstants() {
Flags |= SDivOperator::IsExact;
}
}
- V = ConstantExpr::get(Opc, LHS, RHS, Flags);
+ V = ConstantExpr::get(Opc, LHS, RHS, Flags, nullptr, true, true);
}
break;
}
diff --git a/test/Assembler/do-not-fold-fp-consts.ll b/test/Assembler/do-not-fold-fp-consts.ll
new file mode 100644
index 0000000..6b2ddcb
--- /dev/null
+++ b/test/Assembler/do-not-fold-fp-consts.ll
@@ -0,0 +1,36 @@
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s
+
+define double @do-not-fold-fadd() {
+; CHECK-LABEL: @do-not-fold-fadd
+; CHECK: fadd
+entry:
+ ret double fadd (double 1.000000e+308, double 1.000000e+308)
+}
+
+define double @do-not-fold-fsub() {
+; CHECK-LABEL: @do-not-fold-fsub
+; CHECK: fsub
+entry:
+ ret double fsub (double 1.000000e-308, double 1.000000e+308)
+}
+
+define double @do-not-fold-fmul() {
+; CHECK-LABEL: @do-not-fold-fmul
+; CHECK: fmul
+entry:
+ ret double fmul (double 1.000000e+300, double 1.000000e+300)
+}
+
+define double @do-not-fold-fdiv() {
+; CHECK-LABEL: @do-not-fold-fdiv
+; CHECK: fdiv
+entry:
+ ret double fdiv (double 1.000000e+300, double 1.000000e-300)
+}
+
+define double @do-not-fold-frem() {
+; CHECK-LABEL: @do-not-fold-frem
+; CHECK: frem
+entry:
+ ret double frem (double 1.000000e+300, double 1.000000e-300)
+}
--
1.8.4
-------------- next part --------------
>From d49d5ff522c7ed624c809fad49509cf3fb99c527 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 28 Nov 2014 12:58:45 +0200
Subject: [PATCH 07/12] Prevent undesired folding by InstSimplify
---
lib/Analysis/InstructionSimplify.cpp | 9 ++++--
test/Transforms/InstSimplify/fast-math.ll | 48 +++++++++++++++++++++++++++++++
2 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/lib/Analysis/InstructionSimplify.cpp b/lib/Analysis/InstructionSimplify.cpp
index 3fbbd7c..357f76b 100644
--- a/lib/Analysis/InstructionSimplify.cpp
+++ b/lib/Analysis/InstructionSimplify.cpp
@@ -796,7 +796,8 @@ static Value *SimplifyFAddInst(Value *Op0, Value *Op1, FastMathFlags FMF,
if (Constant *CRHS = dyn_cast<Constant>(Op1)) {
Constant *Ops[] = { CLHS, CRHS };
return ConstantFoldInstOperands(Instruction::FAdd, CLHS->getType(),
- Ops, Q.DL, Q.TLI);
+ Ops, Q.DL, Q.TLI,
+ FMF.keepExceptions(), FMF.keepRounding());
}
// Canonicalize the constant to the RHS.
@@ -838,7 +839,8 @@ static Value *SimplifyFSubInst(Value *Op0, Value *Op1, FastMathFlags FMF,
if (Constant *CRHS = dyn_cast<Constant>(Op1)) {
Constant *Ops[] = { CLHS, CRHS };
return ConstantFoldInstOperands(Instruction::FSub, CLHS->getType(),
- Ops, Q.DL, Q.TLI);
+ Ops, Q.DL, Q.TLI,
+ FMF.keepExceptions(), FMF.keepRounding());
}
}
@@ -876,7 +878,8 @@ static Value *SimplifyFMulInst(Value *Op0, Value *Op1,
if (Constant *CRHS = dyn_cast<Constant>(Op1)) {
Constant *Ops[] = { CLHS, CRHS };
return ConstantFoldInstOperands(Instruction::FMul, CLHS->getType(),
- Ops, Q.DL, Q.TLI);
+ Ops, Q.DL, Q.TLI,
+ FMF.keepExceptions(), FMF.keepRounding());
}
// Canonicalize the constant to the RHS.
diff --git a/test/Transforms/InstSimplify/fast-math.ll b/test/Transforms/InstSimplify/fast-math.ll
index 71d1ed8..255a283 100644
--- a/test/Transforms/InstSimplify/fast-math.ll
+++ b/test/Transforms/InstSimplify/fast-math.ll
@@ -105,3 +105,51 @@ define float @nofold_fadd_x_0(float %a) {
; CHECK: ret float %no_zero
ret float %no_zero
}
+
+; CHECK-LABEL: @nofold_fadd_kexc(
+define double @nofold_fadd_kexc() {
+; CHECK: fadd
+entry:
+ %val = fadd kexc double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fsub_kexc(
+define double @nofold_fsub_kexc() {
+; CHECK: fsub
+entry:
+ %val = fsub kexc double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fmul_kexc(
+define double @nofold_fmul_kexc() {
+; CHECK: fmul
+entry:
+ %val = fmul kexc double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fadd_kround(
+define double @nofold_fadd_kround() {
+; CHECK: fadd
+entry:
+ %val = fadd kround double 1.000000e+308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fsub_kround(
+define double @nofold_fsub_kround() {
+; CHECK: fsub
+entry:
+ %val = fsub kround double 1.000000e-308, 1.000000e+308
+ ret double %val
+}
+
+; CHECK-LABEL: @nofold_fmul_kround(
+define double @nofold_fmul_kround() {
+; CHECK: fmul
+entry:
+ %val = fmul kround double 1.000000e+300, 1.000000e+300
+ ret double %val
+}
--
1.8.4
-------------- next part --------------
>From 6d621e176f56532a055ab96be5f63d56de64988c Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 28 Nov 2014 15:53:44 +0200
Subject: [PATCH 08/12] Do not simplify expressions with FPEnv access
ConstantExpression can't store these flags, which can lead to
going against them.
---
include/llvm/IR/Constants.h | 4 ++--
lib/Analysis/ConstantFolding.cpp | 2 +-
lib/IR/Constants.cpp | 8 +++++++-
test/Transforms/EarlyCSE/fpenv.ll | 24 ++++++++++++++++++++++++
4 files changed, 34 insertions(+), 4 deletions(-)
create mode 100644 test/Transforms/EarlyCSE/fpenv.ll
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index c96136b..6bb0dc8 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -1029,8 +1029,8 @@ public:
/// \param OnlyIfReducedTy see \a getWithOperands() docs.
static Constant *get(unsigned Opcode, Constant *C1, Constant *C2,
unsigned Flags = 0, Type *OnlyIfReducedTy = nullptr,
- bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepExceptions = false, bool KeepRounding = false,
+ bool Strict = false);
/// \brief Return an ICmp or FCmp comparison operator constant expression.
///
diff --git a/lib/Analysis/ConstantFolding.cpp b/lib/Analysis/ConstantFolding.cpp
index 494b678..5b20ce2 100644
--- a/lib/Analysis/ConstantFolding.cpp
+++ b/lib/Analysis/ConstantFolding.cpp
@@ -1045,7 +1045,7 @@ Constant *llvm::ConstantFoldInstOperands(unsigned Opcode, Type *DestTy,
}
return ConstantExpr::get(Opcode, Ops[0], Ops[1], 0, nullptr,
- KeepExceptions, KeepRounding);
+ KeepExceptions, KeepRounding, /* Strict = */ true);
}
switch (Opcode) {
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index 50a6e1d..0145198 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -1848,7 +1848,8 @@ Constant *ConstantExpr::getAddrSpaceCast(Constant *C, Type *DstTy,
Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
unsigned Flags, Type *OnlyIfReducedTy,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
// Check the operands for consistency first.
assert(Opcode >= Instruction::BinaryOpsBegin &&
Opcode < Instruction::BinaryOpsEnd &&
@@ -1923,6 +1924,11 @@ Constant *ConstantExpr::get(unsigned Opcode, Constant *C1, Constant *C2,
if (OnlyIfReducedTy == C1->getType())
return nullptr;
+ // ConstantExpression can't store these flags, which can lead to going against
+ // them.
+ if (Strict && (KeepExceptions || KeepRounding))
+ return nullptr;
+
Constant *ArgVec[] = { C1, C2 };
ConstantExprKeyType Key(Opcode, ArgVec, 0, Flags);
diff --git a/test/Transforms/EarlyCSE/fpenv.ll b/test/Transforms/EarlyCSE/fpenv.ll
new file mode 100644
index 0000000..aba632c
--- /dev/null
+++ b/test/Transforms/EarlyCSE/fpenv.ll
@@ -0,0 +1,24 @@
+; %val.true and %val.false should not be moved inside phi-node as constant
+; expressions as such transformation looses fast-math flags.
+; RUN: opt < %s -S -early-cse | FileCheck %s
+
+define double @do-not-turn-into-constexpr(double %x) {
+; CHECK-LABEL: @do-not-turn-into-constexpr(
+; CHECK: %val.true = fmul
+; CHECK: %val.false = fmul
+entry:
+ %cmp = fcmp oeq double %x, 0.000000e+00
+ br i1 %cmp, label %cond.true, label %cond.false
+
+cond.true: ; preds = %entry
+ %val.true = fmul kexc double 1.000000e+300, 1.000000e+300
+ br label %cond.end
+
+cond.false: ; preds = %entry
+ %val.false = fmul kexc double 1.000000e-300, 1.000000e-300
+ br label %cond.end
+
+cond.end: ; preds = %cond.false, %cond.true
+ %cond = phi double [ %val.true, %cond.true ], [ %val.false, %cond.false ]
+ ret double %cond
+}
--
1.8.4
-------------- next part --------------
>From 50b9c60f9809e3e8f2381d387b463800583d9076 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 12 Dec 2014 16:35:01 +0200
Subject: [PATCH 09/12] Make Strict flag available for more clients
---
include/llvm/Analysis/TargetFolder.h | 35 +++++++++++++++++++++++------------
include/llvm/IR/ConstantFolder.h | 35 +++++++++++++++++++++++------------
include/llvm/IR/Constants.h | 18 ++++++++++++------
lib/IR/Constants.cpp | 30 ++++++++++++++++++------------
4 files changed, 76 insertions(+), 42 deletions(-)
diff --git a/include/llvm/Analysis/TargetFolder.h b/include/llvm/Analysis/TargetFolder.h
index 52e2721..a20c499 100644
--- a/include/llvm/Analysis/TargetFolder.h
+++ b/include/llvm/Analysis/TargetFolder.h
@@ -53,8 +53,10 @@ public:
}
Constant *CreateFAdd(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
@@ -62,20 +64,25 @@ public:
}
Constant *CreateFSub(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
+ bool KeepRounding = false,
+ bool Strict = false) const {
return Fold(ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW,
- KeepExceptions, KeepRounding));
+ KeepExceptions, KeepRounding, Strict));
}
Constant *CreateFMul(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS, bool isExact = false)const{
return Fold(ConstantExpr::getUDiv(LHS, RHS, isExact));
@@ -85,8 +92,10 @@ public:
}
Constant *CreateFDiv(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return Fold(ConstantExpr::getURem(LHS, RHS));
@@ -96,8 +105,10 @@ public:
}
Constant *CreateFRem(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return Fold(ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding));
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/IR/ConstantFolder.h b/include/llvm/IR/ConstantFolder.h
index 93f25dd..daa88f5 100644
--- a/include/llvm/IR/ConstantFolder.h
+++ b/include/llvm/IR/ConstantFolder.h
@@ -37,8 +37,10 @@ public:
}
Constant *CreateFAdd(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFAdd(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateSub(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
@@ -46,20 +48,25 @@ public:
}
Constant *CreateFSub(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFSub(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateMul(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
+ bool KeepRounding = false,
+ bool Strict = false) const {
return ConstantExpr::getMul(LHS, RHS, HasNUW, HasNSW, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *CreateFMul(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFMul(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateUDiv(Constant *LHS, Constant *RHS,
bool isExact = false) const {
@@ -71,8 +78,10 @@ public:
}
Constant *CreateFDiv(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFDiv(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateURem(Constant *LHS, Constant *RHS) const {
return ConstantExpr::getURem(LHS, RHS);
@@ -82,8 +91,10 @@ public:
}
Constant *CreateFRem(Constant *LHS, Constant *RHS,
bool KeepExceptions = false,
- bool KeepRounding = false) const {
- return ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding);
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index 6bb0dc8..6a4744f 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -843,29 +843,35 @@ public:
bool HasNUW = false, bool HasNSW = false);
static Constant *getFAdd(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getSub(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false);
static Constant *getFSub(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getMul(Constant *C1, Constant *C2,
bool HasNUW = false, bool HasNSW = false,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getFMul(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getUDiv(Constant *C1, Constant *C2, bool isExact = false);
static Constant *getSDiv(Constant *C1, Constant *C2, bool isExact = false);
static Constant *getFDiv(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getURem(Constant *C1, Constant *C2);
static Constant *getSRem(Constant *C1, Constant *C2);
static Constant *getFRem(Constant *C1, Constant *C2,
bool KeepExceptions = false,
- bool KeepRounding = false);
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getAnd(Constant *C1, Constant *C2);
static Constant *getOr(Constant *C1, Constant *C2);
static Constant *getXor(Constant *C1, Constant *C2);
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index 0145198..cdbe424 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -2248,9 +2248,10 @@ Constant *ConstantExpr::getAdd(Constant *C1, Constant *C2,
}
Constant *ConstantExpr::getFAdd(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FAdd, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
@@ -2261,24 +2262,27 @@ Constant *ConstantExpr::getSub(Constant *C1, Constant *C2,
}
Constant *ConstantExpr::getFSub(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FSub, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getMul(Constant *C1, Constant *C2,
bool HasNUW, bool HasNSW,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
unsigned Flags = (HasNUW ? OverflowingBinaryOperator::NoUnsignedWrap : 0) |
(HasNSW ? OverflowingBinaryOperator::NoSignedWrap : 0);
return get(Instruction::Mul, C1, C2, Flags, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getFMul(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FMul, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getUDiv(Constant *C1, Constant *C2, bool isExact) {
@@ -2292,9 +2296,10 @@ Constant *ConstantExpr::getSDiv(Constant *C1, Constant *C2, bool isExact) {
}
Constant *ConstantExpr::getFDiv(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FDiv, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getURem(Constant *C1, Constant *C2) {
@@ -2306,9 +2311,10 @@ Constant *ConstantExpr::getSRem(Constant *C1, Constant *C2) {
}
Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2,
- bool KeepExceptions, bool KeepRounding) {
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
return get(Instruction::FRem, C1, C2, 0, nullptr, KeepExceptions,
- KeepRounding);
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getAnd(Constant *C1, Constant *C2) {
--
1.8.4
-------------- next part --------------
>From c8f9258718d263adf187942188608245dca9ad03 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 12 Dec 2014 16:40:08 +0200
Subject: [PATCH 10/12] Use Strict in IRBuilder
Try to create constant expression, but skip it in case of conflicts
with FPEnv access flags.
This is hard to test in LLVM as IRBuilder isn't extensively used here.
The issue doesn't even appear in tests with IR, only in Clang.
---
include/llvm/IR/IRBuilder.h | 23 ++++++++++++++---------
1 file changed, 14 insertions(+), 9 deletions(-)
diff --git a/include/llvm/IR/IRBuilder.h b/include/llvm/IR/IRBuilder.h
index 3bf8316..d18b5fa 100644
--- a/include/llvm/IR/IRBuilder.h
+++ b/include/llvm/IR/IRBuilder.h
@@ -703,8 +703,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFAdd(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFAdd(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFAdd(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -727,8 +728,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFSub(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFSub(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFSub(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -751,8 +753,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFMul(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFMul(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFMul(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -785,8 +788,9 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFDiv(LC, RC, KeepExceptions, KeepRounding),
- Name);
+ if (Constant *C = Folder.CreateFDiv(LC, RC, KeepExceptions,
+ KeepRounding, true))
+ return Insert(C, Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFDiv(LHS, RHS),
FPMathTag, FMF), Name);
}
@@ -807,7 +811,8 @@ public:
bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFRem(LC, RC, KeepExceptions, KeepRounding),
+ return Insert(Folder.CreateFRem(LC, RC, KeepExceptions, KeepRounding,
+ true),
Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFRem(LHS, RHS),
FPMathTag, FMF), Name);
--
1.8.4
-------------- next part --------------
>From 6e7431963f160116fd8562e149a3a01c6a582e13 Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Mon, 15 Dec 2014 14:18:03 +0200
Subject: [PATCH 11/12] Don't convert fpops to constexprs in SCCP
---
lib/Transforms/Scalar/SCCP.cpp | 18 ++++++++++++----
.../SCCP/do-not-convert-fpop-to-constexpr.ll | 24 ++++++++++++++++++++++
2 files changed, 38 insertions(+), 4 deletions(-)
create mode 100644 test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll
diff --git a/lib/Transforms/Scalar/SCCP.cpp b/lib/Transforms/Scalar/SCCP.cpp
index 05b9608..a8bd9eb 100644
--- a/lib/Transforms/Scalar/SCCP.cpp
+++ b/lib/Transforms/Scalar/SCCP.cpp
@@ -851,10 +851,20 @@ void SCCPSolver::visitBinaryOperator(Instruction &I) {
LatticeVal &IV = ValueState[&I];
if (IV.isOverdefined()) return;
- if (V1State.isConstant() && V2State.isConstant())
- return markConstant(IV, &I,
- ConstantExpr::get(I.getOpcode(), V1State.getConstant(),
- V2State.getConstant()));
+ if (V1State.isConstant() && V2State.isConstant()) {
+ bool KeepExceptions = false;
+ bool KeepRounding = false;
+
+ if (isa<FPMathOperator>(&I)) {
+ KeepExceptions = I.hasKeepExceptions();
+ KeepRounding = I.hasKeepRounding();
+ }
+
+ if (Constant *C = ConstantExpr::get(I.getOpcode(), V1State.getConstant(),
+ V2State.getConstant(), 0, nullptr,
+ KeepExceptions, KeepRounding, true))
+ return markConstant(IV, &I, C);
+ }
// If something is undef, wait for it to resolve.
if (!V1State.isOverdefined() && !V2State.isOverdefined())
diff --git a/test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll b/test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll
new file mode 100644
index 0000000..df3275a
--- /dev/null
+++ b/test/Transforms/SCCP/do-not-convert-fpop-to-constexpr.ll
@@ -0,0 +1,24 @@
+; fmuls shouldn't be converted to constant expressions.
+; RUN: opt < %s -sccp -S -enable-except-access-fp-math | FileCheck %s
+
+define double @do-not-convert-fpop-to-constexpr(double %x) {
+; CHECK-LABEL: @do-not-convert-fpop-to-constexpr(
+; CHECK: fmul
+; CHECK: fmul
+; CHECK-NOT: select
+entry:
+ %cmp = fcmp oeq double %x, 0.000000e+00
+ br i1 %cmp, label %cond.true, label %cond.false
+
+cond.true:
+ %val.true = fmul kexc double 1.000000e+300, 1.000000e+300
+ br label %cond.end
+
+cond.false:
+ %val.false = fmul kexc double 1.000000e-300, 1.000000e-300
+ br label %cond.end
+
+cond.end:
+ %cond = phi double [ %val.true, %cond.true ], [ %val.false, %cond.false ]
+ ret double %cond
+}
--
1.8.4
-------------- next part --------------
>From a18c79d9067a578a1b929b3e1545088078b9bc8f Mon Sep 17 00:00:00 2001
From: Sergey Dmitrouk <sdmitrouk at accesssoftek.com>
Date: Fri, 19 Dec 2014 15:08:59 +0200
Subject: [PATCH 12/12] Fix hanging of InstCombine
InstCombine shouldn't hang in non-terminating sequence of conversions like:
(a + b) => (a - (-b)) => (a + b) => etc.
---
.../InstCombine/InstructionCombining.cpp | 6 ++++
.../do-not-hand-on-fpops-with-fpenv-access.ll | 42 ++++++++++++++++++++++
2 files changed, 48 insertions(+)
create mode 100644 test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll
diff --git a/lib/Transforms/InstCombine/InstructionCombining.cpp b/lib/Transforms/InstCombine/InstructionCombining.cpp
index d27801b..1022321 100644
--- a/lib/Transforms/InstCombine/InstructionCombining.cpp
+++ b/lib/Transforms/InstCombine/InstructionCombining.cpp
@@ -2634,6 +2634,12 @@ bool InstCombiner::run() {
Instruction *I = Worklist.RemoveOne();
if (I == nullptr) continue; // skip null values.
+ // Skip floating-point operations with FP environment access flags as when
+ // they can't be folded to constants simplification never ends.
+ if (isa<FPMathOperator>(I) &&
+ (I->hasKeepExceptions() || I->hasKeepRounding()))
+ continue;
+
// Check to see if we can DCE the instruction.
if (isInstructionTriviallyDead(I, TLI)) {
DEBUG(dbgs() << "IC: DCE: " << *I << '\n');
diff --git a/test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll b/test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll
new file mode 100644
index 0000000..8ea33f0
--- /dev/null
+++ b/test/Transforms/InstCombine/do-not-hand-on-fpops-with-fpenv-access.ll
@@ -0,0 +1,42 @@
+; RUN: opt < %s -instcombine -S
+
+; InstCombine shouldn't hang in non-terminating sequence of conversions like:
+; (a + b) => (a - (-b)) => (a + b) => etc.
+
+define double @test1() {
+entry:
+ %add = fadd kexc double 0x3FF921FB54442D18, 0x3870000000000000
+ ret double %add
+}
+
+define double @test2(double %x) {
+entry:
+ %sub = fsub kexc kround double -0.000000e+00, 1.000000e+00
+ %div = fdiv kexc kround double %sub, %x
+ ret double %div
+}
+
+define double @test3() {
+entry:
+ %sub = fsub kexc kround double -0.000000e+00, 0x400921FB54442D18
+ ret double %sub
+}
+
+define double @test4(i32 %y) {
+entry:
+ %tobool = icmp ne i32 %y, 0
+ br i1 %tobool, label %cond.true, label %cond.false
+
+cond.true:
+ %sub = fsub kexc kround double -0.000000e+00, 0x400921FB54442D18
+ %div = fdiv kexc kround double %sub, 2.000000e+00
+ br label %cond.end
+
+cond.false:
+ %div1 = fdiv kexc kround double 0x400921FB54442D18, 2.000000e+00
+ br label %cond.end
+
+cond.end:
+ %cond = phi double [ %div, %cond.true ], [ %div1, %cond.false ]
+ ret double %cond
+}
--
1.8.4
-------------- next part --------------
include/llvm/Analysis/TargetFolder.h | 8 ++-
include/llvm/IR/ConstantFolder.h | 8 ++-
include/llvm/IR/Constants.h | 11 ++--
include/llvm/IR/IRBuilder.h | 7 ++-
lib/Analysis/ValueTracking.cpp | 15 ++++--
lib/CodeGen/SelectionDAG/SelectionDAG.cpp | 4 +-
lib/IR/ConstantFold.cpp | 69 +++++++++++++------------
lib/IR/Constants.cpp | 41 ++++++++++-----
test/CodeGen/Generic/selectiondag-fenv.ll | 85 ++++++++++++++++++++++++-------
test/Other/fpenv-constant-fold.ll | 52 +++++++++----------
10 files changed, 189 insertions(+), 111 deletions(-)
diff --git a/include/llvm/Analysis/TargetFolder.h b/include/llvm/Analysis/TargetFolder.h
index 4c6f2bc..a20c499 100644
--- a/include/llvm/Analysis/TargetFolder.h
+++ b/include/llvm/Analysis/TargetFolder.h
@@ -103,8 +103,12 @@ public:
Constant *CreateSRem(Constant *LHS, Constant *RHS) const {
return Fold(ConstantExpr::getSRem(LHS, RHS));
}
- Constant *CreateFRem(Constant *LHS, Constant *RHS) const {
- return Fold(ConstantExpr::getFRem(LHS, RHS));
+ Constant *CreateFRem(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return Fold(ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict));
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/IR/ConstantFolder.h b/include/llvm/IR/ConstantFolder.h
index e688c91..daa88f5 100644
--- a/include/llvm/IR/ConstantFolder.h
+++ b/include/llvm/IR/ConstantFolder.h
@@ -89,8 +89,12 @@ public:
Constant *CreateSRem(Constant *LHS, Constant *RHS) const {
return ConstantExpr::getSRem(LHS, RHS);
}
- Constant *CreateFRem(Constant *LHS, Constant *RHS) const {
- return ConstantExpr::getFRem(LHS, RHS);
+ Constant *CreateFRem(Constant *LHS, Constant *RHS,
+ bool KeepExceptions = false,
+ bool KeepRounding = false,
+ bool Strict = false) const {
+ return ConstantExpr::getFRem(LHS, RHS, KeepExceptions, KeepRounding,
+ Strict);
}
Constant *CreateShl(Constant *LHS, Constant *RHS,
bool HasNUW = false, bool HasNSW = false) const {
diff --git a/include/llvm/IR/Constants.h b/include/llvm/IR/Constants.h
index a2cf719..6a4744f 100644
--- a/include/llvm/IR/Constants.h
+++ b/include/llvm/IR/Constants.h
@@ -868,7 +868,10 @@ public:
bool Strict = false);
static Constant *getURem(Constant *C1, Constant *C2);
static Constant *getSRem(Constant *C1, Constant *C2);
- static Constant *getFRem(Constant *C1, Constant *C2);
+ static Constant *getFRem(Constant *C1, Constant *C2,
+ bool KeepExceptions = false,
+ bool KeepRounding = false,
+ bool Strict = false);
static Constant *getAnd(Constant *C1, Constant *C2);
static Constant *getOr(Constant *C1, Constant *C2);
static Constant *getXor(Constant *C1, Constant *C2);
@@ -1156,9 +1159,9 @@ public:
return V->getValueID() == ConstantExprVal;
}
- /// Checks that floating point operation is safe with regard to exceptions.
- /// Returns false for unsupported types of Value.
- static bool isExceptionSafeFPOp(const Value *V);
+ /// Evaluates floating point operation to examine it, does nothing for non-FP
+ /// operations. Returns false for unsupported values.
+ static bool getFPOpExceptions(const Value *V, APFloat::opStatus &S);
private:
// Shadow Value::setValueSubclassData with a private forwarding method so that
diff --git a/include/llvm/IR/IRBuilder.h b/include/llvm/IR/IRBuilder.h
index 38986eb..d18b5fa 100644
--- a/include/llvm/IR/IRBuilder.h
+++ b/include/llvm/IR/IRBuilder.h
@@ -807,10 +807,13 @@ public:
return Insert(BinaryOperator::CreateSRem(LHS, RHS), Name);
}
Value *CreateFRem(Value *LHS, Value *RHS, const Twine &Name = "",
- MDNode *FPMathTag = nullptr) {
+ MDNode *FPMathTag = nullptr,
+ bool KeepExceptions = false, bool KeepRounding = false) {
if (Constant *LC = dyn_cast<Constant>(LHS))
if (Constant *RC = dyn_cast<Constant>(RHS))
- return Insert(Folder.CreateFRem(LC, RC), Name);
+ return Insert(Folder.CreateFRem(LC, RC, KeepExceptions, KeepRounding,
+ true),
+ Name);
return Insert(AddFPMathAttributes(BinaryOperator::CreateFRem(LHS, RHS),
FPMathTag, FMF), Name);
}
diff --git a/lib/Analysis/ValueTracking.cpp b/lib/Analysis/ValueTracking.cpp
index aae1b11..8ada931 100644
--- a/lib/Analysis/ValueTracking.cpp
+++ b/lib/Analysis/ValueTracking.cpp
@@ -2567,10 +2567,17 @@ bool llvm::isSafeToSpeculativelyExecute(const Value *V,
case Instruction::FAdd:
case Instruction::FSub:
case Instruction::FDiv:
- if (KeepRounding)
- return false;
- if (KeepExceptions)
- return ConstantExpr::isExceptionSafeFPOp(Inst);
+ case Instruction::FRem:
+ if (KeepExceptions || KeepRounding) {
+ APFloat::opStatus S;
+ if (!ConstantExpr::getFPOpExceptions(Inst, S))
+ return false;
+
+ if (KeepExceptions && S != APFloat::opOK)
+ return false;
+ if (KeepRounding && (S & APFloat::opInexact))
+ return false;
+ }
return true;
case Instruction::UDiv:
case Instruction::URem: {
diff --git a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index 8bfead1..c060832 100644
--- a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -3610,10 +3610,10 @@ bool SelectionDAG::isSafeToOptimizeFPOp(APFloat::opStatus s,
APFloat::opStatus UnsafeOps) const {
if (!TLI->hasFloatingPointExceptions())
return true;
- if (getTarget().Options.AllowFPRoundAccess)
- return false;
if (getTarget().Options.AllowFPExceptAccess)
return s == APFloat::opOK;
+ if (getTarget().Options.AllowFPRoundAccess && (s & APFloat::opInexact))
+ return false;
return (s & UnsafeOps) == APFloat::opOK;
}
diff --git a/lib/IR/ConstantFold.cpp b/lib/IR/ConstantFold.cpp
index b8010cd..c84cb86 100644
--- a/lib/IR/ConstantFold.cpp
+++ b/lib/IR/ConstantFold.cpp
@@ -1162,40 +1162,43 @@ Constant *llvm::ConstantFoldBinaryInstruction(unsigned Opcode,
}
} else if (ConstantFP *CFP1 = dyn_cast<ConstantFP>(C1)) {
if (ConstantFP *CFP2 = dyn_cast<ConstantFP>(C2)) {
- if (!KeepRounding) {
- APFloat C1V = CFP1->getValueAPF();
- APFloat C2V = CFP2->getValueAPF();
- APFloat C3V = C1V; // copy for modification
- APFloat::opStatus s;
- switch (Opcode) {
- default:
+ APFloat C1V = CFP1->getValueAPF();
+ APFloat C2V = CFP2->getValueAPF();
+ APFloat C3V = C1V; // copy for modification
+ APFloat::opStatus S;
+ switch (Opcode) {
+ default:
+ break;
+ case Instruction::FAdd:
+ S = C3V.add(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
break;
- case Instruction::FAdd:
- s = C3V.add(C2V, APFloat::rmNearestTiesToEven);
- if (KeepExceptions && s != APFloat::opOK)
- break;
- return ConstantFP::get(C1->getContext(), C3V);
- case Instruction::FSub:
- s = C3V.subtract(C2V, APFloat::rmNearestTiesToEven);
- if (KeepExceptions && s != APFloat::opOK)
- break;
- return ConstantFP::get(C1->getContext(), C3V);
- case Instruction::FMul:
- s = C3V.multiply(C2V, APFloat::rmNearestTiesToEven);
- if (KeepExceptions && s != APFloat::opOK)
- break;
- return ConstantFP::get(C1->getContext(), C3V);
- case Instruction::FDiv:
- s = C3V.divide(C2V, APFloat::rmNearestTiesToEven);
- if (KeepExceptions && s != APFloat::opOK)
- break;
- return ConstantFP::get(C1->getContext(), C3V);
- case Instruction::FRem:
- s = C3V.mod(C2V, APFloat::rmNearestTiesToEven);
- if (KeepExceptions && s != APFloat::opOK)
- break;
- return ConstantFP::get(C1->getContext(), C3V);
- }
+ return ConstantFP::get(C1->getContext(), C3V);
+ case Instruction::FSub:
+ S = C3V.subtract(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
+ return ConstantFP::get(C1->getContext(), C3V);
+ case Instruction::FMul:
+ S = C3V.multiply(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
+ return ConstantFP::get(C1->getContext(), C3V);
+ case Instruction::FDiv:
+ S = C3V.divide(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
+ return ConstantFP::get(C1->getContext(), C3V);
+ case Instruction::FRem:
+ S = C3V.mod(C2V, APFloat::rmNearestTiesToEven);
+ if ((KeepExceptions && S != APFloat::opOK) ||
+ (KeepRounding && (S & APFloat::opInexact)))
+ break;
+ return ConstantFP::get(C1->getContext(), C3V);
}
}
} else if (VectorType *VTy = dyn_cast<VectorType>(C1->getType())) {
diff --git a/lib/IR/Constants.cpp b/lib/IR/Constants.cpp
index c733bfb..cdbe424 100644
--- a/lib/IR/Constants.cpp
+++ b/lib/IR/Constants.cpp
@@ -327,15 +327,22 @@ static bool canTrapImpl(const Constant *C,
// Otherwise, only specific operations can trap.
unsigned Opcode = CE->getOpcode();
- if (KeepExceptions) {
+ if (KeepExceptions || KeepRounding) {
switch (Opcode) {
- default:
- break;
case Instruction::FMul:
case Instruction::FAdd:
case Instruction::FSub:
case Instruction::FDiv:
- return !ConstantExpr::isExceptionSafeFPOp(CE);
+ case Instruction::FRem: {
+ APFloat::opStatus S;
+ if (!ConstantExpr::getFPOpExceptions(CE, S))
+ return false;
+
+ if (KeepExceptions && S != APFloat::opOK)
+ return false;
+ if (KeepRounding && (S & APFloat::opInexact))
+ return false;
+ }
}
}
switch (Opcode) {
@@ -2303,8 +2310,11 @@ Constant *ConstantExpr::getSRem(Constant *C1, Constant *C2) {
return get(Instruction::SRem, C1, C2);
}
-Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2) {
- return get(Instruction::FRem, C1, C2);
+Constant *ConstantExpr::getFRem(Constant *C1, Constant *C2,
+ bool KeepExceptions, bool KeepRounding,
+ bool Strict) {
+ return get(Instruction::FRem, C1, C2, 0, nullptr, KeepExceptions,
+ KeepRounding, Strict);
}
Constant *ConstantExpr::getAnd(Constant *C1, Constant *C2) {
@@ -3007,7 +3017,7 @@ Instruction *ConstantExpr::getAsInstruction() {
}
}
-bool ConstantExpr::isExceptionSafeFPOp(const Value *V) {
+bool ConstantExpr::getFPOpExceptions(const Value *V, APFloat::opStatus &S) {
unsigned Opcode;
const User *U;
if (const Operator *Op = dyn_cast<Operator>(V)) {
@@ -3026,7 +3036,8 @@ bool ConstantExpr::isExceptionSafeFPOp(const Value *V) {
case Instruction::FMul:
case Instruction::FAdd:
case Instruction::FSub:
- case Instruction::FDiv: {
+ case Instruction::FDiv:
+ case Instruction::FRem: {
ConstantFP *LHSC = dyn_cast<ConstantFP>(U->getOperand(0));
ConstantFP *RHSC = dyn_cast<ConstantFP>(U->getOperand(1));
@@ -3036,24 +3047,26 @@ bool ConstantExpr::isExceptionSafeFPOp(const Value *V) {
APFloat LHS = LHSC->getValueAPF();
APFloat RHS = LHSC->getValueAPF();
- APFloat::opStatus status;
switch (Opcode) {
case Instruction::FMul:
- status = LHS.multiply(RHS, APFloat::rmNearestTiesToEven);
+ S = LHS.multiply(RHS, APFloat::rmNearestTiesToEven);
break;
case Instruction::FAdd:
- status = LHS.add(RHS, APFloat::rmNearestTiesToEven);
+ S = LHS.add(RHS, APFloat::rmNearestTiesToEven);
break;
case Instruction::FSub:
- status = LHS.subtract(RHS, APFloat::rmNearestTiesToEven);
+ S = LHS.subtract(RHS, APFloat::rmNearestTiesToEven);
break;
case Instruction::FDiv:
- status = LHS.divide(RHS, APFloat::rmNearestTiesToEven);
+ S = LHS.divide(RHS, APFloat::rmNearestTiesToEven);
+ break;
+ case Instruction::FRem:
+ S = LHS.mod(RHS, APFloat::rmNearestTiesToEven);
break;
}
- return (status == APFloat::opOK);
+ return true;
}
}
}
diff --git a/test/CodeGen/Generic/selectiondag-fenv.ll b/test/CodeGen/Generic/selectiondag-fenv.ll
index 365ec3d..ed2e3f6 100644
--- a/test/CodeGen/Generic/selectiondag-fenv.ll
+++ b/test/CodeGen/Generic/selectiondag-fenv.ll
@@ -39,54 +39,101 @@ entry:
ret double %val
}
-; "frem" instruction falls into the same category, but none of tried test-cases
-; could indicated difference between processing with and
-; without -enable-except-access-fp-math flag.
+define double @do-not-fold-frem-that-can-trap() {
+; CHECK-LABEL: do-not-fold-frem-that-can-trap:
+; CHECK: cp#0: 1.0
+entry:
+ %val = frem double 1.0, 0.0
+ ret double %val
+}
define double @do-not-fold-fadd-because-of-rounding() {
; CHECK-LABEL: do-not-fold-fadd-because-of-rounding:
-; EXCEPT: cp#0: 2.0
-; ROUND: cp#0: 1.0
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
entry:
- %val = fadd double 1.0, 1.0
+ %val = fadd double 0.101, 0.93
ret double %val
}
define double @do-not-fold-fsub-because-of-rounding() {
; CHECK-LABEL: do-not-fold-fsub-because-of-rounding:
-; EXCEPT: cp#0: 1.0
-; ROUND: cp#0: 3.0
-; ROUND: cp#1: 2.0
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
entry:
- %val = fsub double 3.0, 2.0
+ %val = fsub double 0.101, 0.93
ret double %val
}
define double @do-not-fold-fmul-because-of-rounding() {
; CHECK-LABEL: do-not-fold-fmul-because-of-rounding:
-; EXCEPT: cp#0: 6.0
-; ROUND: cp#0: 2.0
-; ROUND: cp#1: 3.0
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
entry:
- %val = fmul double 2.0, 3.0
+ %val = fmul double 0.101, 0.93
ret double %val
}
define double @do-not-fold-fdiv-because-of-rounding() {
; CHECK-LABEL: do-not-fold-fdiv-because-of-rounding:
+; EXCEPT: cp#0: 1.010000e-01
+; EXCEPT: cp#1: 9.300000e-01
+; ROUND: cp#0: 1.010000e-01
+; ROUND: cp#1: 9.300000e-01
+entry:
+ %val = fdiv double 0.101, 0.93
+ ret double %val
+}
+
+; "frem" instruction falls into the same category, but none of tried test-cases
+; could raised inexact exception.
+
+define double @fold-fadd-without-rounding() {
+; CHECK-LABEL: fold-fadd-without-rounding:
; EXCEPT: cp#0: 2.0
+; ROUND: cp#0: 2.0
+entry:
+ %val = fadd double 1.0, 1.0
+ ret double %val
+}
+
+define double @fold-fsub-without-rounding() {
+; CHECK-LABEL: fold-fsub-without-rounding:
+; EXCEPT: cp#0: 1.0
+; ROUND: cp#0: 1.0
+entry:
+ %val = fsub double 3.0, 2.0
+ ret double %val
+}
+
+define double @fold-fmul-without-rounding() {
+; CHECK-LABEL: fold-fmul-without-rounding:
+; EXCEPT: cp#0: 6.0
; ROUND: cp#0: 6.0
-; ROUND: cp#1: 3.0
+entry:
+ %val = fmul double 2.0, 3.0
+ ret double %val
+}
+
+define double @fold-fdiv-without-rounding() {
+; CHECK-LABEL: fold-fdiv-without-rounding:
+; EXCEPT: cp#0: 2.0
+; ROUND: cp#0: 2.0
entry:
%val = fdiv double 6.0, 3.0
ret double %val
}
-define double @do-not-fold-frem-because-of-rounding() {
-; CHECK-LABEL: do-not-fold-frem-because-of-rounding:
+define double @fold-frem-without-rounding() {
+; CHECK-LABEL: fold-frem-without-rounding:
; EXCEPT: cp#0: 1.0
-; ROUND: cp#0: 7.0
-; ROUND: cp#1: 3.0
+; ROUND: cp#0: 1.0
entry:
%val = frem double 7.0, 3.0
ret double %val
diff --git a/test/Other/fpenv-constant-fold.ll b/test/Other/fpenv-constant-fold.ll
index 45169f9..01dbfa0 100644
--- a/test/Other/fpenv-constant-fold.ll
+++ b/test/Other/fpenv-constant-fold.ll
@@ -2,9 +2,8 @@
; RUN: opt -S -instcombine -o - < %s | FileCheck %s
; Target independent constant folder should not fold floating-point operations
-; that have constant ; expression as their operands when FEnv access is
-; requested and such folding leads to differences in observable effects at
-; runtime.
+; that have constant expression as their operands when FEnv access is requested
+; and such folding leads to differences in observable effects at runtime.
define double @do-not-fold-fadd-that-can-trap() {
; CHECK-LABEL: @do-not-fold-fadd-that-can-trap
@@ -50,7 +49,7 @@ define double @do-not-fold-fadd-because-of-rounding() {
; CHECK-LABEL: @do-not-fold-fadd-because-of-rounding
; CHECK: fadd
entry:
- %val = fadd kround double 1.0, 1.0
+ %val = fadd kround double 0.101, 0.93
ret double %val
}
@@ -58,7 +57,7 @@ define double @do-not-fold-fsub-because-of-rounding() {
; CHECK-LABEL: @do-not-fold-fsub-because-of-rounding
; CHECK: fsub
entry:
- %val = fsub kround double 3.0, 2.0
+ %val = fsub kround double 0.101, 0.93
ret double %val
}
@@ -66,7 +65,7 @@ define double @do-not-fold-fmul-because-of-rounding() {
; CHECK-LABEL: @do-not-fold-fmul-because-of-rounding
; CHECK: fmul
entry:
- %val = fmul kround double 2.0, 3.0
+ %val = fmul kround double 0.101, 0.93
ret double %val
}
@@ -74,55 +73,50 @@ define double @do-not-fold-fdiv-because-of-rounding() {
; CHECK-LABEL: @do-not-fold-fdiv-because-of-rounding
; CHECK: fdiv
entry:
- %val = fdiv kround double 6.0, 3.0
+ %val = fdiv kround double 0.101, 0.93
ret double %val
}
-define double @do-not-fold-frem-because-of-rounding() {
-; CHECK-LABEL: @do-not-fold-frem-because-of-rounding
-; CHECK: frem
-entry:
- %val = frem kround double 7.0, 3.0
- ret double %val
-}
+; "frem" instruction falls into the same category, but none of tried test-cases
+; could raised inexact exception.
-define double @do-not-fold-fadd-that-cant-trap() {
-; CHECK-LABEL: @do-not-fold-fadd-that-cant-trap
+define double @fold-fadd-that-cant-trap() {
+; CHECK-LABEL: @fold-fadd-that-cant-trap
; CHECK-NOT: fadd
entry:
- %val = fadd kexc double 1.0, 1.0
+ %val = fadd kexc kround double 1.0, 1.0
ret double %val
}
-define double @do-not-fold-fsub-that-cant-trap() {
-; CHECK-LABEL: @do-not-fold-fsub-that-cant-trap
+define double @fold-fsub-that-cant-trap() {
+; CHECK-LABEL: @fold-fsub-that-cant-trap
; CHECK-NOT: fsub
entry:
- %val = fsub kexc double 1.0, 1.0
+ %val = fsub kexc kround double 1.0, 1.0
ret double %val
}
-define double @do-not-fold-fmul-that-cant-trap() {
-; CHECK-LABEL: @do-not-fold-fmul-that-cant-trap
+define double @fold-fmul-that-cant-trap() {
+; CHECK-LABEL: @fold-fmul-that-cant-trap
; CHECK-NOT: fmul
entry:
- %val = fmul kexc double 1.0, 1.0
+ %val = fmul kexc kround double 1.0, 1.0
ret double %val
}
-define double @do-not-fold-fdiv-that-cant-trap() {
-; CHECK-LABEL: @do-not-fold-fdiv-that-cant-trap
+define double @fold-fdiv-that-cant-trap() {
+; CHECK-LABEL: @fold-fdiv-that-cant-trap
; CHECK-NOT: fdiv
entry:
- %val = fdiv kexc double 1.0, 1.0
+ %val = fdiv kexc kround double 1.0, 1.0
ret double %val
}
-define double @do-not-fold-frem-that-cant-trap() {
-; CHECK-LABEL: @do-not-fold-frem-that-cant-trap
+define double @fold-frem-that-cant-trap() {
+; CHECK-LABEL: @fold-frem-that-cant-trap
; CHECK-NOT: frem
entry:
- %val = frem kexc double 1.0, 1.0
+ %val = frem kexc kround double 1.0, 1.0
ret double %val
}
More information about the llvm-commits
mailing list