[llvm-commits] [PATCH][FastMath, InstCombine] Fadd/Fsub optimizations
Shuxin Yang
shuxin.llvm at gmail.com
Fri Dec 14 10:45:42 PST 2012
Hi, Eli:
I'm sorry. I have to took back what I said. I try to make you happy
by returning
back to pattern-match approach:-). Unfortunately it is so messy, I have
to give
up again.
It seems that you have certain degree of misunderstanding of what I'm
doing and
which code can be "reused" by my work. The functions that you mentioned
can be
reused are following:
1. InstCombiner::SimplifyAssociateOrCommutative()
It is already automatically reused two weeks ago (due to my change to
Instruction::isAssociative())
2. InstCombiner::SimplifyUsingDistributiveLaws()
I don't need distrubution at all. At very least at this time.
3. dyn_castFoldableMul
This is your misunderstanding because of my typo in the mail (1.5
vs 1*5).
It seems that you left out two functions that can really be
"re-used": the
InstCombine::{visitAdd, visitSub}. They are pattern-match based, can be
"re-used" by cloning-and-modify! We can quntify the complexity by
examing the LOCs
they have: the visitAdd and visitSub and 262 and 156 LOCs respectively,
not including
helper routines...
I could be wrong. But if you can show me your implementation of
(x+y)+/-(x-y)
within 10 lines code by reusing existing code. I would certainly like to
change
my mind a bit.
I'd like to resurrect my previous change. See the attachment. It was
revised per
your requests. In addition to that, I change the Coeffficient class a
bit such
that the cost of default constructor is merely a 4-byte-store-zero.
This cost
renders it possible to "abuse" this data structure.
As to the determinstic sorting, I have not yet figure out a cost
effective way
for that matter. Note that stable_sort() won't help as all Addends sharing
the same symbolic value will be folded, we don't care their relative
posittion
before folding.
Tons of thanks in advance for the code review. (testing-case is not
included in this
patch for clarity purpose)
Shuxin
~
On 12/12/12 6:56 PM, Shuxin Yang wrote:
> I has typo in my mail. I meant 1.5 not 1*5.
> dyn_castFoldableMul() is not applicable.
>
> Forget my change, I return back to patten match approach.
>
> On 12/12/2012 06:41 PM, Eli Friedman wrote:
>> On Wed, Dec 12, 2012 at 4:58 PM, Shuxin Yang <shuxin.llvm at gmail.com>
>> wrote:
>>> I just re-read the InstCombiner::SimplifyAssociateOrCommutative().
>>> It is pretty simple, it can only handle adjacent instructions with
>>> exactly
>>> the same opcode.
>>> (1*5 * x) - x is certainly beyond its scope.
>> instcombine uses a different codepath to handle (5*x) - x in
>> particular; see dyn_castFoldableMul in InstCombineAddSub.cpp . And
>> InstCombiner::SimplifyUsingDistributiveLaws can handle 5*x+5*x. (Not
>> sure what the 1 is doing in your example.)
>>
>>> Do we need to expand it to be more capable? I don't like this idea
>>> because
>> We are capable of performing all the transformations from your
>> original email on integer +/-/*.
>>
>> -Eli
>
-------------- next part --------------
Index: Transforms/InstCombine/InstCombineAddSub.cpp
===================================================================
--- Transforms/InstCombine/InstCombineAddSub.cpp (revision 170178)
+++ Transforms/InstCombine/InstCombineAddSub.cpp (working copy)
@@ -19,10 +19,718 @@
using namespace llvm;
using namespace PatternMatch;
+namespace {
+
+ /// Class representing coefficient of floating-point addend.
+ /// This class needs to be highly efficient, which is especially true for
+ /// the constructor. As of I write this comment, the cost of the default
+ /// constructor is merely 4-byte-store-zero (Assuming compiler is able to
+ /// perform write-merging).
+ ///
+ class FAddendCoef {
+ public:
+ // The constructor has to initialize a APFloat, which is uncessary for
+ // most addends which have coefficient either 1 or -1. So, the constructor
+ // is expensive. In order to avoid the cost of the constructor, we should
+ // reuse some instances whenever possible. The pre-created instances
+ // FAddCombine::Add[0-5] embodies this idea.
+ //
+ FAddendCoef() : IsFp(false), BufHasFpVal(false), IntVal(0) {}
+ ~FAddendCoef();
+
+ void set(short C) {
+ assert(!insaneIntVal(C) && "Insane coefficient");
+ IsFp = false; IntVal = C;
+ }
+
+ void set(const APFloat& C);
+
+ void negate();
+
+ bool isZero() const { return isInt() ? !IntVal : getFpVal().isZero(); }
+ Value *getValue(Type *) const;
+
+ // If possible, don't define operator+/operator- etc because these
+ // operators inevitably call FAddendCoef's constructor which is not cheap.
+ void operator=(const FAddendCoef &A);
+ void operator+=(const FAddendCoef &A);
+ void operator-=(const FAddendCoef &A);
+ void operator*=(const FAddendCoef &S);
+
+ bool isOne() const { return isInt() && IntVal == 1; }
+ bool isTwo() const { return isInt() && IntVal == 2; }
+ bool isMinusOne() const { return isInt() && IntVal == -1; }
+ bool isMinusTwo() const { return isInt() && IntVal == -2; }
+
+ private:
+ bool insaneIntVal(int V) { return V > 4 || V < -4; }
+ APFloat *getFpValPtr(void)
+ { return reinterpret_cast<APFloat*>(&FpValBuf[0]); }
+
+ const APFloat &getFpVal(void) const {
+ assert(IsFp && BufHasFpVal && "Incorret state");
+ return *reinterpret_cast<const APFloat*>(&FpValBuf[0]);
+ }
+
+ APFloat &getFpVal(void)
+ { assert(IsFp && BufHasFpVal && "Incorret state"); return *getFpValPtr(); }
+
+ bool isInt() const { return !IsFp; }
+
+ private:
+ bool IsFp;
+
+ // True iff FpValBuf contains a instance of APFloat.
+ bool BufHasFpVal;
+
+ // The integer coefficient of an individual addend is either 1 or -1,
+ // and we try to simplify at most 4 addends from neighboring at most
+ // two instructions. So the range if <IntVal> falls in [-4, 4]. APInt
+ // is overkill of this end.
+ short IntVal;
+
+ union {
+ char FpValBuf[sizeof(APFloat)];
+ int dummy; // So this structure has at least 4-byte alignment.
+ };
+ };
+
+ /// FAddend is used to represent floating-point addend. An addend is
+ /// represented as <C, V>, where the V is is symbolic value, and C is a
+ /// constant coefficient. A constant addend is represented as <C, 0>.
+ ///
+ class FAddend {
+ public:
+ typedef enum {
+ Simpler, // addend1 = c1*x, addend2 = c2*x, result = (c1+c2)*x
+ FlushToZero, // similar to the case of Simpler, except that (c1+c2) is a
+ // denormal, and the result is flushed to zero.
+ Zero, // addend1 = c1*x, addend2 = -c1*x , result = 0
+ Fail // addend1 = c1*x, addend2 = c2*y, and x != y
+ } SimpResult;
+
+ FAddend() { Val = 0; }
+
+ Value *getSymVal (void) const { return Val; }
+ const FAddendCoef& getCoef(void) const { return Coeff; }
+
+ bool isConstant() const { return Val == 0; }
+
+ void set(short Coefficient, Value *V) { Coeff.set(Coefficient), Val = V; }
+ void set(const APFloat& Coefficient, Value *V)
+ { Coeff.set(Coefficient); Val = V; }
+ void set(const ConstantFP* Coefficient, Value *V)
+ { Coeff.set(Coefficient->getValueAPF()); Val = V; }
+
+ void negate() { Coeff.negate(); }
+
+ /// Try to simplify "\p this + \p Addend2". Iff simplification was
+ /// successful, the resulting value will be saved to "this" instance.
+ SimpResult trySimplifyAdd(const FAddend& Addend2, bool FlushToZero=false);
+
+ /// Drill down the U-D chain one step to find the definition of V, and
+ /// try to break the definition into one or two addends.
+ static unsigned drillDownOneStep(Value* V, FAddend &A0, FAddend &A1);
+
+ /// Similar to FAddend::drillDownOneStep() except that the value being
+ /// splitted is the addend itself.
+ unsigned drillDownOneStep(FAddend &Addend0, FAddend &Addend1) const;
+
+ private:
+ void Scale(const FAddendCoef& ScaleAmt) { Coeff *= ScaleAmt; }
+
+ // This addend has the value of "Coeff * Val".
+ FAddendCoef Coeff;
+ Value *Val;
+ };
+
+ /// This functor works with std::sort to permute addends such that those
+ /// having same symbolic-value are clustered together.
+ struct FAddendCmp {
+ bool operator()(const FAddend *A1, const FAddend *A2) {
+ return A1->getSymVal() < A2->getSymVal();
+ }
+ };
+
+ /// FAddCombine is the class for optimizing an unsafe fadd/fsub along
+ /// with its neighboring at most two instructions.
+ ///
+ class FAddCombine {
+ public:
+ FAddCombine(InstCombiner::BuilderTy *B) : Builder(B), Instr(0) {}
+ Value *simplify(Instruction *FAdd);
+
+ private:
+ typedef SmallVector<const FAddend*, 4> AddendVect;
+
+ Value *simplifyFAdd(AddendVect& V, unsigned InstrQuota);
+
+ /// Convert given addend to a Value
+ Value *createAddendVal(const FAddend &A, bool& NeedNeg);
+
+ /// Return the number of instruction needed to emit the N-ary addition.
+ unsigned calcInstrNumber(const AddendVect& Vect);
+ Value *createFSub(Value *Opnd0, Value *Opnd1);
+ Value *createFAdd(Value *Opnd0, Value *Opnd1);
+ Value *createFMul(Value *Opnd0, Value *Opnd1);
+ Value *createFNeg(Value *V);
+ Value *createNaryFAdd(const AddendVect& Opnds, unsigned InstrQuota);
+ void createInstPostProc(Instruction *NewInst);
+
+ InstCombiner::BuilderTy *Builder;
+ Instruction *Instr;
+
+ private:
+ // Debugging stuff are clustered here.
+ #ifndef NDEBUG
+ unsigned CreateInstrNum;
+ void initCreateInstNum() { CreateInstrNum = 0; }
+ void incCreateInstNum() { CreateInstrNum++; }
+ #else
+ void initCreateInstNum() {}
+ void incCreateInstNum() {}
+ #endif
+ };
+}
+
+//===----------------------------------------------------------------------===//
+//
+// Implementation of
+// {FAddendCoef, FAddend, FAddition, FAddCombine}.
+//
+//===----------------------------------------------------------------------===//
+FAddendCoef::~FAddendCoef() {
+ if (BufHasFpVal)
+ getFpValPtr()->~APFloat();
+}
+
+void FAddendCoef::set(const APFloat& C) {
+ APFloat *P = getFpValPtr();
+
+ if (isInt()) {
+ // As the buffer is meanless byte stream, we cannot call
+ // APFloat::operator=().
+ new(P) APFloat(C);
+ } else
+ *P = C;
+
+ IsFp = BufHasFpVal = true;
+}
+
+void FAddendCoef::operator=(const FAddendCoef& That) {
+ if (That.isInt())
+ set(That.IntVal);
+ else
+ set(That.getFpVal());
+}
+
+void FAddendCoef::operator+=(const FAddendCoef &That) {
+ enum APFloat::roundingMode RndMode = APFloat::rmNearestTiesToEven;
+ if (isInt() == That.isInt()) {
+ if (isInt())
+ IntVal += That.IntVal;
+ else
+ getFpVal().add(That.getFpVal(), RndMode);
+ return;
+ }
+
+ if (isInt()) {
+ const APFloat &T = That.getFpVal();
+ set(T);
+ getFpVal().add(APFloat(T.getSemantics(), IntVal), RndMode);
+ return;
+ }
+
+ APFloat &T = getFpVal();
+ T.add(APFloat(T.getSemantics(), That.IntVal), RndMode);
+}
+
+void FAddendCoef::operator-=(const FAddendCoef &That) {
+ enum APFloat::roundingMode RndMode = APFloat::rmNearestTiesToEven;
+ if (isInt() == That.isInt()) {
+ if (isInt())
+ IntVal -= That.IntVal;
+ else
+ getFpVal().subtract(That.getFpVal(), RndMode);
+ return;
+ }
+
+ if (isInt()) {
+ const APFloat &T = That.getFpVal();
+ set(T);
+ getFpVal().subtract(APFloat(T.getSemantics(), IntVal), RndMode);
+ return;
+ }
+
+ APFloat &T = getFpVal();
+ T.subtract(APFloat(T.getSemantics(), IntVal), RndMode);
+}
+
+void FAddendCoef::operator*=(const FAddendCoef &That) {
+ if (That.isOne())
+ return;
+
+ if (That.isMinusOne()) {
+ negate();
+ return;
+ }
+
+ if (isInt() && That.isInt()) {
+ int Res = IntVal * (int)That.IntVal;
+ assert(!insaneIntVal(Res) && "Insane int value");
+ IntVal = Res;
+ return;
+ }
+
+ const fltSemantics &Semantic =
+ isInt() ? That.getFpVal().getSemantics() : getFpVal().getSemantics();
+
+ APFloat &F0 = getFpVal();
+ if (isInt())
+ F0 = APFloat(Semantic, IntVal);
+
+ if (That.isInt())
+ F0.multiply(APFloat(Semantic, That.IntVal), APFloat::rmNearestTiesToEven);
+ else
+ F0.multiply(That.getFpVal(), APFloat::rmNearestTiesToEven);
+
+ return;
+}
+
+void FAddendCoef::negate() {
+ if (isInt())
+ IntVal = 0 - IntVal;
+ else
+ getFpVal().changeSign();
+}
+
+Value *FAddendCoef::getValue(Type *Ty) const {
+ return isInt() ?
+ ConstantFP::get(Ty, float(IntVal)) :
+ ConstantFP::get(Ty->getContext(), getFpVal());
+}
+
+FAddend::SimpResult
+FAddend::trySimplifyAdd
+ (const FAddend& Addend2, bool FlushToZero) {
+ // Currently flush-to-0 is ignored. Following stmtement is to suppress
+ // compile-warning.
+ FlushToZero = !FlushToZero;
+
+ if (Val != Addend2.Val)
+ return Fail;
+
+ Coeff += Addend2.Coeff;
+
+ return Coeff.isZero() ? Zero : Simpler;
+}
+
+// The definition of <Val> Addends
+// =========================================
+// A + B <1, A>, <1,B>
+// A - B <1, A>, <1,B>
+// 0 - B <-1, B>
+// C * A, <C, A>
+// A + C <1, A> <C, NULL>
+// 0 +/- 0 <0, NULL> (corner case)
+//
+// Legend: A, B are not constant, C is constant
+//
+unsigned FAddend::drillDownOneStep
+ (Value *Val, FAddend &Addend0, FAddend &Addend1) {
+ Instruction *I = 0;
+ if (Val == 0 || !(I = dyn_cast<Instruction>(Val)))
+ return 0;
+
+ unsigned Opcode = I->getOpcode();
+
+ if (Opcode == Instruction::FAdd || Opcode == Instruction::FSub) {
+ ConstantFP *C0, *C1;
+ Value *Opnd0 = I->getOperand(0);
+ Value *Opnd1 = I->getOperand(1);
+ if ((C0 = dyn_cast<ConstantFP>(Opnd0)) && C0->isZero())
+ Opnd0 = 0;
+
+ if ((C1 = dyn_cast<ConstantFP>(Opnd1)) && C1->isZero())
+ Opnd1 = 0;
+
+ if (Opnd0) {
+ if (!C0)
+ Addend0.set(1, Opnd0);
+ else
+ Addend0.set(C0, 0);
+ }
+
+ if (Opnd1) {
+ FAddend &Addend = Opnd0 ? Addend1 : Addend0;
+ if (!C1)
+ Addend.set(1, Opnd1);
+ else
+ Addend.set(C1, 0);
+ if (Opcode == Instruction::FSub)
+ Addend.negate();
+ }
+
+ if (Opnd0 || Opnd1)
+ return Opnd0 && Opnd1 ? 2 : 1;
+
+ // Both operands are zero. Weird!
+ Addend0.set(APFloat(0.0f), 0);
+ return 1;
+ }
+
+ if (I->getOpcode() == Instruction::FMul) {
+ Value *V0 = I->getOperand(0);
+ Value *V1 = I->getOperand(1);
+ if (ConstantFP *C = dyn_cast<ConstantFP>(V0)) {
+ Addend0.set(C, V1);
+ return 1;
+ }
+
+ if (ConstantFP *C = dyn_cast<ConstantFP>(V1)) {
+ Addend0.set(C, V0);
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+// Try to break *this* addend into two addends. e.g. Suppose This addend is
+// <2.3, V>, and V = X + Y, by calling this function, we obtain two addends,
+// i.e. <2.3, X> and <2.3, Y>.
+//
+unsigned FAddend::drillDownOneStep
+ (FAddend &Addend0, FAddend &Addend1) const {
+ if (isConstant())
+ return 0;
+
+ unsigned BreakNum = FAddend::drillDownOneStep(Val, Addend0, Addend1);
+ if (!BreakNum || Coeff.isOne())
+ return BreakNum;
+
+ Addend0.Scale(Coeff);
+
+ if (BreakNum == 2)
+ Addend1.Scale(Coeff);
+
+ return BreakNum;
+}
+
+Value *FAddCombine::simplify(Instruction *I) {
+
+ assert(I->hasUnsafeAlgebra() && "Should be in unsafe mode");
+
+ // Currently we are able to handle vector type.
+ if (I->getType()->isVectorTy())
+ return 0;
+
+ assert((I->getOpcode() == Instruction::FAdd ||
+ I->getOpcode() == Instruction::FSub) && "Expect add/sub");
+
+ // Save the instruction before calling other member-functions.
+ Instr = I;
+
+ FAddend Opnd0, Opnd1, Opnd0_0, Opnd0_1, Opnd1_0, Opnd1_1;
+
+ unsigned OpndNum = FAddend::drillDownOneStep(I, Opnd0, Opnd1);
+
+ // Step 1: Expand the 1st addend into Opnd0_0 and Opnd0_1
+ unsigned Opnd0_ExpNum = 0;
+ unsigned Opnd1_ExpNum = 0;
+
+ if (!Opnd0.isConstant())
+ Opnd0_ExpNum = Opnd0.drillDownOneStep(Opnd0_0, Opnd0_1);
+
+ // Step 2: Expand the 2nd addend into Opnd1_0 and Opnd1_1.
+ if (OpndNum == 2 && !Opnd1.isConstant())
+ Opnd1_ExpNum = Opnd1.drillDownOneStep(Opnd1_0, Opnd1_1);
+
+ // Step 3: try to optimize Opnd0_0 + Opnd0_1 + Opnd1_0 + Opnd1_1
+ if (Opnd0_ExpNum && Opnd1_ExpNum) {
+ AddendVect AllOpnds;
+ AllOpnds.push_back(&Opnd0_0);
+ AllOpnds.push_back(&Opnd1_0);
+ if (Opnd0_ExpNum == 2)
+ AllOpnds.push_back(&Opnd0_1);
+ if (Opnd1_ExpNum == 2)
+ AllOpnds.push_back(&Opnd1_1);
+
+ // Compute instruction quota. We should save at least one instruction.
+ unsigned InstQuota = 0;
+
+ Value *V0 = I->getOperand(0);
+ Value *V1 = I->getOperand(1);
+ InstQuota = ((!isa<Constant>(V0) && V0->hasOneUse()) &&
+ (!isa<Constant>(V1) && V1->hasOneUse())) ? 2 : 1;
+
+ if (Value *R = simplifyFAdd(AllOpnds, InstQuota))
+ return R;
+ }
+
+ if (OpndNum != 2) {
+ // The input instruction is : "I=0.0 +/- V". If the "V" were able to be
+ // splitted into two addends, say "V = X - Y", the instruction would have
+ // been optimized into "I = Y - X" in the previous steps.
+ //
+ const FAddendCoef& CE = Opnd0.getCoef();
+ return CE.isOne() ? Opnd0.getSymVal() : 0;
+ }
+
+ // step 4: Try to optimize Opnd0 + Opnd1_0 [+ Opnd1_1]
+ if (Opnd1_ExpNum) {
+ AddendVect AllOpnds;
+ AllOpnds.push_back(&Opnd0);
+ AllOpnds.push_back(&Opnd1_0);
+ if (Opnd1_ExpNum == 2)
+ AllOpnds.push_back(&Opnd1_1);
+
+ if (Value *R = simplifyFAdd(AllOpnds, 1))
+ return R;
+ }
+
+ // step 4: Try to optimize Opnd1 + Opnd0_0 [+ Opnd0_1]
+ if (Opnd0_ExpNum) {
+ AddendVect AllOpnds;
+ AllOpnds.push_back(&Opnd1);
+ AllOpnds.push_back(&Opnd0_0);
+ if (Opnd0_ExpNum == 2)
+ AllOpnds.push_back(&Opnd0_1);
+
+ if (Value *R = simplifyFAdd(AllOpnds, 1))
+ return R;
+ }
+
+ return 0;
+}
+
+
+Value *FAddCombine::simplifyFAdd
+ (AddendVect& Addends, unsigned InstrQuota) {
+
+ assert(Addends.size() <= 4 && "At most 4 addends are involved");
+
+ // For saving intermediate results;
+ unsigned NextTmpIdx = 0;
+ FAddend TmpResult[3];
+
+ // Permute the input addends such that addends sharing same symbolic-value
+ // are clustered together. e.g. { c1*x, c2*y, c3*x, c4*y, ... } =>
+ /// { c1*x, c3*x, c2*y, c4*y, ...}.
+ std::sort(Addends.begin(), Addends.end(), FAddendCmp());
+
+ // Walk forward along the sorted addends, trying to combine adjacent two
+ // addends into a single one.
+ //
+ // Suppose the input addends are <a, X>, <b, X>, ..., <m, Y>, <n, Y>, ... .
+ // The simplifed addends would be <a + b + ..., X>, <m + n + ..., Y>, ....
+ //
+ AddendVect SimpVect;
+ for (AddendVect::iterator I = Addends.begin(), E = Addends.end();
+ I != E; I++) {
+ const FAddend* Opnd = *I;
+ if (SimpVect.empty()) {
+ SimpVect.push_back(Opnd);
+ continue;
+ }
+
+ // Try to combine current addend with the previous adjacent addent
+ const FAddend *Opnd0 = SimpVect.back();
+ if (Opnd0->getSymVal() != Opnd->getSymVal()) {
+ // case 1: Opnd0 + Opnd can not be simplified.
+ SimpVect.push_back(Opnd);
+ continue;
+ }
+
+ SimpVect.pop_back();
+ FAddend *T = &TmpResult[NextTmpIdx++];
+ *T = *Opnd0;
+
+ FAddend::SimpResult R = T->trySimplifyAdd(*Opnd);
+
+ // case 2: Opnd0 + Opnd = 0
+ if (R == FAddend::Zero || R == FAddend::FlushToZero)
+ continue;
+
+ // case 3: Opnd0 + Opnd = C * X
+ assert (R == FAddend::Simpler);
+ SimpVect.push_back(T);
+ }
+
+ Value *Result;
+ if (!SimpVect.empty())
+ Result = createNaryFAdd(SimpVect, InstrQuota);
+ else {
+ // The addition is folded to 0.0
+ Result = ConstantFP::get(Instr->getType(), 0.0);
+ }
+
+ return Result;
+}
+
+Value *FAddCombine::createNaryFAdd
+ (const AddendVect& Opnds, unsigned InstrQuota) {
+ assert(!Opnds.empty() && "Exect at least one addend");
+
+ // Step 1: Check if the # of instruction needed exceeds the quota.
+ //
+ unsigned InstrNeeded = calcInstrNumber(Opnds);
+ if (InstrNeeded > InstrQuota)
+ return 0;
+
+ initCreateInstNum();
+
+ // step 2: Emit the N-ary addition.
+ // Note that at most threee instructions involved in Fadd-InstCombine: the
+ // addition in question, and at most two neighboring instructions.
+ // The resulting optimized addition should have at least one less instruction
+ // than the original addition expression tree. This implies the resulting
+ // N-ary addition has at most two instructions, and we don't need to worry
+ // about tree-height when constructing the N-ary addition.
+
+ Value *LastVal = 0;
+ bool LastValNeedNeg = false;
+
+ // Iterate the addends, creating fadd/fsub using adjacent two addends.
+ for (AddendVect::const_iterator I = Opnds.begin(), E = Opnds.end();
+ I != E; I++) {
+ bool NeedNeg;
+ Value *V = createAddendVal(**I, NeedNeg);
+ if (!LastVal) {
+ LastVal = V;
+ LastValNeedNeg = NeedNeg;
+ continue;
+ }
+
+ if (LastValNeedNeg == NeedNeg) {
+ LastVal = createFAdd(LastVal, V);
+ continue;
+ }
+
+ if (LastValNeedNeg)
+ LastVal = createFSub(V, LastVal);
+ else
+ LastVal = createFSub(LastVal, V);
+
+ LastValNeedNeg = false;
+ }
+
+ if (LastValNeedNeg) {
+ LastVal = createFNeg(LastVal);
+ }
+
+ #ifndef NDEBUG
+ assert(CreateInstrNum == InstrNeeded &&
+ "Inconsistent in instruction numbers");
+ #endif
+
+ return LastVal;
+}
+
+Value *FAddCombine::createFSub
+ (Value *Opnd0, Value *Opnd1) {
+ Value *V = Builder->CreateFSub(Opnd0, Opnd1);
+ createInstPostProc(cast<Instruction>(V));
+ return V;
+}
+
+Value *FAddCombine::createFNeg(Value *V) {
+ Value *Zero = cast<Value>(ConstantFP::get(V->getType(), 0.0));
+ return createFSub(Zero, V);
+}
+
+Value *FAddCombine::createFAdd
+ (Value *Opnd0, Value *Opnd1) {
+ Value *V = Builder->CreateFAdd(Opnd0, Opnd1);
+ createInstPostProc(cast<Instruction>(V));
+ return V;
+}
+
+Value *FAddCombine::createFMul(Value *Opnd0, Value *Opnd1) {
+ Value *V = Builder->CreateFMul(Opnd0, Opnd1);
+ createInstPostProc(cast<Instruction>(V));
+ return V;
+}
+
+void FAddCombine::createInstPostProc(Instruction *NewInstr) {
+ NewInstr->setDebugLoc(Instr->getDebugLoc());
+
+ // keep track of the number of instruction created.
+ incCreateInstNum();
+
+ // Propagate fast-math flags
+ NewInstr->setFastMathFlags(Instr->getFastMathFlags());
+}
+
+// Return the number of instruction needed to emit the N-ary addition.
+// NOTE: Keep this function in sync with createAddendVal().
+unsigned FAddCombine::calcInstrNumber
+ (const AddendVect &Opnds) {
+ unsigned OpndNum = Opnds.size();
+ unsigned InstrNeeded = OpndNum - 1;
+
+ // The number of addends in the form the "(-1)*x".
+ unsigned NegOpndNum = 0;
+
+ // Adjust the the number of instruction needed to emit the N-ary add.
+ for (AddendVect::const_iterator I = Opnds.begin(), E = Opnds.end();
+ I != E; I++) {
+ const FAddend *Opnd = *I;
+ if (Opnd->isConstant())
+ continue;
+
+ const FAddendCoef& CE = Opnd->getCoef();
+ if (CE.isMinusOne() || CE.isMinusTwo())
+ NegOpndNum++;
+
+ // Let the addend be "c * x". If "c == +/-1", the value of the addend
+ // is immediately aviable; otherwise, it needs exactly one instruction
+ // to evaluate the value.
+ if (!CE.isMinusOne() && !CE.isOne())
+ InstrNeeded++;
+ }
+ if (NegOpndNum == OpndNum)
+ InstrNeeded++;
+ return InstrNeeded;
+}
+
+// Input Addend Value NeedNeg(output)
+// ================================================================
+// Constant C C false
+// <+/-1, V> V coefficient is -1
+// <2/-2, V> "fadd V, V" coefficient is -2
+// <C, V> "fmul V, C" false
+//
+Value *FAddCombine::createAddendVal
+ (const FAddend &Opnd, bool& NeedNeg) {
+ const FAddendCoef& Coeff = Opnd.getCoef();
+
+ if (Opnd.isConstant()) {
+ NeedNeg = false;
+ return Coeff.getValue(Instr->getType());
+ }
+
+ Value *OpndVal = Opnd.getSymVal();
+
+ if (Coeff.isMinusOne() || Coeff.isOne()) {
+ NeedNeg = Coeff.isMinusOne();
+ return OpndVal;
+ }
+
+ if (Coeff.isTwo() || Coeff.isMinusTwo()) {
+ NeedNeg = Coeff.isMinusTwo();
+ return createFAdd(OpndVal, OpndVal);
+ }
+
+ NeedNeg = false;
+ return createFMul(OpndVal, Coeff.getValue(Instr->getType()));
+}
+
/// AddOne - Add one to a ConstantInt.
static Constant *AddOne(Constant *C) {
return ConstantExpr::getAdd(C, ConstantInt::get(C->getType(), 1));
}
+
/// SubOne - Subtract one from a ConstantInt.
static Constant *SubOne(ConstantInt *C) {
return ConstantInt::get(C->getContext(), C->getValue()-1);
@@ -402,6 +1110,12 @@
}
}
+ I.setHasUnsafeAlgebra(true);
+ if (I.hasUnsafeAlgebra()) {
+ if (Value *V = FAddCombine(Builder).simplify(&I))
+ return ReplaceInstUsesWith(I, V);
+ }
+
return Changed ? &I : 0;
}
@@ -645,5 +1359,11 @@
if (Value *V = dyn_castFNegVal(Op1))
return BinaryOperator::CreateFAdd(Op0, V);
+ I.setHasUnsafeAlgebra(true);
+ if (I.hasUnsafeAlgebra()) {
+ if (Value *V = FAddCombine(Builder).simplify(&I))
+ return ReplaceInstUsesWith(I, V);
+ }
+
return 0;
}
More information about the llvm-commits
mailing list