<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hal,<div><br></div><div>FMA's should be formed aggressively under "fast math" mode (as long as they're fast on the target), though that more or less just means forcing FP_CONTRACT on in the frontend.</div><div><br></div><div>The semantics of TM.Options.NoExcessFPPrecision are less clear. It was originally intended to allow the implicit use of 80-bit floating point. This <i>does</i> change the semantics of floating point operations, but does so in a relatively uniform manner. It doesn't introduce the kind of expression asymmetries that one can run into with FMA. My interpretation that a true TM.Options.NoExcessFPPrecision should definitely <i>dis</i>allow FMA formation, but a false value does not mean we should always form them either. </div><div><br></div><div>I really do encourage everyone to look at Lang's work on @llvm.fmuladd() to more details of how all the pieces fit together.</div><div><br></div><div>--Owen</div><div><br></div><div><br><div><div>On Jun 1, 2012, at 11:39 AM, Hal Finkel <<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Craig,<br><br>Maybe I missed part of this, but should these things also be enabled<br>under the appropriate "fast math" mode. We also have<br>TM.Options.NoExcessFPPrecision, etc. how do those fit in here?<br><br>I'd think that we should have patterns predicated on these options<br>because these can apply to instructions that come from combinations of<br>C/C++ language statements (or other frontends).<br><br>Thanks in advance,<br>Hal<br><br>On Thu, 31 May 2012 23:06:00 -0700<br>Craig Topper <<a href="mailto:craig.topper@gmail.com">craig.topper@gmail.com</a>> wrote:<br><br><blockquote type="cite">I'm going to remove the fmul(fadd) patterns and enable automatic<br>feature detection. We need to wait till FP_CONTRACT is implemented<br>before we can automatically generate FMAs.<br><br>On Thu, May 31, 2012 at 10:13 PM, Craig Topper<br><<a href="mailto:craig.topper@gmail.com">craig.topper@gmail.com</a>>wrote:<br><br><blockquote type="cite">The tablegen part of this patch is full of 80 column violations.<br>There also a lot of dangling spaces. I'll go ahead and clean these<br>up.<br><br><br>On Thu, May 31, 2012 at 9:46 AM, Owen Anderson <<a href="mailto:resistor@mac.com">resistor@mac.com</a>><br>wrote:<br><br><blockquote type="cite">I agree with Craig. These should match fma @llvm.fma. The trick<br>is that @llvm.fma should not (necessarily) be generated in every<br>where the user wrote fmul+fadd. See Lang's thread yesterday for a<br>proposal for implement this correctly, respecting FP_CONTRACT.<br><br>--Owen<br><br>On May 31, 2012, at 7:19 AM, Craig Topper <<a href="mailto:craig.topper@gmail.com">craig.topper@gmail.com</a>><br>wrote:<br><br>Wouldn't it have been better to enable the intrinsics, but not<br>allow converting fmul(fadd) to FMA? That way people can still use<br>FMA if they explicitly request it through intrinsics?<br><br>On Thu, May 31, 2012 at 2:20 AM, Elena Demikhovsky <<br><a href="mailto:elena.demikhovsky@intel.com">elena.demikhovsky@intel.com</a>> wrote:<br><br><blockquote type="cite">Author: delena<br>Date: Thu May 31 04:20:20 2012<br>New Revision: 157737<br><br>URL: <a href="http://llvm.org/viewvc/llvm-project?rev=157737&view=rev">http://llvm.org/viewvc/llvm-project?rev=157737&view=rev</a><br>Log:<br>Added FMA3 Intel instructions.<br>I disabled FMA3 autodetection, since the result may differ from<br>expected for some benchmarks.<br>I added tests for GodeGen and intrinsics.<br>I did not change llvm.fma.f32/64 - it may be done later.<br><br><br>Added:<br> llvm/trunk/test/CodeGen/X86/fma3-intrinsics.ll (with props)<br> llvm/trunk/test/CodeGen/X86/fma3.ll (with props)<br>Modified:<br> llvm/trunk/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp<br> llvm/trunk/lib/Target/X86/X86CodeEmitter.cpp<br> llvm/trunk/lib/Target/X86/X86InstrFMA.td<br> llvm/trunk/lib/Target/X86/X86InstrInfo.cpp<br> llvm/trunk/lib/Target/X86/X86InstrInfo.h<br> llvm/trunk/lib/Target/X86/X86Subtarget.cpp<br><br>Modified:<br>llvm/trunk/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp URL:<br><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp?rev=157737&r1=157736&r2=157737&view=diff">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp?rev=157737&r1=157736&r2=157737&view=diff</a><br><br>==============================================================================<br>--- llvm/trunk/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp<br>(original)<br>+++ llvm/trunk/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp<br>Thu May 31 04:20:20 2012<br>@@ -570,7 +570,11 @@<br> }<br><br> // Classify VEX_B, VEX_4V, VEX_R, VEX_X<br>+ unsigned NumOps = Desc.getNumOperands();<br> unsigned CurOp = 0;<br>+ if (NumOps > 1 && Desc.getOperandConstraint(1,<br>MCOI::TIED_TO) != -1)<br>+ ++CurOp;<br>+<br> switch (TSFlags & X86II::FormMask) {<br> case X86II::MRMInitReg: llvm_unreachable("FIXME: Remove this!");<br> case X86II::MRMDestMem: {<br>@@ -603,11 +607,11 @@<br> // FMA4:<br> // dst(ModR/M.reg), src1(VEX_4V), src2(ModR/M),<br>src3(VEX_I8IMM) // dst(ModR/M.reg), src1(VEX_4V),<br>src2(VEX_I8IMM), src3(ModR/M),<br>- if (X86II::isX86_64ExtendedReg(MI.getOperand(0).getReg()))<br>+ if<br>(X86II::isX86_64ExtendedReg(MI.getOperand(CurOp++).getReg()))<br>VEX_R = 0x0;<br><br> if (HasVEX_4V)<br>- VEX_4V = getVEXRegisterEncoding(MI, 1);<br>+ VEX_4V = getVEXRegisterEncoding(MI, CurOp);<br><br> if (X86II::isX86_64ExtendedReg(<br> MI.getOperand(MemOperand+X86::AddrBaseReg).getReg()))<br><br>Modified: llvm/trunk/lib/Target/X86/X86CodeEmitter.cpp<br>URL:<br>http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86CodeEmitter.cpp?rev=157737&r1=157736&r2=157737&view=diff<br><br>==============================================================================<br>--- llvm/trunk/lib/Target/X86/X86CodeEmitter.cpp (original)<br>+++ llvm/trunk/lib/Target/X86/X86CodeEmitter.cpp Thu May 31<br>04:20:20 2012 @@ -933,7 +933,10 @@<br> }<br><br> // Classify VEX_B, VEX_4V, VEX_R, VEX_X<br>+ unsigned NumOps = Desc->getNumOperands();<br> unsigned CurOp = 0;<br>+ if (NumOps > 1 && Desc->getOperandConstraint(1,<br>MCOI::TIED_TO) != -1)<br>+ ++CurOp;<br> switch (TSFlags & X86II::FormMask) {<br> case X86II::MRMInitReg:<br> // Duplicate register.<br><br>Modified: llvm/trunk/lib/Target/X86/X86InstrFMA.td<br>URL:<br>http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86InstrFMA.td?rev=157737&r1=157736&r2=157737&view=diff<br><br>==============================================================================<br>--- llvm/trunk/lib/Target/X86/X86InstrFMA.td (original)<br>+++ llvm/trunk/lib/Target/X86/X86InstrFMA.td Thu May 31 04:20:20<br>2012 @@ -15,83 +15,358 @@<br> // FMA3 - Intel 3 operand Fused Multiply-Add instructions<br><br> //===----------------------------------------------------------------------===//<br><br>+let Constraints = "$src1 = $dst" in {<br> multiclass fma3p_rm<bits<8> opc, string OpcodeStr> {<br> def r : FMA3<opc, MRMSrcReg, (outs VR128:$dst),<br>- (ins VR128:$src1, VR128:$src2),<br>- !strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst,<br>$src1, $src2}"),<br>+ (ins VR128:$src1, VR128:$src2, VR128:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br> []>;<br>+ let mayLoad = 1 in<br> def m : FMA3<opc, MRMSrcMem, (outs VR128:$dst),<br>- (ins VR128:$src1, f128mem:$src2),<br>- !strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst,<br>$src1, $src2}"),<br>+ (ins VR128:$src1, VR128:$src2, f128mem:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br> []>;<br> def rY : FMA3<opc, MRMSrcReg, (outs VR256:$dst),<br>- (ins VR256:$src1, VR256:$src2),<br>- !strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst,<br>$src1, $src2}"),<br>+ (ins VR256:$src1, VR256:$src2, VR256:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br> []>;<br>+ let mayLoad = 1 in<br> def mY : FMA3<opc, MRMSrcMem, (outs VR256:$dst),<br>- (ins VR256:$src1, f256mem:$src2),<br>- !strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst,<br>$src1, $src2}"),<br>+ (ins VR256:$src1, VR256:$src2, f256mem:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br> []>;<br> }<br><br>+// Intrinsic for 132 pattern<br>+multiclass fma3p_rm_int<bits<8> opc, string OpcodeStr,<br>+ PatFrag MemFrag128, PatFrag MemFrag256,<br>+ Intrinsic Int128, Intrinsic Int256> {<br>+ def r_Int : FMA3<opc, MRMSrcReg, (outs VR128:$dst),<br>+ (ins VR128:$src1, VR128:$src2, VR128:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br>+ [(set VR128:$dst, (Int128 VR128:$src1, VR128:$src3,<br>VR128:$src2))]>;<br>+ //let mayLoad = 1 in<br>+ def m_Int : FMA3<opc, MRMSrcMem, (outs VR128:$dst),<br>+ (ins VR128:$src1, VR128:$src2, f128mem:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br>+ [(set VR128:$dst, (Int128 VR128:$src1, (MemFrag128<br>addr:$src3), VR128:$src2))]>;<br>+ def rY_Int : FMA3<opc, MRMSrcReg, (outs VR256:$dst),<br>+ (ins VR256:$src1, VR256:$src2, VR256:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br>+ [(set VR256:$dst, (Int256 VR256:$src1, VR256:$src3,<br>VR256:$src2))]>;<br>+ //let mayLoad = 1 in<br>+ def mY_Int : FMA3<opc, MRMSrcMem, (outs VR256:$dst),<br>+ (ins VR256:$src1, VR256:$src2, f256mem:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br>+ [(set VR256:$dst, (Int256 VR256:$src1, (MemFrag256<br>addr:$src3), VR256:$src2))]>;<br>+}<br>+}<br>+<br> multiclass fma3p_forms<bits<8> opc132, bits<8> opc213, bits<8><br>opc231,<br>- string OpcodeStr, string PackTy> {<br>- defm r132 :<br>fma3p_rm<opc132, !strconcat(OpcodeStr, !strconcat("132",<br>PackTy))>;<br>- defm r213 :<br>fma3p_rm<opc213, !strconcat(OpcodeStr, !strconcat("213",<br>PackTy))>;<br>- defm r231 :<br>fma3p_rm<opc231, !strconcat(OpcodeStr, !strconcat("231",<br>PackTy))>;<br>+ string OpcodeStr, string PackTy,<br>+ PatFrag MemFrag128, PatFrag MemFrag256,<br>+ Intrinsic Int128, Intrinsic Int256> {<br>+ defm r132 : fma3p_rm_int <opc132, !strconcat(OpcodeStr,<br>!strconcat("132", PackTy)),<br>+ MemFrag128, MemFrag256, Int128,<br>Int256>;<br>+ defm r132 : fma3p_rm <opc132, !strconcat(OpcodeStr,<br>!strconcat("132", PackTy))>;<br>+ defm r213 : fma3p_rm <opc213, !strconcat(OpcodeStr,<br>!strconcat("213", PackTy))>;<br>+ defm r231 : fma3p_rm <opc231, !strconcat(OpcodeStr,<br>!strconcat("231", PackTy))>;<br> }<br><br> // Fused Multiply-Add<br> let ExeDomain = SSEPackedSingle in {<br>- defm VFMADDPS : fma3p_forms<0x98, 0xA8, 0xB8, "vfmadd",<br>"ps">;<br>- defm VFMSUBPS : fma3p_forms<0x9A, 0xAA, 0xBA, "vfmsub",<br>"ps">;<br>- defm VFMADDSUBPS : fma3p_forms<0x96, 0xA6, 0xB6, "vfmaddsub",<br>"ps">;<br>- defm VFMSUBADDPS : fma3p_forms<0x97, 0xA7, 0xB7, "vfmsubadd",<br>"ps">;<br>+ defm VFMADDPS : fma3p_forms<0x98, 0xA8, 0xB8, "vfmadd",<br>"ps", memopv4f32, memopv8f32,<br>+ int_x86_fma4_vfmadd_ps, int_x86_fma4_vfmadd_ps_256>;<br>+ defm VFMSUBPS : fma3p_forms<0x9A, 0xAA, 0xBA, "vfmsub",<br>"ps", memopv4f32, memopv8f32,<br>+ int_x86_fma4_vfmsub_ps, int_x86_fma4_vfmsub_ps_256>;<br>+ defm VFMADDSUBPS : fma3p_forms<0x96, 0xA6, 0xB6, "vfmaddsub",<br>"ps", memopv4f32, memopv8f32,<br>+ int_x86_fma4_vfmaddsub_ps, int_x86_fma4_vfmaddsub_ps_256>;<br>+ defm VFMSUBADDPS : fma3p_forms<0x97, 0xA7, 0xB7, "vfmsubadd",<br>"ps", memopv4f32, memopv8f32,<br>+ int_x86_fma4_vfmsubadd_ps, int_x86_fma4_vfmaddsub_ps_256>;<br> }<br><br> let ExeDomain = SSEPackedDouble in {<br>- defm VFMADDPD : fma3p_forms<0x98, 0xA8, 0xB8, "vfmadd",<br>"pd">, VEX_W;<br>- defm VFMSUBPD : fma3p_forms<0x9A, 0xAA, 0xBA, "vfmsub",<br>"pd">, VEX_W;<br>- defm VFMADDSUBPD : fma3p_forms<0x96, 0xA6, 0xB6, "vfmaddsub",<br>"pd">, VEX_W;<br>- defm VFMSUBADDPD : fma3p_forms<0x97, 0xA7, 0xB7, "vfmsubadd",<br>"pd">, VEX_W;<br>+ defm VFMADDPD : fma3p_forms<0x98, 0xA8, 0xB8, "vfmadd",<br>"pd", memopv2f64, memopv4f64,<br>+ int_x86_fma4_vfmadd_pd, int_x86_fma4_vfmadd_pd_256>, VEX_W;<br>+ defm VFMSUBPD : fma3p_forms<0x9A, 0xAA, 0xBA, "vfmsub",<br>"pd", memopv2f64, memopv4f64,<br>+ int_x86_fma4_vfmsub_pd, int_x86_fma4_vfmsub_pd_256>, VEX_W;<br>+ defm VFMADDSUBPD : fma3p_forms<0x96, 0xA6, 0xB6, "vfmaddsub",<br>"pd", memopv2f64, memopv4f64,<br>+ int_x86_fma4_vfmaddsub_pd, int_x86_fma4_vfmaddsub_pd_256>,<br>VEX_W;<br>+ defm VFMSUBADDPD : fma3p_forms<0x97, 0xA7, 0xB7, "vfmsubadd",<br>"pd", memopv2f64, memopv4f64,<br>+ int_x86_fma4_vfmsubadd_pd, int_x86_fma4_vfmsubadd_pd_256>,<br>VEX_W; }<br><br> // Fused Negative Multiply-Add<br> let ExeDomain = SSEPackedSingle in {<br>- defm VFNMADDPS : fma3p_forms<0x9C, 0xAC, 0xBC, "vfnmadd",<br>"ps">;<br>- defm VFNMSUBPS : fma3p_forms<0x9E, 0xAE, 0xBE, "vfnmsub",<br>"ps">;<br>+ defm VFNMADDPS : fma3p_forms<0x9C, 0xAC, 0xBC, "vfnmadd", "ps",<br> memopv4f32, memopv8f32,<br>+ int_x86_fma4_vfnmadd_ps, int_x86_fma4_vfnmadd_ps_256>;<br>+ defm VFNMSUBPS : fma3p_forms<0x9E, 0xAE, 0xBE, "vfnmsub", "ps",<br> memopv4f32, memopv8f32,<br>+ int_x86_fma4_vfnmsub_ps, int_x86_fma4_vfnmsub_ps_256>;<br> }<br> let ExeDomain = SSEPackedDouble in {<br>- defm VFNMADDPD : fma3p_forms<0x9C, 0xAC, 0xBC, "vfnmadd",<br>"pd">, VEX_W;<br>- defm VFNMSUBPD : fma3p_forms<0x9E, 0xAE, 0xBE, "vfnmsub",<br>"pd">, VEX_W;<br>+ defm VFNMADDPD : fma3p_forms<0x9C, 0xAC, 0xBC, "vfnmadd", "pd",<br>memopv2f64, memopv4f64,<br>+ int_x86_fma4_vfnmadd_pd, int_x86_fma4_vfnmadd_pd_256>, VEX_W;<br>+ defm VFNMSUBPD : fma3p_forms<0x9E, 0xAE, 0xBE, "vfnmsub", "pd",<br>memopv2f64, memopv4f64,<br>+ int_x86_fma4_vfnmsub_pd, int_x86_fma4_vfnmsub_pd_256>, VEX_W;<br> }<br><br>-multiclass fma3s_rm<bits<8> opc, string OpcodeStr, X86MemOperand<br>x86memop> {<br>- def r : FMA3<opc, MRMSrcReg, (outs VR128:$dst),<br>- (ins VR128:$src1, VR128:$src2),<br>- !strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst,<br>$src1, $src2}"),<br>+let Predicates = [HasFMA3], AddedComplexity = 20 in {<br>+//------------<br>+// FP double precision ADD - 256<br>+//------------<br>+<br>+// FMA231: src1 = src2*src3 + src1<br>+def : Pat<(v4f64 (fadd (fmul VR256:$src2, (memopv4f64<br>addr:$src3)), VR256:$src1)),<br>+ (VFMADDPDr231mY VR256:$src1, VR256:$src2, addr:$src3)>;<br>+<br>+// FMA231: src1 = src2*src3 + src1<br>+def : Pat<(v4f64 (fadd (fmul VR256:$src2, VR256:$src3),<br>VR256:$src1)),<br>+ (VFMADDPDr231rY VR256:$src1, VR256:$src2, VR256:$src3)>;<br>+<br>+<br>+//------------<br>+// FP double precision ADD - 128<br>+//------------<br>+<br>+<br>+// FMA231: src1 = src2*src3 + src1<br>+def : Pat<(v2f64 (fadd (fmul VR128:$src2, (memopv2f64<br>addr:$src3)), VR128:$src1)),<br>+ (VFMADDPDr231m VR128:$src1, VR128:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = src2*src3 + src1<br>+def : Pat<(v2f64 (fadd (fmul VR128:$src2, VR128:$src3),<br>VR128:$src1)),<br>+ (VFMADDPDr231r VR128:$src1, VR128:$src2,<br>VR128:$src3)>; +<br>+//------------<br>+// FP double precision SUB - 256<br>+//------------<br>+// FMA231: src1 = src2*src3 - src1<br>+def : Pat<(v4f64 (fsub (fmul VR256:$src2, (memopv4f64<br>addr:$src3)), VR256:$src1)),<br>+ (VFMSUBPDr231mY VR256:$src1, VR256:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = src2*src3 - src1<br>+def : Pat<(v4f64 (fsub (fmul VR256:$src2, VR256:$src3),<br>VR256:$src1)),<br>+ (VFMSUBPDr231rY VR256:$src1, VR256:$src2,<br>VR256:$src3)>; +<br>+<br>+//------------<br>+// FP double precision SUB - 128<br>+//------------<br>+<br>+// FMA231: src1 = src2*src3 - src1<br>+def : Pat<(v2f64 (fsub (fmul VR128:$src2, (memopv2f64<br>addr:$src3)), VR128:$src1)),<br>+ (VFMSUBPDr231m VR128:$src1, VR128:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = src2*src3 - src1<br>+def : Pat<(v2f64 (fsub (fmul VR128:$src2, VR128:$src3),<br>VR128:$src1)),<br>+ (VFMSUBPDr231r VR128:$src1, VR128:$src2,<br>VR128:$src3)>; +<br>+//------------<br>+// FP double precision FNMADD - 256<br>+//------------<br>+// FMA231: src1 = - src2*src3 + src1<br>+def : Pat<(v4f64 (fsub VR256:$src1, (fmul VR256:$src2,<br>(memopv4f64 addr:$src3)))),<br>+ (VFNMADDPDr231mY VR256:$src1, VR256:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = - src2*src3 + src1<br>+def : Pat<(v4f64 (fsub VR256:$src1, (fmul VR256:$src2,<br>VR256:$src3))),<br>+ (VFNMADDPDr231rY VR256:$src1, VR256:$src2,<br>VR256:$src3)>; +<br>+//------------<br>+// FP double precision FNMADD - 128<br>+//------------<br>+<br>+// FMA231: src1 = - src2*src3 + src1<br>+def : Pat<(v2f64 (fsub VR128:$src1, (fmul VR128:$src2,<br>(memopv2f64 addr:$src3)))),<br>+ (VFNMADDPDr231m VR128:$src1, VR128:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = - src2*src3 + src1<br>+def : Pat<(v2f64 (fsub VR128:$src1, (fmul VR128:$src2,<br>VR128:$src3))),<br>+ (VFNMADDPDr231r VR128:$src1, VR128:$src2,<br>VR128:$src3)>; +<br>+//------------<br>+// FP single precision ADD - 256<br>+//------------<br>+<br>+// FMA231: src1 = src2*src3 + src1<br>+def : Pat<(v8f32 (fadd (fmul VR256:$src2, VR256:$src3),<br>VR256:$src1)),<br>+ (VFMADDPSr231rY VR256:$src1, VR256:$src2,<br>VR256:$src3)>; +<br>+// FMA213 : src1 = src2*src1 + src3<br>+def : Pat<(v8f32 (fadd (fmul VR256:$src1, VR256:$src2),<br>(memopv8f32 addr:$src3))),<br>+ (VFMADDPSr213mY VR256:$src1, VR256:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = src2*src3 + src1<br>+def : Pat<(v8f32 (fadd (fmul (memopv8f32 addr:$src3),<br>VR256:$src2), VR256:$src1)),<br>+ (VFMADDPSr231mY VR256:$src1, VR256:$src2,<br>addr:$src3)>; +<br>+// FMA213: src1 = src2*src1 + src3<br>+def : Pat<(v8f32 (fadd (fmul VR256:$src2, VR256:$src1),<br>VR256:$src3)),<br>+ (VFMADDPSr213rY VR256:$src1, VR256:$src2,<br>VR256:$src3)>; +<br>+//------------<br>+// FP single precision ADD - 128<br>+//------------<br>+<br>+// FMA231 : src1 = src2*src3 + src1<br>+def : Pat<(v4f32 (fadd (fmul VR128:$src2, (memopv4f32<br>addr:$src3)), VR128:$src1)),<br>+ (VFMADDPSr231m VR128:$src1, VR128:$src2,<br>addr:$src3)>; +<br>+// FMA231 : src1 = src2*src3 + src1<br>+def : Pat<(v4f32 (fadd (fmul VR128:$src2, VR128:$src3),<br>VR128:$src1)),<br>+ (VFMADDPSr231r VR128:$src1, VR128:$src2, VR128:$src3)>;<br>+<br>+//------------<br>+// FP single precision SUB - 256<br>+//------------<br>+// FMA231: src1 = src2*src3 - src1<br>+def : Pat<(v8f32 (fsub (fmul VR256:$src2, (memopv8f32<br>addr:$src3)), VR256:$src1)),<br>+ (VFMSUBPSr231mY VR256:$src1, VR256:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = src2*src3 - src1<br>+def : Pat<(v8f32 (fsub (fmul VR256:$src2, VR256:$src3),<br>VR256:$src1)),<br>+ (VFMSUBPSr231rY VR256:$src1, VR256:$src2,<br>VR256:$src3)>; +<br>+//------------<br>+// FP single precision SUB - 128<br>+//------------<br>+// FMA231 : src1 = src2*src3 - src1<br>+def : Pat<(v4f32 (fsub (fmul VR128:$src2, (memopv4f32<br>addr:$src3)), VR128:$src1)),<br>+ (VFMSUBPSr231m VR128:$src1, VR128:$src2,<br>addr:$src3)>; +<br>+// FMA231 : src1 = src2*src3 - src1<br>+def : Pat<(v4f32 (fsub (fmul VR128:$src2, VR128:$src3),<br>VR128:$src1)),<br>+ (VFMSUBPSr231r VR128:$src1, VR128:$src2,<br>VR128:$src3)>; +<br>+//------------<br>+// FP single precision FNMADD - 256<br>+//------------<br>+// FMA231: src1 = - src2*src3 + src1<br>+def : Pat<(v8f32 (fsub VR256:$src1, (fmul VR256:$src2,<br>(memopv8f32 addr:$src3)))),<br>+ (VFNMADDPSr231mY VR256:$src1, VR256:$src2,<br>addr:$src3)>; +<br>+// FMA231: src1 = - src2*src3 + src1<br>+def : Pat<(v8f32 (fsub VR256:$src1, (fmul VR256:$src2,<br>VR256:$src3))),<br>+ (VFNMADDPSr231rY VR256:$src1, VR256:$src2,<br>VR256:$src3)>; +<br>+//------------<br>+// FP single precision FNMADD - 128<br>+//------------<br>+<br>+// FMA231 : src1 = src2*src3 - src1<br>+def : Pat<(v4f32 (fsub VR128:$src1, (fmul VR128:$src2,<br>(memopv4f32 addr:$src3)))),<br>+ (VFNMADDPSr231m VR128:$src1, VR128:$src2,<br>addr:$src3)>; +<br>+// FMA231 : src1 = src2*src3 - src1<br>+def : Pat<(v4f32 (fsub VR128:$src1, (fmul VR128:$src2,<br>VR128:$src3))),<br>+ (VFNMADDPSr231r VR128:$src1, VR128:$src2,<br>VR128:$src3)>; +<br>+} // HasFMA3<br>+<br>+//------------------------------<br>+// SCALAR<br>+//------------------------------<br>+<br>+let Constraints = "$src1 = $dst" in {<br>+multiclass fma3s_rm<bits<8> opc, string OpcodeStr, X86MemOperand<br>x86memop, RegisterClass RC> {<br>+ def r : FMA3<opc, MRMSrcReg, (outs RC:$dst),<br>+ (ins RC:$src1, RC:$src2, RC:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br> []>;<br>- def m : FMA3<opc, MRMSrcMem, (outs VR128:$dst),<br>- (ins VR128:$src1, x86memop:$src2),<br>- !strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst,<br>$src1, $src2}"),<br>+ def m : FMA3<opc, MRMSrcMem, (outs RC:$dst),<br>+ (ins RC:$src1, RC:$src2, x86memop:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br> []>;<br> }<br><br>+multiclass fma3s_rm_int<bits<8> opc, string<br>OpcodeStr,X86MemOperand x86memop, RegisterClass RC,<br>+ Intrinsic IntId> {<br>+ def r_Int : FMA3<opc, MRMSrcReg, (outs RC:$dst),<br>+ (ins RC:$src1, RC:$src2, RC:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br>+ [(set RC:$dst, (IntId RC:$src1, RC:$src3,<br>RC:$src2))]>;<br>+ def m_Int : FMA3<opc, MRMSrcMem, (outs RC:$dst),<br>+ (ins RC:$src1, VR128:$src2, x86memop:$src3),<br>+ !strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst,<br>$src2, $src3}"),<br>+ [(set RC:$dst, (IntId RC:$src1, (load addr:$src3),<br>RC:$src2))]>;<br>+}<br>+}<br>+<br> multiclass fma3s_forms<bits<8> opc132, bits<8> opc213, bits<8><br>opc231,<br>- string OpcodeStr> {<br>- defm SSr132 : fma3s_rm<opc132, !strconcat(OpcodeStr, "132ss"),<br>f32mem>;<br>- defm SSr213 : fma3s_rm<opc213, !strconcat(OpcodeStr, "213ss"),<br>f32mem>;<br>- defm SSr231 : fma3s_rm<opc231, !strconcat(OpcodeStr, "231ss"),<br>f32mem>;<br>- defm SDr132 : fma3s_rm<opc132, !strconcat(OpcodeStr, "132sd"),<br>f64mem>, VEX_W;<br>- defm SDr213 : fma3s_rm<opc213, !strconcat(OpcodeStr, "213sd"),<br>f64mem>, VEX_W;<br>- defm SDr231 : fma3s_rm<opc231, !strconcat(OpcodeStr, "231sd"),<br>f64mem>, VEX_W;<br>+ string OpcodeStr, string PackTy,<br>X86MemOperand MemOp,<br>+ RegisterClass RC, Intrinsic IntId> {<br>+ defm r132 : fma3s_rm <opc132, !strconcat(OpcodeStr,<br>!strconcat("132", PackTy)), MemOp, RC>;<br>+ defm r213 : fma3s_rm <opc213, !strconcat(OpcodeStr,<br>!strconcat("213", PackTy)), MemOp, RC>;<br>+ defm r231 : fma3s_rm <opc231, !strconcat(OpcodeStr,<br>!strconcat("231", PackTy)), MemOp, RC>;<br>+ defm r132_Int: fma3s_rm_int <opc132, !strconcat(OpcodeStr,<br>!strconcat("132", PackTy)), MemOp, VR128, IntId>;<br> }<br><br>-defm VFMADD : fma3s_forms<0x99, 0xA9, 0xB9, "vfmadd">, VEX_LIG;<br>-defm VFMSUB : fma3s_forms<0x9B, 0xAB, 0xBB, "vfmsub">, VEX_LIG;<br>+defm VFMADDSS : fma3s_forms<0x99, 0xA9, 0xB9, "vfmadd", "ss",<br>f32mem, FR32, int_x86_fma4_vfmadd_ss>, VEX_LIG;<br>+defm VFMADDSD : fma3s_forms<0x99, 0xA9, 0xB9, "vfmadd", "sd",<br>f64mem, FR64, int_x86_fma4_vfmadd_sd>, VEX_W, VEX_LIG;<br>+defm VFMSUBSS : fma3s_forms<0x9B, 0xAB, 0xBB, "vfmsub", "ss",<br>f32mem, FR32, int_x86_fma4_vfmsub_ss>, VEX_LIG;<br>+defm VFMSUBSD : fma3s_forms<0x9B, 0xAB, 0xBB, "vfmsub", "sd",<br>f64mem, FR64, int_x86_fma4_vfmsub_sd>, VEX_W, VEX_LIG;<br>+<br>+defm VFNMADDSS : fma3s_forms<0x9D, 0xAD, 0xBD, "vfnmadd", "ss",<br>f32mem, FR32, int_x86_fma4_vfnmadd_ss>, VEX_LIG;<br>+defm VFNMADDSD : fma3s_forms<0x9D, 0xAD, 0xBD, "vfnmadd", "sd",<br>f64mem, FR64, int_x86_fma4_vfnmadd_sd>, VEX_W, VEX_LIG;<br>+defm VFNMSUBSS : fma3s_forms<0x9F, 0xAF, 0xBF, "vfnmsub", "ss",<br>f32mem, FR32, int_x86_fma4_vfnmsub_ss>, VEX_LIG;<br>+defm VFNMSUBSD : fma3s_forms<0x9F, 0xAF, 0xBF, "vfnmsub", "sd",<br>f64mem, FR64, int_x86_fma4_vfnmsub_sd>, VEX_W, VEX_LIG;<br>+<br>+<br>+let Predicates = [HasFMA3], AddedComplexity = 20 in {<br>+<br>+//------------<br>+// FP scalar ADD<br>+//------------<br><br>-defm VFNMADD : fma3s_forms<0x9D, 0xAD, 0xBD, "vfnmadd">, VEX_LIG;<br>-defm VFNMSUB : fma3s_forms<0x9F, 0xAF, 0xBF, "vfnmsub">, VEX_LIG;<br>+<br>+// FMADD231 : src1 = src2*src3 + src1<br>+def : Pat<(f32 (fadd (fmul FR32:$src2, FR32:$src3), FR32:$src1)),<br>+ (VFMADDSSr231r FR32:$src1, FR32:$src2, FR32:$src3)>;<br>+<br>+def : Pat<(f32 (fadd (fmul FR32:$src2, (loadf32 addr:$src3)),<br>FR32:$src1)),<br>+ (VFMADDSSr231m FR32:$src1, FR32:$src2, addr:$src3)>;<br>+<br>+def : Pat<(f64 (fadd (fmul FR64:$src2, FR64:$src3), FR64:$src1)),<br>+ (VFMADDSDr231r FR64:$src1, FR64:$src2, FR64:$src3)>;<br>+<br>+def : Pat<(f64 (fadd (fmul FR64:$src2, (loadf64 addr:$src3)),<br>FR64:$src1)),<br>+ (VFMADDSDr231m FR64:$src1, FR64:$src2, addr:$src3)>;<br>+<br>+<br>+<br>+//------------<br>+// FP scalar SUB src2*src3 - src1<br>+//------------<br>+<br>+def : Pat<(f32 (fsub (fmul FR32:$src2, FR32:$src3), FR32:$src1)),<br>+ (VFMSUBSSr231r FR32:$src1, FR32:$src2, FR32:$src3)>;<br>+<br>+def : Pat<(f32 (fsub (fmul FR32:$src2, (loadf32 addr:$src3)),<br>FR32:$src1)),<br>+ (VFMSUBSSr231m FR32:$src1, FR32:$src2, addr:$src3)>;<br>+<br>+def : Pat<(f64 (fsub (fmul FR64:$src2, FR64:$src3), FR64:$src1)),<br>+ (VFMSUBSDr231r FR64:$src1, FR64:$src2, FR64:$src3)>;<br>+<br>+def : Pat<(f64 (fsub (fmul FR64:$src2, (loadf64 addr:$src3)),<br>FR64:$src1)),<br>+ (VFMSUBSDr231m FR64:$src1, FR64:$src2, addr:$src3)>;<br>+<br>+//------------<br>+// FP scalar NADD src1 - src2*src3<br>+//------------<br>+<br>+def : Pat<(f32 (fsub FR32:$src1, (fmul FR32:$src2, FR32:$src3))),<br>+ (VFNMADDSSr231r FR32:$src1, FR32:$src2, FR32:$src3)>;<br>+<br>+def : Pat<(f32 (fsub FR32:$src1, (fmul FR32:$src2, (loadf32<br>addr:$src3)))),<br>+ (VFNMADDSSr231m FR32:$src1, FR32:$src2, addr:$src3)>;<br>+<br>+def : Pat<(f64 (fsub FR64:$src1, (fmul FR64:$src2, FR64:$src3))),<br>+ (VFNMADDSDr231r FR64:$src1, FR64:$src2, FR64:$src3)>;<br>+<br>+def : Pat<(f64 (fsub FR64:$src1, (fmul FR64:$src2, (loadf64<br>addr:$src3)))),<br>+ (VFNMADDSDr231m FR64:$src1, FR64:$src2, addr:$src3)>;<br>+<br>+} // HasFMA3<br><br><br> //===----------------------------------------------------------------------===//<br> // FMA4 - AMD 4 operand Fused Multiply-Add instructions<br>@@ -178,6 +453,8 @@<br> } // isCodeGenOnly = 1<br> }<br><br>+let Predicates = [HasFMA4] in {<br>+<br> defm VFMADDSS4 : fma4s<0x6A, "vfmaddss", ssmem, sse_load_f32,<br> int_x86_fma4_vfmadd_ss>;<br> defm VFMADDSD4 : fma4s<0x6B, "vfmaddsd", sdmem, sse_load_f64,<br>@@ -218,3 +495,5 @@<br> int_x86_fma4_vfmsubadd_ps_256,<br>memopv4f32,<br>memopv8f32>;<br> defm VFMSUBADDPD4 : fma4p<0x5F, "vfmsubaddpd",<br>int_x86_fma4_vfmsubadd_pd,<br> int_x86_fma4_vfmsubadd_pd_256,<br>memopv2f64,<br>memopv4f64>;<br>+} // HasFMA4<br>+<br><br>Modified: llvm/trunk/lib/Target/X86/X86InstrInfo.cpp<br>URL:<br>http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86InstrInfo.cpp?rev=157737&r1=157736&r2=157737&view=diff<br><br>==============================================================================<br>--- llvm/trunk/lib/Target/X86/X86InstrInfo.cpp (original)<br>+++ llvm/trunk/lib/Target/X86/X86InstrInfo.cpp Thu May 31<br>04:20:20 2012 @@ -58,6 +58,7 @@<br> TB_INDEX_0 = 0,<br> TB_INDEX_1 = 1,<br> TB_INDEX_2 = 2,<br>+ TB_INDEX_3 = 3,<br> TB_INDEX_MASK = 0xff,<br><br> // Minimum alignment required for load/store.<br>@@ -1122,6 +1123,75 @@<br> // Index 2, folded load<br> Flags | TB_INDEX_2 | TB_FOLDED_LOAD);<br> }<br>+<br>+ static const X86OpTblEntry OpTbl3[] = {<br>+ // FMA foldable instructions<br>+ { X86::VFMADDSSr231r, X86::VFMADDSSr231m, 0 },<br>+ { X86::VFMADDSDr231r, X86::VFMADDSDr231m, 0 },<br>+ { X86::VFMADDSSr132r, X86::VFMADDSSr132m, 0 },<br>+ { X86::VFMADDSDr132r, X86::VFMADDSDr132m, 0 },<br>+<br>+ { X86::VFMADDPSr231r, X86::VFMADDPSr231m,<br>TB_ALIGN_16 },<br>+ { X86::VFMADDPDr231r, X86::VFMADDPDr231m,<br>TB_ALIGN_16 },<br>+ { X86::VFMADDPSr132r, X86::VFMADDPSr132m,<br>TB_ALIGN_16 },<br>+ { X86::VFMADDPDr132r, X86::VFMADDPDr132m,<br>TB_ALIGN_16 },<br>+ { X86::VFMADDPSr213r, X86::VFMADDPSr213m,<br>TB_ALIGN_16 },<br>+ { X86::VFMADDPDr213r, X86::VFMADDPDr213m,<br>TB_ALIGN_16 },<br>+ { X86::VFMADDPSr231rY, X86::VFMADDPSr231mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMADDPDr231rY, X86::VFMADDPDr231mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMADDPSr132rY, X86::VFMADDPSr132mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMADDPDr132rY, X86::VFMADDPDr132mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMADDPSr213rY, X86::VFMADDPSr213mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMADDPDr213rY, X86::VFMADDPDr213mY,<br>TB_ALIGN_32 }, +<br>+ { X86::VFNMADDSSr231r, X86::VFNMADDSSr231m, 0 },<br>+ { X86::VFNMADDSDr231r, X86::VFNMADDSDr231m, 0 },<br>+ { X86::VFNMADDSSr132r, X86::VFNMADDSSr132m, 0 },<br>+ { X86::VFNMADDSDr132r, X86::VFNMADDSDr132m, 0 },<br>+<br>+ { X86::VFNMADDPSr231r, X86::VFNMADDPSr231m,<br>TB_ALIGN_16 },<br>+ { X86::VFNMADDPDr231r, X86::VFNMADDPDr231m,<br>TB_ALIGN_16 },<br>+ { X86::VFNMADDPSr132r, X86::VFNMADDPSr132m,<br>TB_ALIGN_16 },<br>+ { X86::VFNMADDPDr132r, X86::VFNMADDPDr132m,<br>TB_ALIGN_16 },<br>+ { X86::VFNMADDPSr213r, X86::VFNMADDPSr213m,<br>TB_ALIGN_16 },<br>+ { X86::VFNMADDPDr213r, X86::VFNMADDPDr213m,<br>TB_ALIGN_16 },<br>+ { X86::VFNMADDPSr231rY, X86::VFNMADDPSr231mY,<br>TB_ALIGN_32 },<br>+ { X86::VFNMADDPDr231rY, X86::VFNMADDPDr231mY,<br>TB_ALIGN_32 },<br>+ { X86::VFNMADDPSr132rY, X86::VFNMADDPSr132mY,<br>TB_ALIGN_32 },<br>+ { X86::VFNMADDPDr132rY, X86::VFNMADDPDr132mY,<br>TB_ALIGN_32 },<br>+ { X86::VFNMADDPSr213rY, X86::VFNMADDPSr213mY,<br>TB_ALIGN_32 },<br>+ { X86::VFNMADDPDr213rY, X86::VFNMADDPDr213mY,<br>TB_ALIGN_32 }, +<br>+ { X86::VFMSUBSSr231r, X86::VFMSUBSSr231m, 0 },<br>+ { X86::VFMSUBSDr231r, X86::VFMSUBSDr231m, 0 },<br>+ { X86::VFMSUBSSr132r, X86::VFMSUBSSr132m, 0 },<br>+ { X86::VFMSUBSDr132r, X86::VFMSUBSDr132m, 0 },<br>+<br>+ { X86::VFMSUBPSr231r, X86::VFMSUBPSr231m,<br>TB_ALIGN_16 },<br>+ { X86::VFMSUBPDr231r, X86::VFMSUBPDr231m,<br>TB_ALIGN_16 },<br>+ { X86::VFMSUBPSr132r, X86::VFMSUBPSr132m,<br>TB_ALIGN_16 },<br>+ { X86::VFMSUBPDr132r, X86::VFMSUBPDr132m,<br>TB_ALIGN_16 },<br>+ { X86::VFMSUBPSr213r, X86::VFMSUBPSr213m,<br>TB_ALIGN_16 },<br>+ { X86::VFMSUBPDr213r, X86::VFMSUBPDr213m,<br>TB_ALIGN_16 },<br>+ { X86::VFMSUBPSr231rY, X86::VFMSUBPSr231mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMSUBPDr231rY, X86::VFMSUBPDr231mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMSUBPSr132rY, X86::VFMSUBPSr132mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMSUBPDr132rY, X86::VFMSUBPDr132mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMSUBPSr213rY, X86::VFMSUBPSr213mY,<br>TB_ALIGN_32 },<br>+ { X86::VFMSUBPDr213rY, X86::VFMSUBPDr213mY,<br>TB_ALIGN_32 }, +<br>+ };<br>+<br>+ for (unsigned i = 0, e = array_lengthof(OpTbl3); i != e; ++i) {<br>+ unsigned RegOp = OpTbl3[i].RegOp;<br>+ unsigned MemOp = OpTbl3[i].MemOp;<br>+ unsigned Flags = OpTbl3[i].Flags;<br>+ AddTableEntry(RegOp2MemOpTable3, MemOp2RegOpTable,<br>+ RegOp, MemOp,<br>+ // Index 3, folded load<br>+ Flags | TB_INDEX_3 | TB_FOLDED_LOAD);<br>+ }<br>+<br> }<br><br> void<br><br>Modified: llvm/trunk/lib/Target/X86/X86InstrInfo.h<br>URL:<br>http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86InstrInfo.h?rev=157737&r1=157736&r2=157737&view=diff<br><br>==============================================================================<br>--- llvm/trunk/lib/Target/X86/X86InstrInfo.h (original)<br>+++ llvm/trunk/lib/Target/X86/X86InstrInfo.h Thu May 31 04:20:20<br>2012 @@ -128,7 +128,8 @@<br> X86TargetMachine &TM;<br> const X86RegisterInfo RI;<br><br>- /// RegOp2MemOpTable2Addr, RegOp2MemOpTable0,<br>RegOp2MemOpTable1,<br>+ /// RegOp2MemOpTable3Addr, RegOp2MemOpTable2Addr,<br>+ /// RegOp2MemOpTable0, RegOp2MemOpTable1,<br> /// RegOp2MemOpTable2 - Load / store folding opcode maps.<br> ///<br> typedef DenseMap<unsigned,<br>@@ -137,6 +138,7 @@<br> RegOp2MemOpTableType RegOp2MemOpTable0;<br> RegOp2MemOpTableType RegOp2MemOpTable1;<br> RegOp2MemOpTableType RegOp2MemOpTable2;<br>+ RegOp2MemOpTableType RegOp2MemOpTable3;<br><br> /// MemOp2RegOpTable - Load / store unfolding opcode map.<br> ///<br><br>Modified: llvm/trunk/lib/Target/X86/X86Subtarget.cpp<br>URL:<br>http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86Subtarget.cpp?rev=157737&r1=157736&r2=157737&view=diff<br><br>==============================================================================<br>--- llvm/trunk/lib/Target/X86/X86Subtarget.cpp (original)<br>+++ llvm/trunk/lib/Target/X86/X86Subtarget.cpp Thu May 31<br>04:20:20 2012 @@ -205,10 +205,12 @@<br> HasCLMUL = true;<br> ToggleFeature(X86::FeatureCLMUL);<br> }<br>- if ((ECX >> 12) & 0x1) {<br>- HasFMA3 = true;<br>- ToggleFeature(X86::FeatureFMA3);<br>- }<br>+ // FMA3 autodetection is switched off until we have a special<br>flag<br>+ // in code generator<br>+ //if ((ECX >> 12) & 0x1) {<br>+ // HasFMA3 = true;<br>+ // ToggleFeature(X86::FeatureFMA3);<br>+ //}<br> if (IsIntel && ((ECX >> 22) & 0x1)) {<br> HasMOVBE = true;<br> ToggleFeature(X86::FeatureMOVBE);<br><br>Added: llvm/trunk/test/CodeGen/X86/fma3-intrinsics.ll<br>URL:<br>http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/fma3-intrinsics.ll?rev=157737&view=auto<br><br>==============================================================================<br>--- llvm/trunk/test/CodeGen/X86/fma3-intrinsics.ll (added)<br>+++ llvm/trunk/test/CodeGen/X86/fma3-intrinsics.ll Thu May 31<br>04:20:20 2012<br>@@ -0,0 +1,132 @@<br>+; RUN: llc < %s -mtriple=x86_64-pc-win32 -mcpu=core-avx2<br>-mattr=avx2,+fma3 | FileCheck %s<br>+<br>+define <4 x float> @test_x86_fmadd_ss(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fmadd132ss {{.*\(%r.*}}, %xmm<br>+ %res = call <4 x float> @llvm.x86.fma4.vfmadd.ss(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfmadd.ss(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+define <4 x float> @test_x86_fmadd_ps(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fmadd132ps<br>+ %res = call <4 x float> @llvm.x86.fma4.vfmadd.ps(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfmadd.ps(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+define <8 x float> @test_x86_fmadd_ps_y(<8 x float> %a0, <8 x<br>float> %a1, <8 x float> %a2) {<br>+ ; CHECK: fmadd132ps {{.*\(%r.*}}, %ymm<br>+ %res = call <8 x float> @llvm.x86.fma4.vfmadd.ps.256(<8 x<br>float> %a0, <8 x float> %a1, <8 x float> %a2) nounwind<br>+ ret <8 x float> %res<br>+}<br>+declare <8 x float> @llvm.x86.fma4.vfmadd.ps.256(<8 x float>, <8<br>x<br>float>, <8 x float>) nounwind readnone<br>+<br>+define <4 x float> @test_x86_fnmadd_ss(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fnmadd132ss {{.*\(%r.*}}, %xmm<br>+ %res = call <4 x float> @llvm.x86.fma4.vfnmadd.ss(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfnmadd.ss(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+define <4 x float> @test_x86_fnmadd_ps(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fnmadd132ps<br>+ %res = call <4 x float> @llvm.x86.fma4.vfnmadd.ps(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfnmadd.ps(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+define <8 x float> @test_x86_fnmadd_ps_y(<8 x float> %a0, <8 x<br>float> %a1, <8 x float> %a2) {<br>+ ; CHECK: fnmadd132ps {{.*\(%r.*}}, %ymm<br>+ %res = call <8 x float> @llvm.x86.fma4.vfnmadd.ps.256(<8 x<br>float> %a0, <8 x float> %a1, <8 x float> %a2) nounwind<br>+ ret <8 x float> %res<br>+}<br>+declare <8 x float> @llvm.x86.fma4.vfnmadd.ps.256(<8 x float>,<br><8 x<br>float>, <8 x float>) nounwind readnone<br>+<br>+<br>+define <4 x float> @test_x86_fmsub_ss(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fmsub132ss<br>+ %res = call <4 x float> @llvm.x86.fma4.vfmsub.ss(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfmsub.ss(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+define <4 x float> @test_x86_fmsub_ps(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fmsub132ps<br>+ %res = call <4 x float> @llvm.x86.fma4.vfmsub.ps(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfmsub.ps(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+define <4 x float> @test_x86_fnmsub_ss(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fnmsub132ss<br>+ %res = call <4 x float> @llvm.x86.fma4.vfnmsub.ss(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfnmsub.ss(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+define <4 x float> @test_x86_fnmsub_ps(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fnmsub132ps<br>+ %res = call <4 x float> @llvm.x86.fma4.vfnmsub.ps(<4 x float><br>%a0, <4 x float> %a1, <4 x float> %a2) nounwind<br>+ ret <4 x float> %res<br>+}<br>+declare <4 x float> @llvm.x86.fma4.vfnmsub.ps(<4 x float>, <4 x<br>float>, <4 x float>) nounwind readnone<br>+<br>+;;;;<br>+<br>+define <2 x double> @test_x86_fmadd_sd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fmadd132sd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfmadd.sd(<2 x double><br>%a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfmadd.sd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br>+<br>+define <2 x double> @test_x86_fmadd_pd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fmadd132pd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfmadd.pd(<2 x double><br>%a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfmadd.pd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br>+<br>+define <2 x double> @test_x86_fnmadd_sd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fnmadd132sd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfnmadd.sd(<2 x<br>double> %a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfnmadd.sd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br>+<br>+define <2 x double> @test_x86_fnmadd_pd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fnmadd132pd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfnmadd.pd(<2 x<br>double> %a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfnmadd.pd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br>+<br>+<br>+<br>+define <2 x double> @test_x86_fmsub_sd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fmsub132sd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfmsub.sd(<2 x double><br>%a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfmsub.sd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br>+<br>+define <2 x double> @test_x86_fmsub_pd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fmsub132pd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfmsub.pd(<2 x double><br>%a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfmsub.pd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br>+<br>+define <2 x double> @test_x86_fnmsub_sd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fnmsub132sd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfnmsub.sd(<2 x<br>double> %a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfnmsub.sd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br>+<br>+define <2 x double> @test_x86_fnmsub_pd(<2 x double> %a0, <2 x<br>double> %a1, <2 x double> %a2) {<br>+ ; CHECK: fnmsub132pd<br>+ %res = call <2 x double> @llvm.x86.fma4.vfnmsub.pd(<2 x<br>double> %a0, <2 x double> %a1, <2 x double> %a2) nounwind<br>+ ret <2 x double> %res<br>+}<br>+declare <2 x double> @llvm.x86.fma4.vfnmsub.pd(<2 x double>, <2 x<br>double>, <2 x double>) nounwind readnone<br><br>Propchange: llvm/trunk/test/CodeGen/X86/fma3-intrinsics.ll<br><br>------------------------------------------------------------------------------<br> svn:executable = *<br><br>Added: llvm/trunk/test/CodeGen/X86/fma3.ll<br>URL:<br>http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/fma3.ll?rev=157737&view=auto<br><br>==============================================================================<br>--- llvm/trunk/test/CodeGen/X86/fma3.ll (added)<br>+++ llvm/trunk/test/CodeGen/X86/fma3.ll Thu May 31 04:20:20 2012<br>@@ -0,0 +1,66 @@<br>+; RUN: llc < %s -mtriple=x86_64-pc-win32 -mcpu=core-avx2<br>-mattr=avx2,+fma3 | FileCheck %s<br>+<br>+define <4 x float> @test_x86_fmadd_ps(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fmadd231ps {{.*\(%r.*}}, %xmm<br>+ %x = fmul <4 x float> %a0, %a1<br>+ %res = fadd <4 x float> %x, %a2<br>+ ret <4 x float> %res<br>+}<br>+<br>+define <4 x float> @test_x86_fmsub_ps(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fmsub231ps {{.*\(%r.*}}, %xmm<br>+ %x = fmul <4 x float> %a0, %a1<br>+ %res = fsub <4 x float> %x, %a2<br>+ ret <4 x float> %res<br>+}<br>+<br>+define <4 x float> @test_x86_fnmadd_ps(<4 x float> %a0, <4 x<br>float> %a1, <4 x float> %a2) {<br>+ ; CHECK: fnmadd231ps {{.*\(%r.*}}, %xmm<br>+ %x = fmul <4 x float> %a0, %a1<br>+ %res = fsub <4 x float> %a2, %x<br>+ ret <4 x float> %res<br>+}<br>+<br>+define <8 x float> @test_x86_fmadd_ps_y(<8 x float> %a0, <8 x<br>float> %a1, <8 x float> %a2) {<br>+ ; CHECK: vfmadd213ps {{.*\(%r.*}}, %ymm<br>+ %x = fmul <8 x float> %a0, %a1<br>+ %res = fadd <8 x float> %x, %a2<br>+ ret <8 x float> %res<br>+}<br>+<br>+define <4 x double> @test_x86_fmadd_pd_y(<4 x double> %a0, <4 x<br>double> %a1, <4 x double> %a2) {<br>+ ; CHECK: vfmadd231pd {{.*\(%r.*}}, %ymm<br>+ %x = fmul <4 x double> %a0, %a1<br>+ %res = fadd <4 x double> %x, %a2<br>+ ret <4 x double> %res<br>+}<br>+<br>+<br>+define <8 x float> @test_x86_fmsub_ps_y(<8 x float> %a0, <8 x<br>float> %a1, <8 x float> %a2) {<br>+ ; CHECK: fmsub231ps {{.*\(%r.*}}, %ymm<br>+ %x = fmul <8 x float> %a0, %a1<br>+ %res = fsub <8 x float> %x, %a2<br>+ ret <8 x float> %res<br>+}<br>+<br>+define <8 x float> @test_x86_fnmadd_ps_y(<8 x float> %a0, <8 x<br>float> %a1, <8 x float> %a2) {<br>+ ; CHECK: fnmadd231ps {{.*\(%r.*}}, %ymm<br>+ %x = fmul <8 x float> %a0, %a1<br>+ %res = fsub <8 x float> %a2, %x<br>+ ret <8 x float> %res<br>+}<br>+<br>+define float @test_x86_fnmadd_ss(float %a0, float %a1, float<br>%a2) {<br>+ ; CHECK: vfnmadd231ss %xmm1, %xmm0, %xmm2<br>+ %x = fmul float %a0, %a1<br>+ %res = fsub float %a2, %x<br>+ ret float %res<br>+}<br>+<br>+define double @test_x86_fnmadd_sd(double %a0, double %a1, double<br>%a2) {<br>+ ; CHECK: vfnmadd231sd %xmm1, %xmm0, %xmm2<br>+ %x = fmul double %a0, %a1<br>+ %res = fsub double %a2, %x<br>+ ret double %res<br>+}<br>+<br><br>Propchange: llvm/trunk/test/CodeGen/X86/fma3.ll<br><br>------------------------------------------------------------------------------<br> svn:executable = *<br><br><br>_______________________________________________<br>llvm-commits mailing list<br>llvm-commits@cs.uiuc.edu<br>http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits<br><br></blockquote><br><br><br>--<br>~Craig<br>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits<br><br><br><br></blockquote><br><br>--<br>~Craig<br><br></blockquote><br><br><br></blockquote><br><br><br>-- <br>Hal Finkel<br>Postdoctoral Appointee<br>Leadership Computing Facility<br>Argonne National Laboratory<br></blockquote></div><br></div></body></html>