R600: Generate native ALU instructions
Tom Stellard
tom at stellard.net
Thu Apr 25 10:30:26 PDT 2013
On Mon, Apr 22, 2013 at 08:59:56PM -0700, Tom Stellard wrote:
> On Mon, Apr 22, 2013 at 09:28:59AM -0700, Vincent Lejeune wrote:
> > Hi,
> >
> > this set of patches make llvm generate native instructions for ALU.
> > Now r600 backend emits all instructions as natively encoded ;
> > however we still emit the INSTR_NATIVE data before each instructions atm.
> > We can drop this behaviour but I'd like to keep it one or 2 weeks so that people can test it against mesa.
> >
> > Vincent
>
> Hi Vincent,
>
> Nice work, this is a major milestone for the backend.
>
> Just a few minor comments, other than that this series is:
>
> Reviewed-by: Tom Stellard <thomas.stellard at amd.com>
>
> Also, I think you could add a few test cases for the trans only
> instructions by testing that they are the last instruction in a group.
>
> I would like to test these with compute before you push them, but I'm
> having trouble applying them. Could you push a branch somewhere with
> all your outstanding patches? I will try to review the other patches
> you posted today, so you can push those to master.
>
Hi Vincent,
I just tested and there are no OpenCL piglit regressions with your native
branch. These patches did uncover a bug in the backend where we weren't
initializing the ShaderType, but I have sent a patch for this to the
list.
-Tom
>
> > From 3a6137894564734037007241d928879706d70d8f Mon Sep 17 00:00:00 2001
> > From: Vincent Lejeune <vljn at ovi.com>
> > Date: Mon, 22 Apr 2013 17:23:01 +0200
> > Subject: [PATCH 1/5] R600: add a istransonly function
> >
> > ---
> > lib/Target/R600/R600InstrInfo.cpp | 47 +++++++++++++++++++++++++++++++++++++++
> > lib/Target/R600/R600InstrInfo.h | 3 +++
> > 2 files changed, 50 insertions(+)
> >
> > diff --git a/lib/Target/R600/R600InstrInfo.cpp b/lib/Target/R600/R600InstrInfo.cpp
> > index e96f563..bfdf4c9 100644
> > --- a/lib/Target/R600/R600InstrInfo.cpp
> > +++ b/lib/Target/R600/R600InstrInfo.cpp
> > @@ -140,6 +140,53 @@ bool R600InstrInfo::isALUInstr(unsigned Opcode) const {
> > (TargetFlags & R600_InstFlag::OP3));
> > }
> >
> > +bool R600InstrInfo::isTransOnly(unsigned Opcode) const {
> > + //TODO: use FuncUnit in R600Instructions.td
> > + switch (Opcode) {
> > + case AMDGPU::INT_TO_FLT_eg:
> > + case AMDGPU::INT_TO_FLT_r600:
> > + case AMDGPU::FLT_TO_INT_eg:
> > + case AMDGPU::FLT_TO_INT_r600:
> > + case AMDGPU::UINT_TO_FLT_eg:
> > + case AMDGPU::UINT_TO_FLT_r600:
> > + case AMDGPU::FLT_TO_UINT_eg:
> > + case AMDGPU::FLT_TO_UINT_r600:
> > + case AMDGPU::COS_eg:
> > + case AMDGPU::COS_r600:
> > + case AMDGPU::SIN_eg:
> > + case AMDGPU::SIN_r600:
> > + case AMDGPU::LOG_IEEE_eg:
> > + case AMDGPU::LOG_IEEE_r600:
> > + case AMDGPU::EXP_IEEE_eg:
> > + case AMDGPU::EXP_IEEE_r600:
> > + case AMDGPU::RECIP_IEEE_eg:
> > + case AMDGPU::RECIP_IEEE_r600:
> > + case AMDGPU::RECIP_UINT_eg:
> > + case AMDGPU::RECIP_UINT_r600:
> > + case AMDGPU::MULLO_INT_eg:
> > + case AMDGPU::MULLO_INT_r600:
> > + case AMDGPU::MULHI_INT_eg:
> > + case AMDGPU::MULHI_INT_r600:
> > + case AMDGPU::MULLO_UINT_eg:
> > + case AMDGPU::MULLO_UINT_r600:
> > + case AMDGPU::MULHI_UINT_eg:
> > + case AMDGPU::MULHI_UINT_r600:
> > + case AMDGPU::RECIPSQRT_CLAMPED_eg:
> > + case AMDGPU::RECIPSQRT_CLAMPED_r600:
> > + case AMDGPU::RECIP_CLAMPED_eg:
> > + case AMDGPU::RECIP_CLAMPED_r600:
> > + case AMDGPU::RECIPSQRT_IEEE_eg:
> > + case AMDGPU::RECIPSQRT_IEEE_r600:
> > + return true;
> > + default:
> > + return false;
> > + }
> > +}
> > +
>
> I would rather see this function implementing using TSFlags, so that we
> can just set the TransOnly bit in the .td file. Or even better, is
> there some way to query FuncUnit from the instruction (maybe this is
> what you are talking about in your TODO comment)?
>
> > +bool R600InstrInfo::isTransOnly(const MachineInstr *MI) const {
> > + return isTransOnly(MI->getOpcode());
> > +}
> > +
> > bool R600InstrInfo::isCayman() const {
> > return ST.device()->getGeneration() > AMDGPUDeviceInfo::HD5XXX;
> > }
> > diff --git a/lib/Target/R600/R600InstrInfo.h b/lib/Target/R600/R600InstrInfo.h
> > index 136023f..e0ba12b 100644
> > --- a/lib/Target/R600/R600InstrInfo.h
> > +++ b/lib/Target/R600/R600InstrInfo.h
> > @@ -55,6 +55,9 @@ namespace llvm {
> > /// \returns true if this \p Opcode represents an ALU instruction.
> > bool isALUInstr(unsigned Opcode) const;
> >
> > + bool isTransOnly(unsigned Opcode) const;
> > + bool isTransOnly(const MachineInstr *MI) const;
> > +
> > bool usesVertexCache(unsigned Opcode) const;
> > bool usesVertexCache(const MachineInstr *MI) const;
> > bool usesTextureCache(unsigned Opcode) const;
> > --
> > 1.8.1.4
> >
> > From 428138a6c9ec4bb7d6483658acf5c2ab262bafb5 Mon Sep 17 00:00:00 2001
> > From: Vincent Lejeune <vljn at ovi.com>
> > Date: Sat, 20 Apr 2013 03:10:21 +0200
> > Subject: [PATCH 5/5] R600: use native for alu
> >
> > ---
> > lib/Target/R600/MCTargetDesc/R600MCCodeEmitter.cpp | 8 +-
> > lib/Target/R600/R600ControlFlowFinalizer.cpp | 109 ++++++++++++++++++++-
> > lib/Target/R600/R600Instructions.td | 17 ++++
> > lib/Target/R600/R600RegisterInfo.td | 5 +-
> > 4 files changed, 135 insertions(+), 4 deletions(-)
> >
> > diff --git a/lib/Target/R600/MCTargetDesc/R600MCCodeEmitter.cpp b/lib/Target/R600/MCTargetDesc/R600MCCodeEmitter.cpp
> > index 02ce566..caeab9a 100644
> > --- a/lib/Target/R600/MCTargetDesc/R600MCCodeEmitter.cpp
> > +++ b/lib/Target/R600/MCTargetDesc/R600MCCodeEmitter.cpp
> > @@ -143,6 +143,7 @@ void R600MCCodeEmitter::EncodeInstruction(const MCInst &MI, raw_ostream &OS,
> > EmitFCInstr(MI, OS);
> > } else if (MI.getOpcode() == AMDGPU::RETURN ||
> > MI.getOpcode() == AMDGPU::FETCH_CLAUSE ||
> > + MI.getOpcode() == AMDGPU::ALU_CLAUSE ||
> > MI.getOpcode() == AMDGPU::BUNDLE ||
> > MI.getOpcode() == AMDGPU::KILL) {
> > return;
> > @@ -254,7 +255,7 @@ void R600MCCodeEmitter::EncodeInstruction(const MCInst &MI, raw_ostream &OS,
> > case AMDGPU::CF_ALU:
> > case AMDGPU::CF_ALU_PUSH_BEFORE: {
> > uint64_t Inst = getBinaryCodeForInstr(MI, Fixups);
> > - EmitByte(INSTR_CFALU, OS);
> > + EmitByte(INSTR_NATIVE, OS);
> > Emit(Inst, OS);
> > break;
> > }
> > @@ -293,7 +294,10 @@ void R600MCCodeEmitter::EncodeInstruction(const MCInst &MI, raw_ostream &OS,
> > break;
> > }
> > default:
> > - EmitALUInstr(MI, Fixups, OS);
> > + uint64_t Inst = getBinaryCodeForInstr(MI, Fixups);
> > + EmitByte(INSTR_NATIVE, OS);
> > + Emit(Inst, OS);
> > +// EmitALUInstr(MI, Fixups, OS);
>
> Stray commented out code here.
>
> > break;
> > }
> > }
> > diff --git a/lib/Target/R600/R600ControlFlowFinalizer.cpp b/lib/Target/R600/R600ControlFlowFinalizer.cpp
> > index def2fa5..09acc6c 100644
> > --- a/lib/Target/R600/R600ControlFlowFinalizer.cpp
> > +++ b/lib/Target/R600/R600ControlFlowFinalizer.cpp
> > @@ -173,6 +173,96 @@ private:
> > return ClauseFile(MIb, ClauseContent);
> > }
> >
> > + void getLiteral(MachineInstr *MI, std::vector<unsigned> &Lits) const {
> > + for (unsigned i = 0, e = MI->getNumOperands(); i < e; ++i) {
> > + MachineOperand &MO = MI->getOperand(i);
> > + if (!MO.isReg())
> > + continue;
> > + if (MO.getReg() != AMDGPU::ALU_LITERAL_X)
> > + continue;
> > + switch (Lits.size()) {
> > + default: llvm_unreachable("Too many literals in Instruction Group");
> > + case 3:
> > + MO.setReg(AMDGPU::ALU_LITERAL_W);
> > + break;
> > + case 2:
> > + MO.setReg(AMDGPU::ALU_LITERAL_Z);
> > + break;
> > + case 1:
> > + MO.setReg(AMDGPU::ALU_LITERAL_Y);
> > + break;
> > + case 0:
> > + break;
> > + }
> > + unsigned ImmIdx = TII->getOperandIdx(MI->getOpcode(), R600Operands::IMM);
> > + Lits.push_back(MI->getOperand(ImmIdx).getImm());
> > + break;
> > + }
> > + }
> > +
> > + MachineBasicBlock::iterator insertLiterals(
> > + MachineBasicBlock::iterator InsertPos,
> > + const std::vector<unsigned> &Literals) const {
> > + MachineBasicBlock *MBB = InsertPos->getParent();
> > + for (unsigned i = 0, e = Literals.size(); i < e; i+=2) {
> > + unsigned LiteralPair0 = Literals[i];
> > + unsigned LiteralPair1 = (i + 1 < e)?Literals[i + 1]:0;
> > + InsertPos = BuildMI(MBB, InsertPos->getDebugLoc(),
> > + TII->get(AMDGPU::LITERALS))
> > + .addImm(LiteralPair0)
> > + .addImm(LiteralPair1);
> > + }
> > + return InsertPos;
> > + }
> > +
> > + ClauseFile
> > + MakeALUClause(MachineBasicBlock &MBB, MachineBasicBlock::iterator &I)
> > + const {
> > + MachineBasicBlock::iterator ClauseHead = I;
> > + std::vector<MachineInstr *> ClauseContent;
> > + I++;
> > + for (MachineBasicBlock::instr_iterator E = MBB.instr_end(); I != E;) {
> > + if (IsTrivialInst(I)) {
> > + ++I;
> > + continue;
> > + }
> > + if (!I->isBundle() && !TII->isALUInstr(I->getOpcode()))
> > + break;
> > + std::vector<unsigned> Literals;
> > + if (I->isBundle()) {
> > + MachineInstr *DeleteMI = I;
> > + MachineBasicBlock::instr_iterator BI = I.getInstrIterator();
> > + while (++BI != E && BI->isBundledWithPred()) {
> > + BI->unbundleFromPred();
> > + for (unsigned i = 0, e = BI->getNumOperands(); i != e; ++i) {
> > + MachineOperand &MO = BI->getOperand(i);
> > + if (MO.isReg() && MO.isInternalRead())
> > + MO.setIsInternalRead(false);
> > + }
> > + getLiteral(BI, Literals);
> > + ClauseContent.push_back(BI);
> > + }
> > + I = BI;
> > + DeleteMI->eraseFromParent();
> > + } else {
> > + getLiteral(I, Literals);
> > + ClauseContent.push_back(I);
> > + I++;
> > + }
> > + for (unsigned i = 0, e = Literals.size(); i < e; i+=2) {
> > + unsigned literal0 = Literals[i];
> > + unsigned literal2 = (i + 1 < e)?Literals[i + 1]:0;
> > + MachineInstr *MILit = BuildMI(MBB, I, I->getDebugLoc(),
> > + TII->get(AMDGPU::LITERALS))
> > + .addImm(literal0)
> > + .addImm(literal2);
> > + ClauseContent.push_back(MILit);
> > + }
> > + }
> > + ClauseHead->getOperand(7).setImm(ClauseContent.size() - 1);
> > + return ClauseFile(ClauseHead, ClauseContent);
> > + }
> > +
> > void
> > EmitFetchClause(MachineBasicBlock::iterator InsertPos, ClauseFile &Clause,
> > unsigned &CfCount) {
> > @@ -186,6 +276,19 @@ private:
> > CfCount += 2 * Clause.second.size();
> > }
> >
> > + void
> > + EmitALUClause(MachineBasicBlock::iterator InsertPos, ClauseFile &Clause,
> > + unsigned &CfCount) {
> > + CounterPropagateAddr(Clause.first, CfCount);
> > + MachineBasicBlock *BB = Clause.first->getParent();
> > + BuildMI(BB, InsertPos->getDebugLoc(), TII->get(AMDGPU::ALU_CLAUSE))
> > + .addImm(CfCount);
> > + for (unsigned i = 0, e = Clause.second.size(); i < e; ++i) {
> > + BB->splice(InsertPos, BB, Clause.second[i]);
> > + }
> > + CfCount += Clause.second.size();
> > + }
> > +
> > void CounterPropagateAddr(MachineInstr *MI, unsigned Addr) const {
> > MI->getOperand(0).setImm(Addr + MI->getOperand(0).getImm());
> > }
> > @@ -242,7 +345,7 @@ public:
> > getHWInstrDesc(CF_CALL_FS));
> > CfCount++;
> > }
> > - std::vector<ClauseFile> FetchClauses;
> > + std::vector<ClauseFile> FetchClauses, AluClauses;
> > for (MachineBasicBlock::iterator I = MBB.begin(), E = MBB.end();
> > I != E;) {
> > if (TII->usesTextureCache(I) || TII->usesVertexCache(I)) {
> > @@ -260,6 +363,8 @@ public:
> > MaxStack = std::max(MaxStack, CurrentStack);
> > hasPush = true;
> > case AMDGPU::CF_ALU:
> > + I = MI;
> > + AluClauses.push_back(MakeALUClause(MBB, I));
> > case AMDGPU::EG_ExportBuf:
> > case AMDGPU::EG_ExportSwz:
> > case AMDGPU::R600_ExportBuf:
> > @@ -373,6 +478,8 @@ public:
> > }
> > for (unsigned i = 0, e = FetchClauses.size(); i < e; i++)
> > EmitFetchClause(I, FetchClauses[i], CfCount);
> > + for (unsigned i = 0, e = AluClauses.size(); i < e; i++)
> > + EmitALUClause(I, AluClauses[i], CfCount);
> > }
> > default:
> > break;
> > diff --git a/lib/Target/R600/R600Instructions.td b/lib/Target/R600/R600Instructions.td
> > index 0b52dd1..7ef28c4 100644
> > --- a/lib/Target/R600/R600Instructions.td
> > +++ b/lib/Target/R600/R600Instructions.td
> > @@ -941,6 +941,23 @@ def FETCH_CLAUSE : AMDGPUInst <(outs),
> > let Inst = num;
> > }
> >
> > +def ALU_CLAUSE : AMDGPUInst <(outs),
> > +(ins i32imm:$addr), "ALU clause starting at $addr:", [] > {
> > + field bits<8> Inst;
> > + bits<8> num;
> > + let Inst = num;
> > +}
> > +
> > +def LITERALS : AMDGPUInst <(outs),
> > +(ins LITERAL:$literal1, LITERAL:$literal2), "$literal1, $literal2", [] > {
> > + field bits<64> Inst;
> > + bits<32> literal1;
> > + bits<32> literal2;
> > +
> > + let Inst{31-0} = literal1;
> > + let Inst{63-32} = literal2;
> > +}
> > +
> > def PAD : AMDGPUInst <(outs), (ins), "PAD", [] > {
> > field bits<64> Inst;
> > }
> > diff --git a/lib/Target/R600/R600RegisterInfo.td b/lib/Target/R600/R600RegisterInfo.td
> > index 03f4976..ec5838b 100644
> > --- a/lib/Target/R600/R600RegisterInfo.td
> > +++ b/lib/Target/R600/R600RegisterInfo.td
> > @@ -88,7 +88,10 @@ def NEG_ONE : R600Reg<"-1.0", 249>;
> > def ONE_INT : R600Reg<"1", 250>;
> > def HALF : R600Reg<"0.5", 252>;
> > def NEG_HALF : R600Reg<"-0.5", 252>;
> > -def ALU_LITERAL_X : R600Reg<"literal.x", 253>;
> > +def ALU_LITERAL_X : R600RegWithChan<"literal.x", 253, "X">;
> > +def ALU_LITERAL_Y : R600RegWithChan<"literal.x", 253, "Y">;
> > +def ALU_LITERAL_Z : R600RegWithChan<"literal.x", 253, "Z">;
> > +def ALU_LITERAL_W : R600RegWithChan<"literal.x", 253, "W">;
> > def PV_X : R600Reg<"pv.x", 254>;
> > def PREDICATE_BIT : R600Reg<"PredicateBit", 0>;
> > def PRED_SEL_OFF: R600Reg<"Pred_sel_off", 0>;
> > --
> > 1.8.1.4
> >
>
> > From 16e6d43a151e77c0507e363302ca7bb0a24f6aca Mon Sep 17 00:00:00 2001
> > From: Vincent Lejeune <vljn at ovi.com>
> > Date: Tue, 9 Apr 2013 15:10:06 +0200
> > Subject: [PATCH 4/5] R600: Packetize instructions
> >
> > ---
> > lib/Target/R600/AMDGPU.h | 1 +
> > lib/Target/R600/AMDGPUTargetMachine.cpp | 3 +-
> > lib/Target/R600/CMakeLists.txt | 1 +
> > lib/Target/R600/R600InstrInfo.cpp | 11 +-
> > lib/Target/R600/R600Packetizer.cpp | 365 ++++++++++++++++++++++++++++++++
> > 5 files changed, 379 insertions(+), 2 deletions(-)
> > create mode 100644 lib/Target/R600/R600Packetizer.cpp
> >
> > diff --git a/lib/Target/R600/AMDGPU.h b/lib/Target/R600/AMDGPU.h
> > index 0b01433..9792bd8 100644
> > --- a/lib/Target/R600/AMDGPU.h
> > +++ b/lib/Target/R600/AMDGPU.h
> > @@ -24,6 +24,7 @@ class AMDGPUTargetMachine;
> > FunctionPass* createR600KernelParametersPass(const DataLayout *TD);
> > FunctionPass *createR600ExpandSpecialInstrsPass(TargetMachine &tm);
> > FunctionPass *createR600EmitClauseMarkers(TargetMachine &tm);
> > +FunctionPass *createR600Packetizer(TargetMachine &tm);
> > FunctionPass *createR600ControlFlowFinalizer(TargetMachine &tm);
> >
> > // SI Passes
> > diff --git a/lib/Target/R600/AMDGPUTargetMachine.cpp b/lib/Target/R600/AMDGPUTargetMachine.cpp
> > index e7ea876..0ec67ce 100644
> > --- a/lib/Target/R600/AMDGPUTargetMachine.cpp
> > +++ b/lib/Target/R600/AMDGPUTargetMachine.cpp
> > @@ -153,8 +153,9 @@ bool AMDGPUPassConfig::addPreEmitPass() {
> > addPass(createAMDGPUCFGStructurizerPass(*TM));
> > addPass(createR600EmitClauseMarkers(*TM));
> > addPass(createR600ExpandSpecialInstrsPass(*TM));
> > - addPass(createR600ControlFlowFinalizer(*TM));
> > addPass(&FinalizeMachineBundlesID);
> > + addPass(createR600Packetizer(*TM));
> > + addPass(createR600ControlFlowFinalizer(*TM));
> > } else {
> > addPass(createSILowerControlFlowPass(*TM));
> > }
> > diff --git a/lib/Target/R600/CMakeLists.txt b/lib/Target/R600/CMakeLists.txt
> > index 8efba58..2ad2047 100644
> > --- a/lib/Target/R600/CMakeLists.txt
> > +++ b/lib/Target/R600/CMakeLists.txt
> > @@ -42,6 +42,7 @@ add_llvm_target(R600CodeGen
> > R600ISelLowering.cpp
> > R600MachineFunctionInfo.cpp
> > R600MachineScheduler.cpp
> > + R600Packetizer.cpp
> > R600RegisterInfo.cpp
> > SIAnnotateControlFlow.cpp
> > SIInsertWaits.cpp
> > diff --git a/lib/Target/R600/R600InstrInfo.cpp b/lib/Target/R600/R600InstrInfo.cpp
> > index 8e5ee7c..ecb2476 100644
> > --- a/lib/Target/R600/R600InstrInfo.cpp
> > +++ b/lib/Target/R600/R600InstrInfo.cpp
> > @@ -252,10 +252,19 @@ R600InstrInfo::canBundle(const std::vector<MachineInstr *> &MIs) const {
> > int SrcIdx = getOperandIdx(MI->getOpcode(), OpTable[j][0]);
> > if (SrcIdx < 0)
> > break;
> > - if (MI->getOperand(SrcIdx).getReg() == AMDGPU::ALU_CONST) {
> > + unsigned Reg = MI->getOperand(SrcIdx).getReg();
> > + if (Reg == AMDGPU::ALU_CONST) {
> > unsigned Const = MI->getOperand(
> > getOperandIdx(MI->getOpcode(), OpTable[j][1])).getImm();
> > Consts.push_back(Const);
> > + continue;
> > + }
> > + if (AMDGPU::R600_KC0RegClass.contains(Reg) ||
> > + AMDGPU::R600_KC1RegClass.contains(Reg)) {
> > + unsigned Index = RI.getEncodingValue(Reg) & 0xff;
> > + unsigned Chan = RI.getHWRegChan(Reg);
> > + Consts.push_back((Index << 2) | Chan);
> > + continue;
> > }
> > }
> > }
> > diff --git a/lib/Target/R600/R600Packetizer.cpp b/lib/Target/R600/R600Packetizer.cpp
> > new file mode 100644
> > index 0000000..a66b3ea
> > --- /dev/null
> > +++ b/lib/Target/R600/R600Packetizer.cpp
> > @@ -0,0 +1,365 @@
> > +//===----- HexagonPacketizer.cpp - vliw packetizer ---------------------===//
>
> Should be R600Packetizer.cpp
>
> > +//
> > +// The LLVM Compiler Infrastructure
> > +//
> > +// This file is distributed under the University of Illinois Open Source
> > +// License. See LICENSE.TXT for details.
> > +//
> > +//===----------------------------------------------------------------------===//
> > +//
> > +
> You need to add /// \file here for doxygen.
> > +//
> > +//===----------------------------------------------------------------------===//
> > +
> > +#ifndef R600PACKETIZER_CPP
> > +#define R600PACKETIZER_CPP
> > +
> > +#define DEBUG_TYPE "packets"
> > +#include "llvm/Support/Debug.h"
> > +#include "llvm/Support/raw_ostream.h"
> > +#include "llvm/CodeGen/DFAPacketizer.h"
> > +#include "llvm/CodeGen/Passes.h"
> > +#include "llvm/CodeGen/MachineFunctionPass.h"
> > +#include "llvm/CodeGen/MachineDominators.h"
> > +#include "llvm/CodeGen/MachineLoopInfo.h"
> > +#include "llvm/CodeGen/ScheduleDAG.h"
> > +#include "AMDGPU.h"
> > +#include "R600InstrInfo.h"
> > +
> > +namespace llvm {
> > +
> > +class R600Packetizer : public MachineFunctionPass {
> > +
> > +public:
> > + static char ID;
> > + R600Packetizer(const TargetMachine &TM) : MachineFunctionPass(ID) {}
> > +
> > + void getAnalysisUsage(AnalysisUsage &AU) const {
> > + AU.setPreservesCFG();
> > + AU.addRequired<MachineDominatorTree>();
> > + AU.addPreserved<MachineDominatorTree>();
> > + AU.addRequired<MachineLoopInfo>();
> > + AU.addPreserved<MachineLoopInfo>();
> > + MachineFunctionPass::getAnalysisUsage(AU);
> > + }
> > +
> > + const char *getPassName() const {
> > + return "R600 Packetizer";
> > + }
> > +
> > + bool runOnMachineFunction(MachineFunction &Fn);
> > +};
> > +char R600Packetizer::ID = 0;
> > +
> > +class R600PacketizerList : public VLIWPacketizerList {
> > +
> > +private:
> > + const R600InstrInfo *TII;
> > + const R600RegisterInfo &TRI;
> > +
> > + enum BankSwizzle {
> > + ALU_VEC_012 = 0,
> > + ALU_VEC_021,
> > + ALU_VEC_120,
> > + ALU_VEC_102,
> > + ALU_VEC_201,
> > + ALU_VEC_210
> > + };
> > +
> > + unsigned getSlot(const MachineInstr *MI) const {
> > + return TRI.getHWRegChan(MI->getOperand(0).getReg());
> > + }
> > +
> > +public:
> > + // Ctor.
> > + R600PacketizerList(MachineFunction &MF, MachineLoopInfo &MLI,
> > + MachineDominatorTree &MDT)
> > + : VLIWPacketizerList(MF, MLI, MDT, true),
> > + TII (static_cast<const R600InstrInfo *>(MF.getTarget().getInstrInfo())),
> > + TRI(TII->getRegisterInfo()) { }
> > +
> > + // initPacketizerState - initialize some internal flags.
> > + void initPacketizerState() { }
> > +
> > + // ignorePseudoInstruction - Ignore bundling of pseudo instructions.
> > + bool ignorePseudoInstruction(MachineInstr *MI, MachineBasicBlock *MBB) {
> > + return false;
> > + }
> > +
> > + // isSoloInstruction - return true if instruction MI can not be packetized
> > + // with any other instruction, which means that MI itself is a packet.
> > + bool isSoloInstruction(MachineInstr *MI) {
> > + if (TII->isVector(*MI))
> > + return true;
> > + if (!TII->isALUInstr(MI->getOpcode()))
> > + return true;
> > + if (TII->get(MI->getOpcode()).TSFlags & R600_InstFlag::TRANS_ONLY)
> > + return true;
> > + if (TII->isTransOnly(MI))
> > + return true;
> > + return false;
> > + }
> > +
> > + // isLegalToPacketizeTogether - Is it legal to packetize SUI and SUJ
> > + // together.
> > + bool isLegalToPacketizeTogether(SUnit *SUI, SUnit *SUJ) {
> > + if (getSlot(SUI->getInstr()) <= getSlot(SUJ->getInstr()))
> > + return false;
> > + if (SUJ->isSucc(SUI)) {
> > + for (unsigned i = 0, e = SUJ->Succs.size(); i < e; ++i) {
> > + if (SUJ->Succs[i].getSUnit() != SUI)
> > + continue;
> > + if (SUJ->Succs[i].getKind() != SDep::Anti)
> > + return false;
> > + }
> > + }
> > + return true;
> > + }
> > +
> > + // isLegalToPruneDependencies - Is it legal to prune dependece between SUI
> > + // and SUJ.
> > + bool isLegalToPruneDependencies(SUnit *SUI, SUnit *SUJ) {return false;}
> > +
> > + void setIsLastBit(MachineInstr *MI, unsigned Bit) const {
> > + unsigned LastOp = TII->getOperandIdx(MI->getOpcode(), R600Operands::LAST);
> > + MI->getOperand(LastOp).setImm(Bit);
> > + }
> > +
> > + MachineBasicBlock::iterator addToPacket(MachineInstr *MI) {
> > + CurrentPacketMIs.push_back(MI);
> > + bool FitsConstLimits = TII->canBundle(CurrentPacketMIs);
> > + DEBUG(
> > + if (!FitsConstLimits) {
> > + dbgs() << "Couldn't pack :\n";
> > + MI->dump();
> > + dbgs() << "with the following packets :\n";
> > + for (unsigned i = 0, e = CurrentPacketMIs.size() - 1; i < e; i++) {
> > + CurrentPacketMIs[i]->dump();
> > + dbgs() << "\n";
> > + }
> > + dbgs() << "because of Consts read limitations\n";
> > + });
> > + bool FitsReadPortLimits = fitsReadPortLimitation(CurrentPacketMIs);
> > + DEBUG(
> > + if (!FitsReadPortLimits) {
> > + dbgs() << "Couldn't pack :\n";
> > + MI->dump();
> > + dbgs() << "with the following packets :\n";
> > + for (unsigned i = 0, e = CurrentPacketMIs.size() - 1; i < e; i++) {
> > + CurrentPacketMIs[i]->dump();
> > + dbgs() << "\n";
> > + }
> > + dbgs() << "because of Read port limitations\n";
> > + });
> > + bool isBundlable = FitsConstLimits && FitsReadPortLimits;
> > + CurrentPacketMIs.pop_back();
> > + if (!isBundlable) {
> > + endPacket(MI->getParent(), MI);
> > + return VLIWPacketizerList::addToPacket(MI);
> > + }
> > + if (!CurrentPacketMIs.empty())
> > + setIsLastBit(CurrentPacketMIs.back(), 0);
> > + return VLIWPacketizerList::addToPacket(MI);
> > + }
> > +private:
> > + std::vector<std::pair<int, unsigned> >
> > + ExtractSrcs(const MachineInstr *MI) const {
> > + R600Operands::Ops Ops[] = {
> > + R600Operands::SRC0,
> > + R600Operands::SRC1,
> > + R600Operands::SRC2
> > + };
> > + std::vector<std::pair<int, unsigned> > Result;
> > + for (unsigned i = 0; i < 3; i++) {
> > + int OperandIdx = TII->getOperandIdx(MI->getOpcode(), Ops[i]);
> > + if (OperandIdx < 0){
> > + Result.push_back(std::pair<int, unsigned>(-1,0));
> > + continue;
> > + }
> > + unsigned Src = MI->getOperand(OperandIdx).getReg();
> > + unsigned Reg = TRI.getEncodingValue(Src) & 0xff;
> > + if (Reg > 127) {
> > + Result.push_back(std::pair<int, unsigned>(-1,0));
> > + continue;
> > + }
> > + unsigned Chan = TRI.getHWRegChan(Src);
> > + Result.push_back(std::pair<int, unsigned>(Reg, Chan));
> > + }
> > + return Result;
> > + }
> > +
> > + std::vector<std::pair<int, unsigned> >
> > + Swizzle(std::vector<std::pair<int, unsigned> > Src,
> > + BankSwizzle Swz) const {
> > + switch (Swz) {
> > + case ALU_VEC_012:
> > + break;
> > + case ALU_VEC_021:
> > + std::swap(Src[1], Src[2]);
> > + break;
> > + case ALU_VEC_102:
> > + std::swap(Src[0], Src[1]);
> > + break;
> > + case ALU_VEC_120:
> > + std::swap(Src[0], Src[1]);
> > + std::swap(Src[0], Src[2]);
> > + break;
> > + case ALU_VEC_201:
> > + std::swap(Src[0], Src[2]);
> > + std::swap(Src[0], Src[1]);
> > + break;
> > + case ALU_VEC_210:
> > + std::swap(Src[0], Src[2]);
> > + break;
> > + }
> > + return Src;
> > + }
> > +
> > + bool isLegal(const std::vector<MachineInstr *> &IG,
> > + const std::vector<BankSwizzle> &Swz) const {
> > + assert (Swz.size() == IG.size());
> > + int Vector[4][3];
> > + memset(Vector, -1, sizeof(Vector));
> > + for (unsigned i = 0, e = IG.size(); i < e; i++) {
> > + const std::vector<std::pair<int, unsigned> > &Srcs =
> > + Swizzle(ExtractSrcs(IG[i]), Swz[i]);
> > + for (unsigned j = 0; j < 3; j++) {
> > + const std::pair<int, unsigned> &Src = Srcs[j];
> > + if (Src.first < 0)
> > + continue;
> > + if (Vector[Src.second][j] < 0)
> > + Vector[Src.second][j] = Src.first;
> > + if (Vector[Src.second][j] != Src.first)
> > + return false;
> > + }
> > + }
> > + return true;
> > + }
> > +
> > + bool recursiveFitsFPLimitation(
> > + std::vector<MachineInstr *> IG,
> > + std::vector<BankSwizzle> &SwzCandidate,
> > + std::vector<MachineInstr *> CurrentlyChecked)
> > + const {
> > + if (!isLegal(CurrentlyChecked, SwzCandidate))
> > + return false;
> > + if (IG.size() == CurrentlyChecked.size()) {
> > + return true;
> > + }
> > + BankSwizzle AvailableSwizzle[] = {
> > + ALU_VEC_012,
> > + ALU_VEC_021,
> > + ALU_VEC_120,
> > + ALU_VEC_102,
> > + ALU_VEC_201,
> > + ALU_VEC_210
> > + };
> > + CurrentlyChecked.push_back(IG[CurrentlyChecked.size()]);
> > + for (unsigned i = 0; i < 6; i++) {
> > + SwzCandidate.push_back(AvailableSwizzle[i]);
> > + if (recursiveFitsFPLimitation(IG, SwzCandidate, CurrentlyChecked))
> > + return true;
> > + SwzCandidate.pop_back();
> > + }
> > + return false;
> > + }
> > +
> > + bool fitsReadPortLimitation(
> > + std::vector<MachineInstr *> IG)
> > + const {
> > + std::vector<BankSwizzle> SwzCandidate;
> > + bool Result = recursiveFitsFPLimitation(IG, SwzCandidate,
> > + std::vector<MachineInstr *>());
> > + if (!Result)
> > + return false;
> > + for (unsigned i = 0, e = IG.size(); i < e; i++) {
> > + MachineInstr *MI = IG[i];
> > + unsigned Op = TII->getOperandIdx(MI->getOpcode(),
> > + R600Operands::BANK_SWIZZLE);
> > + MI->getOperand(Op).setImm(SwzCandidate[i]);
> > + }
> > + return true;
> > + }
> > +};
> > +
> > +bool R600Packetizer::runOnMachineFunction(MachineFunction &Fn) {
> > + const TargetInstrInfo *TII = Fn.getTarget().getInstrInfo();
> > + MachineLoopInfo &MLI = getAnalysis<MachineLoopInfo>();
> > + MachineDominatorTree &MDT = getAnalysis<MachineDominatorTree>();
> > +
> > + // Instantiate the packetizer.
> > + R600PacketizerList Packetizer(Fn, MLI, MDT);
> > +
> > + // DFA state table should not be empty.
> > + assert(Packetizer.getResourceTracker() && "Empty DFA table!");
> > +
> > + //
> > + // Loop over all basic blocks and remove KILL pseudo-instructions
> > + // These instructions confuse the dependence analysis. Consider:
> > + // D0 = ... (Insn 0)
> > + // R0 = KILL R0, D0 (Insn 1)
> > + // R0 = ... (Insn 2)
> > + // Here, Insn 1 will result in the dependence graph not emitting an output
> > + // dependence between Insn 0 and Insn 2. This can lead to incorrect
> > + // packetization
> > + //
> > + for (MachineFunction::iterator MBB = Fn.begin(), MBBe = Fn.end();
> > + MBB != MBBe; ++MBB) {
> > + MachineBasicBlock::iterator End = MBB->end();
> > + MachineBasicBlock::iterator MI = MBB->begin();
> > + while (MI != End) {
> > + if (MI->isKill()) {
> > + MachineBasicBlock::iterator DeleteMI = MI;
> > + ++MI;
> > + MBB->erase(DeleteMI);
> > + End = MBB->end();
> > + continue;
> > + }
> > + ++MI;
> > + }
> > + }
> > +
> > + // Loop over all of the basic blocks.
> > + for (MachineFunction::iterator MBB = Fn.begin(), MBBe = Fn.end();
> > + MBB != MBBe; ++MBB) {
> > + // Find scheduling regions and schedule / packetize each region.
> > + unsigned RemainingCount = MBB->size();
> > + for(MachineBasicBlock::iterator RegionEnd = MBB->end();
> > + RegionEnd != MBB->begin();) {
> > + // The next region starts above the previous region. Look backward in the
> > + // instruction stream until we find the nearest boundary.
> > + MachineBasicBlock::iterator I = RegionEnd;
> > + for(;I != MBB->begin(); --I, --RemainingCount) {
> > + if (TII->isSchedulingBoundary(llvm::prior(I), MBB, Fn))
> > + break;
> > + }
> > + I = MBB->begin();
> > +
> > + // Skip empty scheduling regions.
> > + if (I == RegionEnd) {
> > + RegionEnd = llvm::prior(RegionEnd);
> > + --RemainingCount;
> > + continue;
> > + }
> > + // Skip regions with one instruction.
> > + if (I == llvm::prior(RegionEnd)) {
> > + RegionEnd = llvm::prior(RegionEnd);
> > + continue;
> > + }
> > +
> > + Packetizer.PacketizeMIs(MBB, I, RegionEnd);
> > + RegionEnd = I;
> > + }
> > + }
> > +
> > + return true;
> > +
> > +}
> > +
> > +}
> > +
> > +llvm::FunctionPass *llvm::createR600Packetizer(TargetMachine &tm) {
> > + return new R600Packetizer(tm);
> > +}
> > +
> > +#endif // R600PACKETIZER_CPP
> > --
> > 1.8.1.4
> >
>
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list