<div dir="ltr">sorry for the delay - looking now.</div><br><div class="gmail_quote"><div dir="ltr">On Wed, Dec 5, 2018 at 3:19 AM Greg Bedwell <<a href="mailto:gregbedwell@gmail.com">gregbedwell@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>We're seeing a new assertion failure (<span style="color:rgb(0,0,0);white-space:pre-wrap">X86DiscriminateMemOps.cpp:119: </span><span style="color:rgb(0,0,0);white-space:pre-wrap">`MustInsert.second && "New discriminator shouldn't be present in set"') since this commit. I've put a testcase in </span><a href="https://bugs.llvm.org/show_bug.cgi?id=39890" target="_blank">https://bugs.llvm.org/show_bug.cgi?id=39890</a> . Please could you take a look?</div><div><br></div><div>Thanks!</div><div>-Greg</div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, 30 Nov 2018 at 01:04, Mircea Trofin via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Author: mtrofin<br>
Date: Thu Nov 29 17:01:52 2018<br>
New Revision: 347938<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=347938&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=347938&view=rev</a><br>
Log:<br>
Revert "Revert r347596 "Support for inserting profile-directed cache prefetches""<br>
<br>
Summary:<br>
This reverts commit d8517b96dfbd42e6a8db33c50d1fa1e58e63fbb9.<br>
<br>
Fix: correct the use of DenseMap.<br>
<br>
Reviewers: davidxl, hans, wmi<br>
<br>
Reviewed By: wmi<br>
<br>
Subscribers: mgorny, eraman, llvm-commits<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D55088" rel="noreferrer" target="_blank">https://reviews.llvm.org/D55088</a><br>
<br>
Added:<br>
llvm/trunk/lib/Target/X86/X86DiscriminateMemOps.cpp<br>
llvm/trunk/lib/Target/X86/X86InsertPrefetch.cpp<br>
llvm/trunk/test/CodeGen/X86/discriminate-mem-ops.ll<br>
llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.afdo<br>
llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.ll<br>
llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.afdo<br>
llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.ll<br>
llvm/trunk/test/CodeGen/X86/insert-prefetch-other.afdo<br>
llvm/trunk/test/CodeGen/X86/insert-prefetch.afdo<br>
llvm/trunk/test/CodeGen/X86/insert-prefetch.ll<br>
Modified:<br>
llvm/trunk/lib/Target/X86/CMakeLists.txt<br>
llvm/trunk/lib/Target/X86/LLVMBuild.txt<br>
llvm/trunk/lib/Target/X86/X86.h<br>
llvm/trunk/lib/Target/X86/X86TargetMachine.cpp<br>
llvm/trunk/test/CodeGen/X86/O0-pipeline.ll<br>
llvm/trunk/test/CodeGen/X86/O3-pipeline.ll<br>
<br>
Modified: llvm/trunk/lib/Target/X86/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/CMakeLists.txt?rev=347938&r1=347937&r2=347938&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/CMakeLists.txt?rev=347938&r1=347937&r2=347938&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Target/X86/CMakeLists.txt (original)<br>
+++ llvm/trunk/lib/Target/X86/CMakeLists.txt Thu Nov 29 17:01:52 2018<br>
@@ -30,6 +30,7 @@ set(sources<br>
X86CmovConversion.cpp<br>
X86CondBrFolding.cpp<br>
X86DomainReassignment.cpp<br>
+ X86DiscriminateMemOps.cpp<br>
X86ExpandPseudo.cpp<br>
X86FastISel.cpp<br>
X86FixupBWInsts.cpp<br>
@@ -44,6 +45,7 @@ set(sources<br>
X86ISelLowering.cpp<br>
X86IndirectBranchTracking.cpp<br>
X86InterleavedAccess.cpp<br>
+ X86InsertPrefetch.cpp<br>
X86InstrFMA3Info.cpp<br>
X86InstrFoldTables.cpp<br>
X86InstrInfo.cpp<br>
<br>
Modified: llvm/trunk/lib/Target/X86/LLVMBuild.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/LLVMBuild.txt?rev=347938&r1=347937&r2=347938&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/LLVMBuild.txt?rev=347938&r1=347937&r2=347938&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Target/X86/LLVMBuild.txt (original)<br>
+++ llvm/trunk/lib/Target/X86/LLVMBuild.txt Thu Nov 29 17:01:52 2018<br>
@@ -31,5 +31,5 @@ has_jit = 1<br>
type = Library<br>
name = X86CodeGen<br>
parent = X86<br>
-required_libraries = Analysis AsmPrinter CodeGen Core MC SelectionDAG Support Target X86AsmPrinter X86Desc X86Info X86Utils GlobalISel<br>
+required_libraries = Analysis AsmPrinter CodeGen Core MC SelectionDAG Support Target X86AsmPrinter X86Desc X86Info X86Utils GlobalISel ProfileData<br>
add_to_library_groups = X86<br>
<br>
Modified: llvm/trunk/lib/Target/X86/X86.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86.h?rev=347938&r1=347937&r2=347938&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86.h?rev=347938&r1=347937&r2=347938&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Target/X86/X86.h (original)<br>
+++ llvm/trunk/lib/Target/X86/X86.h Thu Nov 29 17:01:52 2018<br>
@@ -122,6 +122,13 @@ FunctionPass *createX86EvexToVexInsts();<br>
/// This pass creates the thunks for the retpoline feature.<br>
FunctionPass *createX86RetpolineThunksPass();<br>
<br>
+/// This pass ensures instructions featuring a memory operand<br>
+/// have distinctive <LineNumber, Discriminator> (with respect to eachother)<br>
+FunctionPass *createX86DiscriminateMemOpsPass();<br>
+<br>
+/// This pass applies profiling information to insert cache prefetches.<br>
+FunctionPass *createX86InsertPrefetchPass();<br>
+<br>
InstructionSelector *createX86InstructionSelector(const X86TargetMachine &TM,<br>
X86Subtarget &,<br>
X86RegisterBankInfo &);<br>
<br>
Added: llvm/trunk/lib/Target/X86/X86DiscriminateMemOps.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86DiscriminateMemOps.cpp?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86DiscriminateMemOps.cpp?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Target/X86/X86DiscriminateMemOps.cpp (added)<br>
+++ llvm/trunk/lib/Target/X86/X86DiscriminateMemOps.cpp Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,132 @@<br>
+//===- X86DiscriminateMemOps.cpp - Unique IDs for Mem Ops -----------------===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// This pass aids profile-driven cache prefetch insertion by ensuring all<br>
+/// instructions that have a memory operand are distinguishible from each other.<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "X86.h"<br>
+#include "X86InstrBuilder.h"<br>
+#include "X86InstrInfo.h"<br>
+#include "X86MachineFunctionInfo.h"<br>
+#include "X86Subtarget.h"<br>
+#include "llvm/CodeGen/MachineModuleInfo.h"<br>
+#include "llvm/IR/DebugInfoMetadata.h"<br>
+#include "llvm/ProfileData/SampleProf.h"<br>
+#include "llvm/ProfileData/SampleProfReader.h"<br>
+#include "llvm/Transforms/IPO/SampleProfile.h"<br>
+using namespace llvm;<br>
+<br>
+namespace {<br>
+<br>
+using Location = std::pair<StringRef, unsigned>;<br>
+<br>
+Location diToLocation(const DILocation *Loc) {<br>
+ return std::make_pair(Loc->getFilename(), Loc->getLine());<br>
+}<br>
+<br>
+/// Ensure each instruction having a memory operand has a distinct <LineNumber,<br>
+/// Discriminator> pair.<br>
+void updateDebugInfo(MachineInstr *MI, const DILocation *Loc) {<br>
+ DebugLoc DL(Loc);<br>
+ MI->setDebugLoc(DL);<br>
+}<br>
+<br>
+class X86DiscriminateMemOps : public MachineFunctionPass {<br>
+ bool runOnMachineFunction(MachineFunction &MF) override;<br>
+ StringRef getPassName() const override {<br>
+ return "X86 Discriminate Memory Operands";<br>
+ }<br>
+<br>
+public:<br>
+ static char ID;<br>
+<br>
+ /// Default construct and initialize the pass.<br>
+ X86DiscriminateMemOps();<br>
+};<br>
+<br>
+} // end anonymous namespace<br>
+<br>
+//===----------------------------------------------------------------------===//<br>
+// Implementation<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+char X86DiscriminateMemOps::ID = 0;<br>
+<br>
+/// Default construct and initialize the pass.<br>
+X86DiscriminateMemOps::X86DiscriminateMemOps() : MachineFunctionPass(ID) {}<br>
+<br>
+bool X86DiscriminateMemOps::runOnMachineFunction(MachineFunction &MF) {<br>
+ DISubprogram *FDI = MF.getFunction().getSubprogram();<br>
+ if (!FDI || !FDI->getUnit()->getDebugInfoForProfiling())<br>
+ return false;<br>
+<br>
+ // Have a default DILocation, if we find instructions with memops that don't<br>
+ // have any debug info.<br>
+ const DILocation *ReferenceDI =<br>
+ DILocation::get(FDI->getContext(), FDI->getLine(), 0, FDI);<br>
+<br>
+ DenseMap<Location, unsigned> MemOpDiscriminators;<br>
+ MemOpDiscriminators[diToLocation(ReferenceDI)] = 0;<br>
+<br>
+ // Figure out the largest discriminator issued for each Location. When we<br>
+ // issue new discriminators, we can thus avoid issuing discriminators<br>
+ // belonging to instructions that don't have memops. This isn't a requirement<br>
+ // for the goals of this pass, however, it avoids unnecessary ambiguity.<br>
+ for (auto &MBB : MF) {<br>
+ for (auto &MI : MBB) {<br>
+ const auto &DI = MI.getDebugLoc();<br>
+ if (!DI)<br>
+ continue;<br>
+ Location Loc = diToLocation(DI);<br>
+ MemOpDiscriminators[Loc] =<br>
+ std::max(MemOpDiscriminators[Loc], DI->getBaseDiscriminator());<br>
+ }<br>
+ }<br>
+<br>
+ // Keep track of the discriminators seen at each Location. If an instruction's<br>
+ // DebugInfo has a Location and discriminator we've already seen, replace its<br>
+ // discriminator with a new one, to guarantee uniqueness.<br>
+ DenseMap<Location, DenseSet<unsigned>> Seen;<br>
+<br>
+ bool Changed = false;<br>
+ for (auto &MBB : MF) {<br>
+ for (auto &MI : MBB) {<br>
+ if (X86II::getMemoryOperandNo(MI.getDesc().TSFlags) < 0)<br>
+ continue;<br>
+ const DILocation *DI = MI.getDebugLoc();<br>
+ if (!DI) {<br>
+ DI = ReferenceDI;<br>
+ }<br>
+ DenseSet<unsigned> &Set = Seen[diToLocation(DI)];<br>
+ const std::pair<DenseSet<unsigned>::iterator, bool> TryInsert =<br>
+ Set.insert(DI->getBaseDiscriminator());<br>
+ if (!TryInsert.second) {<br>
+ DI = DI->setBaseDiscriminator(++MemOpDiscriminators[diToLocation(DI)]);<br>
+ updateDebugInfo(&MI, DI);<br>
+ Changed = true;<br>
+ const std::pair<DenseSet<unsigned>::iterator, bool> MustInsert =<br>
+ Set.insert(DI->getBaseDiscriminator());<br>
+ assert(MustInsert.second);<br>
+ }<br>
+<br>
+ // Bump the reference DI to avoid cramming discriminators on line 0.<br>
+ // FIXME(mtrofin): pin ReferenceDI on blocks or first instruction with DI<br>
+ // in a block. It's more consistent than just relying on the last memop<br>
+ // instruction we happened to see.<br>
+ ReferenceDI = DI;<br>
+ }<br>
+ }<br>
+ return Changed;<br>
+}<br>
+<br>
+FunctionPass *llvm::createX86DiscriminateMemOpsPass() {<br>
+ return new X86DiscriminateMemOps();<br>
+}<br>
<br>
Added: llvm/trunk/lib/Target/X86/X86InsertPrefetch.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86InsertPrefetch.cpp?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86InsertPrefetch.cpp?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Target/X86/X86InsertPrefetch.cpp (added)<br>
+++ llvm/trunk/lib/Target/X86/X86InsertPrefetch.cpp Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,253 @@<br>
+//===------- X86InsertPrefetch.cpp - Insert cache prefetch hints ----------===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// This pass applies cache prefetch instructions based on a profile. The pass<br>
+// assumes DiscriminateMemOps ran immediately before, to ensure debug info<br>
+// matches the one used at profile generation time. The profile is encoded in<br>
+// afdo format (text or binary). It contains prefetch hints recommendations.<br>
+// Each recommendation is made in terms of debug info locations, a type (i.e.<br>
+// nta, t{0|1|2}) and a delta. The debug info identifies an instruction with a<br>
+// memory operand (see X86DiscriminateMemOps). The prefetch will be made for<br>
+// a location at that memory operand + the delta specified in the<br>
+// recommendation.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "X86.h"<br>
+#include "X86InstrBuilder.h"<br>
+#include "X86InstrInfo.h"<br>
+#include "X86MachineFunctionInfo.h"<br>
+#include "X86Subtarget.h"<br>
+#include "llvm/CodeGen/MachineModuleInfo.h"<br>
+#include "llvm/IR/DebugInfoMetadata.h"<br>
+#include "llvm/ProfileData/SampleProf.h"<br>
+#include "llvm/ProfileData/SampleProfReader.h"<br>
+#include "llvm/Transforms/IPO/SampleProfile.h"<br>
+using namespace llvm;<br>
+using namespace sampleprof;<br>
+<br>
+static cl::opt<std::string><br>
+ PrefetchHintsFile("prefetch-hints-file",<br>
+ cl::desc("Path to the prefetch hints profile."),<br>
+ cl::Hidden);<br>
+namespace {<br>
+<br>
+class X86InsertPrefetch : public MachineFunctionPass {<br>
+ void getAnalysisUsage(AnalysisUsage &AU) const override;<br>
+ bool doInitialization(Module &) override;<br>
+<br>
+ bool runOnMachineFunction(MachineFunction &MF) override;<br>
+ struct PrefetchInfo {<br>
+ unsigned InstructionID;<br>
+ int64_t Delta;<br>
+ };<br>
+ typedef SmallVectorImpl<PrefetchInfo> Prefetches;<br>
+ bool findPrefetchInfo(const FunctionSamples *Samples, const MachineInstr &MI,<br>
+ Prefetches &prefetches) const;<br>
+<br>
+public:<br>
+ static char ID;<br>
+ X86InsertPrefetch(const std::string &PrefetchHintsFilename);<br>
+ StringRef getPassName() const override {<br>
+ return "X86 Insert Cache Prefetches";<br>
+ }<br>
+<br>
+private:<br>
+ std::string Filename;<br>
+ std::unique_ptr<SampleProfileReader> Reader;<br>
+};<br>
+<br>
+using PrefetchHints = SampleRecord::CallTargetMap;<br>
+<br>
+// Return any prefetching hints for the specified MachineInstruction. The hints<br>
+// are returned as pairs (name, delta).<br>
+ErrorOr<PrefetchHints> getPrefetchHints(const FunctionSamples *TopSamples,<br>
+ const MachineInstr &MI) {<br>
+ if (const auto &Loc = MI.getDebugLoc())<br>
+ if (const auto *Samples = TopSamples->findFunctionSamples(Loc))<br>
+ return Samples->findCallTargetMapAt(FunctionSamples::getOffset(Loc),<br>
+ Loc->getBaseDiscriminator());<br>
+ return std::error_code();<br>
+}<br>
+<br>
+// The prefetch instruction can't take memory operands involving vector<br>
+// registers.<br>
+bool IsMemOpCompatibleWithPrefetch(const MachineInstr &MI, int Op) {<br>
+ unsigned BaseReg = MI.getOperand(Op + X86::AddrBaseReg).getReg();<br>
+ unsigned IndexReg = MI.getOperand(Op + X86::AddrIndexReg).getReg();<br>
+ return (BaseReg == 0 ||<br>
+ X86MCRegisterClasses[X86::GR64RegClassID].contains(BaseReg) ||<br>
+ X86MCRegisterClasses[X86::GR32RegClassID].contains(BaseReg)) &&<br>
+ (IndexReg == 0 ||<br>
+ X86MCRegisterClasses[X86::GR64RegClassID].contains(IndexReg) ||<br>
+ X86MCRegisterClasses[X86::GR32RegClassID].contains(IndexReg));<br>
+}<br>
+<br>
+} // end anonymous namespace<br>
+<br>
+//===----------------------------------------------------------------------===//<br>
+// Implementation<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+char X86InsertPrefetch::ID = 0;<br>
+<br>
+X86InsertPrefetch::X86InsertPrefetch(const std::string &PrefetchHintsFilename)<br>
+ : MachineFunctionPass(ID), Filename(PrefetchHintsFilename) {}<br>
+<br>
+/// Return true if the provided MachineInstruction has cache prefetch hints. In<br>
+/// that case, the prefetch hints are stored, in order, in the Prefetches<br>
+/// vector.<br>
+bool X86InsertPrefetch::findPrefetchInfo(const FunctionSamples *TopSamples,<br>
+ const MachineInstr &MI,<br>
+ Prefetches &Prefetches) const {<br>
+ assert(Prefetches.empty() &&<br>
+ "Expected caller passed empty PrefetchInfo vector.");<br>
+ static const std::pair<const StringRef, unsigned> HintTypes[] = {<br>
+ {"_nta_", X86::PREFETCHNTA},<br>
+ {"_t0_", X86::PREFETCHT0},<br>
+ {"_t1_", X86::PREFETCHT1},<br>
+ {"_t2_", X86::PREFETCHT2},<br>
+ };<br>
+ static const char *SerializedPrefetchPrefix = "__prefetch";<br>
+<br>
+ const ErrorOr<PrefetchHints> T = getPrefetchHints(TopSamples, MI);<br>
+ if (!T)<br>
+ return false;<br>
+ int16_t max_index = -1;<br>
+ // Convert serialized prefetch hints into PrefetchInfo objects, and populate<br>
+ // the Prefetches vector.<br>
+ for (const auto &S_V : *T) {<br>
+ StringRef Name = S_V.getKey();<br>
+ if (Name.consume_front(SerializedPrefetchPrefix)) {<br>
+ int64_t D = static_cast<int64_t>(S_V.second);<br>
+ unsigned IID = 0;<br>
+ for (const auto &HintType : HintTypes) {<br>
+ if (Name.startswith(HintType.first)) {<br>
+ Name = Name.drop_front(HintType.first.size());<br>
+ IID = HintType.second;<br>
+ break;<br>
+ }<br>
+ }<br>
+ if (IID == 0)<br>
+ return false;<br>
+ uint8_t index = 0;<br>
+ Name.consumeInteger(10, index);<br>
+<br>
+ if (index >= Prefetches.size())<br>
+ Prefetches.resize(index + 1);<br>
+ Prefetches[index] = {IID, D};<br>
+ max_index = std::max(max_index, static_cast<int16_t>(index));<br>
+ }<br>
+ }<br>
+ assert(max_index + 1 >= 0 &&<br>
+ "Possible overflow: max_index + 1 should be positive.");<br>
+ assert(static_cast<size_t>(max_index + 1) == Prefetches.size() &&<br>
+ "The number of prefetch hints received should match the number of "<br>
+ "PrefetchInfo objects returned");<br>
+ return !Prefetches.empty();<br>
+}<br>
+<br>
+bool X86InsertPrefetch::doInitialization(Module &M) {<br>
+ if (Filename.empty())<br>
+ return false;<br>
+<br>
+ LLVMContext &Ctx = M.getContext();<br>
+ ErrorOr<std::unique_ptr<SampleProfileReader>> ReaderOrErr =<br>
+ SampleProfileReader::create(Filename, Ctx);<br>
+ if (std::error_code EC = ReaderOrErr.getError()) {<br>
+ std::string Msg = "Could not open profile: " + EC.message();<br>
+ Ctx.diagnose(DiagnosticInfoSampleProfile(Filename, Msg,<br>
+ DiagnosticSeverity::DS_Warning));<br>
+ return false;<br>
+ }<br>
+ Reader = std::move(ReaderOrErr.get());<br>
+ Reader->read();<br>
+ return true;<br>
+}<br>
+<br>
+void X86InsertPrefetch::getAnalysisUsage(AnalysisUsage &AU) const {<br>
+ AU.setPreservesAll();<br>
+ AU.addRequired<MachineModuleInfo>();<br>
+}<br>
+<br>
+bool X86InsertPrefetch::runOnMachineFunction(MachineFunction &MF) {<br>
+ if (!Reader)<br>
+ return false;<br>
+ const FunctionSamples *Samples = Reader->getSamplesFor(MF.getFunction());<br>
+ if (!Samples)<br>
+ return false;<br>
+<br>
+ bool Changed = false;<br>
+<br>
+ const TargetInstrInfo *TII = MF.getSubtarget().getInstrInfo();<br>
+ SmallVector<PrefetchInfo, 4> Prefetches;<br>
+ for (auto &MBB : MF) {<br>
+ for (auto MI = MBB.instr_begin(); MI != MBB.instr_end();) {<br>
+ auto Current = MI;<br>
+ ++MI;<br>
+<br>
+ int Offset = X86II::getMemoryOperandNo(Current->getDesc().TSFlags);<br>
+ if (Offset < 0)<br>
+ continue;<br>
+ unsigned Bias = X86II::getOperandBias(Current->getDesc());<br>
+ int MemOpOffset = Offset + Bias;<br>
+ // FIXME(mtrofin): ORE message when the recommendation cannot be taken.<br>
+ if (!IsMemOpCompatibleWithPrefetch(*Current, MemOpOffset))<br>
+ continue;<br>
+ Prefetches.clear();<br>
+ if (!findPrefetchInfo(Samples, *Current, Prefetches))<br>
+ continue;<br>
+ assert(!Prefetches.empty() &&<br>
+ "The Prefetches vector should contain at least a value if "<br>
+ "findPrefetchInfo returned true.");<br>
+ for (auto &PrefInfo : Prefetches) {<br>
+ unsigned PFetchInstrID = PrefInfo.InstructionID;<br>
+ int64_t Delta = PrefInfo.Delta;<br>
+ const MCInstrDesc &Desc = TII->get(PFetchInstrID);<br>
+ MachineInstr *PFetch =<br>
+ MF.CreateMachineInstr(Desc, Current->getDebugLoc(), true);<br>
+ MachineInstrBuilder MIB(MF, PFetch);<br>
+<br>
+ assert(X86::AddrBaseReg == 0 && X86::AddrScaleAmt == 1 &&<br>
+ X86::AddrIndexReg == 2 && X86::AddrDisp == 3 &&<br>
+ X86::AddrSegmentReg == 4 &&<br>
+ "Unexpected change in X86 operand offset order.");<br>
+<br>
+ // This assumes X86::AddBaseReg = 0, {...}ScaleAmt = 1, etc.<br>
+ // FIXME(mtrofin): consider adding a:<br>
+ // MachineInstrBuilder::set(unsigned offset, op).<br>
+ MIB.addReg(Current->getOperand(MemOpOffset + X86::AddrBaseReg).getReg())<br>
+ .addImm(<br>
+ Current->getOperand(MemOpOffset + X86::AddrScaleAmt).getImm())<br>
+ .addReg(<br>
+ Current->getOperand(MemOpOffset + X86::AddrIndexReg).getReg())<br>
+ .addImm(Current->getOperand(MemOpOffset + X86::AddrDisp).getImm() +<br>
+ Delta)<br>
+ .addReg(Current->getOperand(MemOpOffset + X86::AddrSegmentReg)<br>
+ .getReg());<br>
+<br>
+ if (!Current->memoperands_empty()) {<br>
+ MachineMemOperand *CurrentOp = *(Current->memoperands_begin());<br>
+ MIB.addMemOperand(MF.getMachineMemOperand(<br>
+ CurrentOp, CurrentOp->getOffset() + Delta, CurrentOp->getSize()));<br>
+ }<br>
+<br>
+ // Insert before Current. This is because Current may clobber some of<br>
+ // the registers used to describe the input memory operand.<br>
+ MBB.insert(Current, PFetch);<br>
+ Changed = true;<br>
+ }<br>
+ }<br>
+ }<br>
+ return Changed;<br>
+}<br>
+<br>
+FunctionPass *llvm::createX86InsertPrefetchPass() {<br>
+ return new X86InsertPrefetch(PrefetchHintsFile);<br>
+}<br>
<br>
Modified: llvm/trunk/lib/Target/X86/X86TargetMachine.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86TargetMachine.cpp?rev=347938&r1=347937&r2=347938&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86TargetMachine.cpp?rev=347938&r1=347937&r2=347938&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Target/X86/X86TargetMachine.cpp (original)<br>
+++ llvm/trunk/lib/Target/X86/X86TargetMachine.cpp Thu Nov 29 17:01:52 2018<br>
@@ -497,6 +497,8 @@ void X86PassConfig::addPreEmitPass() {<br>
addPass(createX86FixupLEAs());<br>
addPass(createX86EvexToVexInsts());<br>
}<br>
+ addPass(createX86DiscriminateMemOpsPass());<br>
+ addPass(createX86InsertPrefetchPass());<br>
}<br>
<br>
void X86PassConfig::addPreEmitPass2() {<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/O0-pipeline.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/O0-pipeline.ll?rev=347938&r1=347937&r2=347938&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/O0-pipeline.ll?rev=347938&r1=347937&r2=347938&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/O0-pipeline.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/O0-pipeline.ll Thu Nov 29 17:01:52 2018<br>
@@ -58,6 +58,8 @@<br>
; CHECK-NEXT: Shadow Call Stack<br>
; CHECK-NEXT: X86 Indirect Branch Tracking<br>
; CHECK-NEXT: X86 vzeroupper inserter<br>
+; CHECK-NEXT: X86 Discriminate Memory Operands<br>
+; CHECK-NEXT: X86 Insert Cache Prefetches<br>
; CHECK-NEXT: Contiguously Lay Out Funclets<br>
; CHECK-NEXT: StackMap Liveness Analysis<br>
; CHECK-NEXT: Live DEBUG_VALUE analysis<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/O3-pipeline.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/O3-pipeline.ll?rev=347938&r1=347937&r2=347938&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/O3-pipeline.ll?rev=347938&r1=347937&r2=347938&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/O3-pipeline.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/O3-pipeline.ll Thu Nov 29 17:01:52 2018<br>
@@ -159,6 +159,8 @@<br>
; CHECK-NEXT: X86 Atom pad short functions<br>
; CHECK-NEXT: X86 LEA Fixup<br>
; CHECK-NEXT: Compressing EVEX instrs to VEX encoding when possible<br>
+; CHECK-NEXT: X86 Discriminate Memory Operands<br>
+; CHECK-NEXT: X86 Insert Cache Prefetches<br>
; CHECK-NEXT: Contiguously Lay Out Funclets<br>
; CHECK-NEXT: StackMap Liveness Analysis<br>
; CHECK-NEXT: Live DEBUG_VALUE analysis<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/discriminate-mem-ops.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/discriminate-mem-ops.ll?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/discriminate-mem-ops.ll?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/discriminate-mem-ops.ll (added)<br>
+++ llvm/trunk/test/CodeGen/X86/discriminate-mem-ops.ll Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,55 @@<br>
+; RUN: llc < %s | FileCheck %s<br>
+;<br>
+; original source, compiled with -O3 -gmlt -fdebug-info-for-profiling:<br>
+; int sum(int* arr, int pos1, int pos2) {<br>
+; return arr[pos1] + arr[pos2];<br>
+; }<br>
+;<br>
+; ModuleID = 'test.cc'<br>
+source_filename = "test.cc"<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+; Function Attrs: norecurse nounwind readonly uwtable<br>
+define i32 @sum(i32* %arr, i32 %pos1, i32 %pos2) !dbg !7 {<br>
+entry:<br>
+ %idxprom = sext i32 %pos1 to i64, !dbg !9<br>
+ %arrayidx = getelementptr inbounds i32, i32* %arr, i64 %idxprom, !dbg !9<br>
+ %0 = load i32, i32* %arrayidx, align 4, !dbg !9, !tbaa !10<br>
+ %idxprom1 = sext i32 %pos2 to i64, !dbg !14<br>
+ %arrayidx2 = getelementptr inbounds i32, i32* %arr, i64 %idxprom1, !dbg !14<br>
+ %1 = load i32, i32* %arrayidx2, align 4, !dbg !14, !tbaa !10<br>
+ %add = add nsw i32 %1, %0, !dbg !15<br>
+ ret i32 %add, !dbg !16<br>
+}<br>
+<br>
+attributes #0 = { "target-cpu"="x86-64" }<br>
+<br>
+!<a href="http://llvm.dbg.cu" rel="noreferrer" target="_blank">llvm.dbg.cu</a> = !{!0}<br>
+!llvm.module.flags = !{!3, !4, !5}<br>
+!llvm.ident = !{!6}<br>
+<br>
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: LineTablesOnly, enums: !2, debugInfoForProfiling: true)<br>
+!1 = !DIFile(filename: "test.cc", directory: "/tmp")<br>
+!2 = !{}<br>
+!3 = !{i32 2, !"Dwarf Version", i32 4}<br>
+!4 = !{i32 2, !"Debug Info Version", i32 3}<br>
+!5 = !{i32 1, !"wchar_size", i32 4}<br>
+!6 = !{!"clang version 7.0.0 (trunk 322155) (llvm/trunk 322159)"}<br>
+!7 = distinct !DISubprogram(name: "sum", linkageName: "sum", scope: !1, file: !1, line: 1, type: !8, isLocal: false, isDefinition: true, scopeLine: 1, flags: DIFlagPrototyped, isOptimized: true, unit: !0)<br>
+!8 = !DISubroutineType(types: !2)<br>
+!9 = !DILocation(line: 2, column: 10, scope: !7)<br>
+!10 = !{!11, !11, i64 0}<br>
+!11 = !{!"int", !12, i64 0}<br>
+!12 = !{!"omnipotent char", !13, i64 0}<br>
+!13 = !{!"Simple C++ TBAA"}<br>
+!14 = !DILocation(line: 2, column: 22, scope: !7)<br>
+!15 = !DILocation(line: 2, column: 20, scope: !7)<br>
+!16 = !DILocation(line: 2, column: 3, scope: !7)<br>
+<br>
+;CHECK-LABEL: sum:<br>
+;CHECK: # %bb.0:<br>
+;CHECK: movl (%rdi,%rax,4), %eax<br>
+;CHECK-NEXT: .loc 1 2 20 discriminator 2 # test.cc:2:20<br>
+;CHECK-NEXT: addl (%rdi,%rcx,4), %eax<br>
+;CHECK-NEXT: .loc 1 2 3 # test.cc:2:3<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.afdo<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.afdo?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.afdo?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.afdo (added)<br>
+++ llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.afdo Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,4 @@<br>
+caller:0:0<br>
+ 2:sum:0<br>
+ 3: 0 __prefetch_nta_0:23456<br>
+ 3.1: 0 __prefetch_nta_0:8764 __prefetch_nta_1:64<br>
\ No newline at end of file<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.ll?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.ll?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.ll (added)<br>
+++ llvm/trunk/test/CodeGen/X86/insert-prefetch-inline.ll Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,76 @@<br>
+; RUN: llc < %s -prefetch-hints-file=%S/insert-prefetch-inline.afdo | FileCheck %s<br>
+;<br>
+; Verify we can insert prefetch instructions in code belonging to inlined<br>
+; functions.<br>
+;<br>
+; ModuleID = 'test.cc'<br>
+<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+; Function Attrs: norecurse nounwind readonly uwtable<br>
+define dso_local i32 @sum(i32* nocapture readonly %arr, i32 %pos1, i32 %pos2) local_unnamed_addr #0 !dbg !7 {<br>
+entry:<br>
+ %idxprom = sext i32 %pos1 to i64, !dbg !10<br>
+ %arrayidx = getelementptr inbounds i32, i32* %arr, i64 %idxprom, !dbg !10<br>
+ %0 = load i32, i32* %arrayidx, align 4, !dbg !10, !tbaa !11<br>
+ %idxprom1 = sext i32 %pos2 to i64, !dbg !15<br>
+ %arrayidx2 = getelementptr inbounds i32, i32* %arr, i64 %idxprom1, !dbg !15<br>
+ %1 = load i32, i32* %arrayidx2, align 4, !dbg !15, !tbaa !11<br>
+ %add = add nsw i32 %1, %0, !dbg !16<br>
+ ret i32 %add, !dbg !17<br>
+}<br>
+<br>
+; "caller" inlines "sum". The associated .afdo file references instructions<br>
+; in "caller" that came from "sum"'s inlining.<br>
+;<br>
+; Function Attrs: norecurse nounwind readonly uwtable<br>
+define dso_local i32 @caller(i32* nocapture readonly %arr) local_unnamed_addr #0 !dbg !18 {<br>
+entry:<br>
+ %0 = load i32, i32* %arr, align 4, !dbg !19, !tbaa !11<br>
+ %arrayidx2.i = getelementptr inbounds i32, i32* %arr, i64 2, !dbg !21<br>
+ %1 = load i32, i32* %arrayidx2.i, align 4, !dbg !21, !tbaa !11<br>
+ %add.i = add nsw i32 %1, %0, !dbg !22<br>
+ ret i32 %add.i, !dbg !23<br>
+}<br>
+<br>
+attributes #0 = { "target-cpu"="x86-64" }<br>
+<br>
+!<a href="http://llvm.dbg.cu" rel="noreferrer" target="_blank">llvm.dbg.cu</a> = !{!0}<br>
+!llvm.module.flags = !{!3, !4, !5}<br>
+!llvm.ident = !{!6}<br>
+<br>
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !1, producer: "clang version 7.0.0 (trunk 324940) (llvm/trunk 324941)", isOptimized: true, runtimeVersion: 0, emissionKind: LineTablesOnly, enums: !2, debugInfoForProfiling: true)<br>
+!1 = !DIFile(filename: "test.cc", directory: "/tmp")<br>
+!2 = !{}<br>
+!3 = !{i32 2, !"Dwarf Version", i32 4}<br>
+!4 = !{i32 2, !"Debug Info Version", i32 3}<br>
+!5 = !{i32 1, !"wchar_size", i32 4}<br>
+!6 = !{!"clang version 7.0.0 (trunk 324940) (llvm/trunk 324941)"}<br>
+!7 = distinct !DISubprogram(name: "sum", linkageName: "sum", scope: !8, file: !8, line: 3, type: !9, isLocal: false, isDefinition: true, scopeLine: 3, flags: DIFlagPrototyped, isOptimized: true, unit: !0)<br>
+!8 = !DIFile(filename: "./test.h", directory: "/tmp")<br>
+!9 = !DISubroutineType(types: !2)<br>
+!10 = !DILocation(line: 6, column: 10, scope: !7)<br>
+!11 = !{!12, !12, i64 0}<br>
+!12 = !{!"int", !13, i64 0}<br>
+!13 = !{!"omnipotent char", !14, i64 0}<br>
+!14 = !{!"Simple C++ TBAA"}<br>
+!15 = !DILocation(line: 6, column: 22, scope: !7)<br>
+!16 = !DILocation(line: 6, column: 20, scope: !7)<br>
+!17 = !DILocation(line: 6, column: 3, scope: !7)<br>
+!18 = distinct !DISubprogram(name: "caller", linkageName: "caller", scope: !1, file: !1, line: 4, type: !9, isLocal: false, isDefinition: true, scopeLine: 4, flags: DIFlagPrototyped, isOptimized: true, unit: !0)<br>
+!19 = !DILocation(line: 6, column: 10, scope: !7, inlinedAt: !20)<br>
+!20 = distinct !DILocation(line: 6, column: 10, scope: !18)<br>
+!21 = !DILocation(line: 6, column: 22, scope: !7, inlinedAt: !20)<br>
+!22 = !DILocation(line: 6, column: 20, scope: !7, inlinedAt: !20)<br>
+!23 = !DILocation(line: 6, column: 3, scope: !18)<br>
+<br>
+; CHECK-LABEL: caller:<br>
+; CHECK-LABEL: # %bb.0:<br>
+; CHECK-NEXT: .loc 1 6 22 prologue_end<br>
+; CHECK-NEXT: prefetchnta 23464(%rdi)<br>
+; CHECK-NEXT: movl 8(%rdi), %eax<br>
+; CHECK-NEXT: .loc 1 6 20 is_stmt 0 discriminator 2<br>
+; CHECK-NEXT: prefetchnta 8764(%rdi)<br>
+; CHECK-NEXT: prefetchnta 64(%rdi)<br>
+; CHECK-NEXT: addl (%rdi), %eax<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.afdo<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.afdo?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.afdo?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.afdo (added)<br>
+++ llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.afdo Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,2 @@<br>
+main:0:0<br>
+ 6: 0 __prefetch_nta_0:42<br>
\ No newline at end of file<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.ll?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.ll?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.ll (added)<br>
+++ llvm/trunk/test/CodeGen/X86/insert-prefetch-invalid-instr.ll Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,46 @@<br>
+; RUN: llc < %s -prefetch-hints-file=%S/insert-prefetch-invalid-instr.afdo | FileCheck %s<br>
+; ModuleID = 'prefetch.cc'<br>
+source_filename = "prefetch.cc"<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+; Function Attrs: norecurse nounwind uwtable<br>
+define dso_local i32 @main() local_unnamed_addr #0 !dbg !7 {<br>
+entry:<br>
+ tail call void @llvm.prefetch(i8* inttoptr (i64 291 to i8*), i32 0, i32 0, i32 1), !dbg !9<br>
+ tail call void @llvm.x86.avx512.gatherpf.dpd.512(i8 97, <8 x i32> undef, i8* null, i32 1, i32 2), !dbg !10<br>
+ ret i32 291, !dbg !11<br>
+}<br>
+<br>
+; Function Attrs: inaccessiblemem_or_argmemonly nounwind<br>
+declare void @llvm.prefetch(i8* nocapture readonly, i32, i32, i32) #1<br>
+<br>
+; Function Attrs: argmemonly nounwind<br>
+declare void @llvm.x86.avx512.gatherpf.dpd.512(i8, <8 x i32>, i8*, i32, i32) #2<br>
+<br>
+attributes #0 = {"target-cpu"="x86-64" "target-features"="+avx512pf,+sse4.2,+ssse3"}<br>
+attributes #1 = { inaccessiblemem_or_argmemonly nounwind }<br>
+attributes #2 = { argmemonly nounwind }<br>
+<br>
+!<a href="http://llvm.dbg.cu" rel="noreferrer" target="_blank">llvm.dbg.cu</a> = !{!0}<br>
+!llvm.module.flags = !{!3, !4, !5}<br>
+!llvm.ident = !{!6}<br>
+<br>
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: LineTablesOnly, enums: !2, debugInfoForProfiling: true)<br>
+!1 = !DIFile(filename: "prefetch.cc", directory: "/tmp")<br>
+!2 = !{}<br>
+!3 = !{i32 2, !"Dwarf Version", i32 4}<br>
+!4 = !{i32 2, !"Debug Info Version", i32 3}<br>
+!5 = !{i32 1, !"wchar_size", i32 4}<br>
+!6 = !{!"clang version 7.0.0 (trunk 327078) (llvm/trunk 327086)"}<br>
+!7 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line: 8, type: !8, isLocal: false, isDefinition: true, scopeLine: 8, flags: DIFlagPrototyped, isOptimized: true, unit: !0)<br>
+!8 = !DISubroutineType(types: !2)<br>
+!9 = !DILocation(line: 12, column: 3, scope: !7)<br>
+!10 = !DILocation(line: 14, column: 3, scope: !7)<br>
+!11 = !DILocation(line: 15, column: 3, scope: !7)<br>
+<br>
+;CHECK-LABEL: main:<br>
+;CHECK: # %bb.0:<br>
+;CHECK: prefetchnta 291<br>
+;CHECK-NOT: prefetchnta 42(%rax,%ymm0)<br>
+;CHECK: vgatherpf1dpd (%rax,%ymm0) {%k1}<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/insert-prefetch-other.afdo<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-other.afdo?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch-other.afdo?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/insert-prefetch-other.afdo (added)<br>
+++ llvm/trunk/test/CodeGen/X86/insert-prefetch-other.afdo Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,3 @@<br>
+sum:0:0<br>
+ 1: 0 __prefetch_t0_1:0 __prefetch_t2_0:42<br>
+ 1.1: 0 __prefetch_t1_0:18446744073709551615<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/insert-prefetch.afdo<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch.afdo?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch.afdo?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/insert-prefetch.afdo (added)<br>
+++ llvm/trunk/test/CodeGen/X86/insert-prefetch.afdo Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,3 @@<br>
+sum:0:0<br>
+ 1: 0 __prefetch_nta_1:0 __prefetch_nta_0:42<br>
+ 1.1: 0 __prefetch_nta_0:18446744073709551615<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/insert-prefetch.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch.ll?rev=347938&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/insert-prefetch.ll?rev=347938&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/insert-prefetch.ll (added)<br>
+++ llvm/trunk/test/CodeGen/X86/insert-prefetch.ll Thu Nov 29 17:01:52 2018<br>
@@ -0,0 +1,101 @@<br>
+; RUN: llc < %s -prefetch-hints-file=%S/insert-prefetch.afdo | FileCheck %s<br>
+; RUN: llc < %s -prefetch-hints-file=%S/insert-prefetch-other.afdo | FileCheck %s -check-prefix=OTHERS<br>
+;<br>
+; original source, compiled with -O3 -gmlt -fdebug-info-for-profiling:<br>
+; int sum(int* arr, int pos1, int pos2) {<br>
+; return arr[pos1] + arr[pos2];<br>
+; }<br>
+;<br>
+; NOTE: debug line numbers were adjusted such that the function would start<br>
+; at line 15 (an arbitrary number). The sample profile file format uses<br>
+; offsets from the start of the symbol instead of file-relative line numbers.<br>
+; The .afdo file reflects that - the instructions are offset '1'.<br>
+;<br>
+; ModuleID = 'test.cc'<br>
+source_filename = "test.cc"<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+define i32 @sum(i32* %arr, i32 %pos1, i32 %pos2) !dbg !35 !prof !37 {<br>
+entry:<br>
+ %idxprom = sext i32 %pos1 to i64, !dbg !38<br>
+ %arrayidx = getelementptr inbounds i32, i32* %arr, i64 %idxprom, !dbg !38<br>
+ %0 = load i32, i32* %arrayidx, align 4, !dbg !38, !tbaa !39<br>
+ %idxprom1 = sext i32 %pos2 to i64, !dbg !43<br>
+ %arrayidx2 = getelementptr inbounds i32, i32* %arr, i64 %idxprom1, !dbg !43<br>
+ %1 = load i32, i32* %arrayidx2, align 4, !dbg !43, !tbaa !39<br>
+ %add = add nsw i32 %1, %0, !dbg !44<br>
+ ret i32 %add, !dbg !45<br>
+}<br>
+<br>
+attributes #0 = { "target-cpu"="x86-64" }<br>
+<br>
+!<a href="http://llvm.dbg.cu" rel="noreferrer" target="_blank">llvm.dbg.cu</a> = !{!0}<br>
+!llvm.module.flags = !{!3, !4, !5, !6}<br>
+!llvm.ident = !{!33}<br>
+<br>
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: LineTablesOnly, enums: !2, debugInfoForProfiling: true)<br>
+!1 = !DIFile(filename: "test.cc", directory: "/tmp")<br>
+!2 = !{}<br>
+!3 = !{i32 2, !"Dwarf Version", i32 4}<br>
+!4 = !{i32 2, !"Debug Info Version", i32 3}<br>
+!5 = !{i32 1, !"wchar_size", i32 4}<br>
+!6 = !{i32 1, !"ProfileSummary", !7}<br>
+!7 = !{!8, !9, !10, !11, !12, !13, !14, !15}<br>
+!8 = !{!"ProfileFormat", !"SampleProfile"}<br>
+!9 = !{!"TotalCount", i64 0}<br>
+!10 = !{!"MaxCount", i64 0}<br>
+!11 = !{!"MaxInternalCount", i64 0}<br>
+!12 = !{!"MaxFunctionCount", i64 0}<br>
+!13 = !{!"NumCounts", i64 2}<br>
+!14 = !{!"NumFunctions", i64 1}<br>
+!15 = !{!"DetailedSummary", !16}<br>
+!16 = !{!17, !18, !19, !20, !21, !22, !22, !23, !23, !24, !25, !26, !27, !28, !29, !30, !31, !32}<br>
+!17 = !{i32 10000, i64 0, i32 0}<br>
+!18 = !{i32 100000, i64 0, i32 0}<br>
+!19 = !{i32 200000, i64 0, i32 0}<br>
+!20 = !{i32 300000, i64 0, i32 0}<br>
+!21 = !{i32 400000, i64 0, i32 0}<br>
+!22 = !{i32 500000, i64 0, i32 0}<br>
+!23 = !{i32 600000, i64 0, i32 0}<br>
+!24 = !{i32 700000, i64 0, i32 0}<br>
+!25 = !{i32 800000, i64 0, i32 0}<br>
+!26 = !{i32 900000, i64 0, i32 0}<br>
+!27 = !{i32 950000, i64 0, i32 0}<br>
+!28 = !{i32 990000, i64 0, i32 0}<br>
+!29 = !{i32 999000, i64 0, i32 0}<br>
+!30 = !{i32 999900, i64 0, i32 0}<br>
+!31 = !{i32 999990, i64 0, i32 0}<br>
+!32 = !{i32 999999, i64 0, i32 0}<br>
+!33 = !{!"clang version 7.0.0 (trunk 322593) (llvm/trunk 322526)"}<br>
+!35 = distinct !DISubprogram(name: "sum", linkageName: "sum", scope: !1, file: !1, line: 15, type: !36, isLocal: false, isDefinition: true, scopeLine: 15, flags: DIFlagPrototyped, isOptimized: true, unit: !0)<br>
+!36 = !DISubroutineType(types: !2)<br>
+!37 = !{!"function_entry_count", i64 -1}<br>
+!38 = !DILocation(line: 16, column: 10, scope: !35)<br>
+!39 = !{!40, !40, i64 0}<br>
+!40 = !{!"int", !41, i64 0}<br>
+!41 = !{!"omnipotent char", !42, i64 0}<br>
+!42 = !{!"Simple C++ TBAA"}<br>
+!43 = !DILocation(line: 16, column: 22, scope: !35)<br>
+!44 = !DILocation(line: 16, column: 20, scope: !35)<br>
+!45 = !DILocation(line: 16, column: 3, scope: !35)<br>
+<br>
+;CHECK-LABEL: sum:<br>
+;CHECK: # %bb.0:<br>
+;CHECK: prefetchnta 42(%rdi,%rax,4)<br>
+;CHECK-NEXT: prefetchnta (%rdi,%rax,4)<br>
+;CHECK-NEXT: movl (%rdi,%rax,4), %eax<br>
+;CHECK-NEXT: .loc 1 16 20 discriminator 2 # test.cc:16:20<br>
+;CHECK-NEXT: prefetchnta -1(%rdi,%rcx,4)<br>
+;CHECK-NEXT: addl (%rdi,%rcx,4), %eax<br>
+;CHECK-NEXT: .loc 1 16 3 # test.cc:16:3<br>
+<br>
+;OTHERS-LABEL: sum:<br>
+;OTHERS: # %bb.0:<br>
+;OTHERS: prefetcht2 42(%rdi,%rax,4)<br>
+;OTHERS-NEXT: prefetcht0 (%rdi,%rax,4)<br>
+;OTHERS-NEXT: movl (%rdi,%rax,4), %eax<br>
+;OTHERS-NEXT: .loc 1 16 20 discriminator 2 # test.cc:16:20<br>
+;OTHERS-NEXT: prefetcht1 -1(%rdi,%rcx,4)<br>
+;OTHERS-NEXT: addl (%rdi,%rcx,4), %eax<br>
+;OTHERS-NEXT: .loc 1 16 3 # test.cc:16:3<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div>
</blockquote></div>