<div dir="ltr">Reverting the patch locally does seem to remove the failure.<br><br>Here's the crash dump from one of the failing tests:<br><br><div>/usr/local/google/home/blaikie/dev/llvm/build/default/./bin/llc < /usr/local/google/home/blaikie/dev/llvm/src/test/CodeGen/WebAssembly/global.ll -asm-verbose=false -disable-wasm-fallthrough-return-opt -disable-wasm-explicit-locals | /usr/local/google/home/blaikie/dev/llvm/build/default/./bin/FileCheck /usr/local/google/home/blaikie/dev/llvm/src/test/CodeGen/WebAssembly/global.ll</div><div>--</div><div>Exit Code: 2</div><div><br></div><div>Command Output (stderr):</div><div>--</div><div>#0 0x000000000471fcb9 llvm::sys::PrintStackTrace(llvm::raw_ostream&) src/lib/Support/Unix/Signals.inc:398:11</div><div>#1 0x000000000471fe69 PrintStackTraceSignalHandler(void*) src/lib/Support/Unix/Signals.inc:462:1</div><div>#2 0x000000000471e4c3 llvm::sys::RunSignalHandlers() src/lib/Support/Signals.cpp:0:5</div><div>#3 0x00000000047201c4 SignalHandler(int) src/lib/Support/Unix/Signals.inc:252:1</div><div>#4 0x00007f770219a330 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10330)</div><div>#5 0x00007f7700d7079a /build/eglibc-SvCtMH/eglibc-2.19/string/../sysdeps/x86_64/multiarch/strcmp-sse2-unaligned.S:29:0</div><div>#6 0x000000000350622b llvm::GetSignature(llvm::WebAssemblySubtarget const&, char const*, llvm::SmallVectorImpl<llvm::wasm::ValType>&, llvm::SmallVectorImpl<llvm::wasm::ValType>&) src/lib/Target/WebAssembly/WebAssemblyRuntimeLibcallSignatures.cpp:1294:3</div><div>#7 0x0000000003501337 llvm::WebAssemblyMCInstLower::GetExternalSymbolSymbol(llvm::MachineOperand const&) const src/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp:100:3</div><div>#8 0x0000000003501ce5 llvm::WebAssemblyMCInstLower::Lower(llvm::MachineInstr const*, llvm::MCInst&) const src/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp:229:33</div><div>#9 0x00000000034fee64 llvm::WebAssemblyAsmPrinter::EmitInstruction(llvm::MachineInstr const*) src/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp:208:5</div><div>#10 0x00000000037a9dd5 llvm::AsmPrinter::EmitFunctionBody() src/lib/CodeGen/AsmPrinter/AsmPrinter.cpp:1037:11</div><div>#11 0x00000000021225e8 llvm::AsmPrinter::runOnMachineFunction(llvm::MachineFunction&) src/include/llvm/CodeGen/AsmPrinter.h:277:5</div><div>#12 0x00000000034ff8d3 llvm::WebAssemblyAsmPrinter::runOnMachineFunction(llvm::MachineFunction&) src/lib/Target/WebAssembly/WebAssemblyAsmPrinter.h:49:5</div><div>#13 0x0000000003a21f61 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) src/lib/CodeGen/MachineFunctionPass.cpp:62:8</div><div>#14 0x0000000003e5d92f llvm::FPPassManager::runOnFunction(llvm::Function&) src/lib/IR/LegacyPassManager.cpp:1519:27</div><div>#15 0x0000000003e5dc45 llvm::FPPassManager::runOnModule(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1540:16</div><div>#16 0x0000000003e5e3da (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1596:27</div><div>#17 0x0000000003e5df06 llvm::legacy::PassManagerImpl::run(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1699:16</div><div>#18 0x0000000003e5e8f1 llvm::legacy::PassManager::run(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1730:3</div><div>#19 0x00000000020ea715 compileModule(char**, llvm::LLVMContext&) src/tools/llc/llc.cpp:619:42</div><div>#20 0x00000000020e883c main src/tools/llc/llc.cpp:358:13</div><div>#21 0x00007f7700cf7f45 __libc_start_main build/eglibc-SvCtMH/eglibc-2.19/csu/libc-start.c:321:0</div><div>#22 0x00000000020e8029 _start (build/default/./bin/llc+0x20e8029)</div><div>Stack dump:</div><div>0. Program arguments: build/default/./bin/llc -asm-verbose=false -disable-wasm-fallthrough-return-opt -disable-wasm-explicit-locals</div><div>1. Running pass 'Function Pass Manager' on module '<stdin>'.</div><div>2. Running pass 'WebAssembly Assembly Printer' on function '@call_memcpy'</div><div>FileCheck error: '-' is empty.</div><div>FileCheck command line: build/default/./bin/FileCheck src/test/CodeGen/WebAssembly/global.ll</div><br><div class="gmail_quote"><div dir="ltr">On Wed, Jul 12, 2017 at 11:31 AM Daniel Neilson <<a href="mailto:dneilson@azul.com">dneilson@azul.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>That’s surprising to me; this should basically be NFC since nothing is actually using the introduced intrinsic. Any information that you can share would be appreciated.</div></div><div style="word-wrap:break-word">
<div><br>
</div>
<div>-Daniel</div></div><div style="word-wrap:break-word">
<br>
<div>
<blockquote type="cite">
<div>On Jul 12, 2017, at 11:26 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:</div>
<br class="m_-6007959335525949403Apple-interchange-newline">
<div>
<div dir="ltr">This seems to be breaking internal buildbots on WebAssembly tests (test/CodeGen/WebAssembli/{byval,f16,f32,f64,frem,global,i128,legalize,mem-intrinsics}.ll<br>
<br>
I've not reproduced this myself, but working on that now to get more info, including stack traces, etc.</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, Jul 12, 2017 at 8:25 AM Daniel Neilson via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Author: dneilson<br>
Date: Wed Jul 12 08:25:26 2017<br>
New Revision: 307796<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=307796&view=rev" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project?rev=307796&view=rev</a><br>
Log:<br>
Add element atomic memmove intrinsic<br>
<br>
Summary: Continuing the work from <a href="https://reviews.llvm.org/D33240" rel="noreferrer" target="_blank">
https://reviews.llvm.org/D33240</a>, this change introduces an element unordered-atomic memmove intrinsic. This intrinsic is essentially memmove with the implementation requirement that all loads/stores used for the copy are done with unordered-atomic loads/stores
of a given element size.<br>
<br>
Reviewers: eli.friedman, reames, mkazantsev, skatkov<br>
<br>
Reviewed By: reames<br>
<br>
Subscribers: llvm-commits<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D34884" rel="noreferrer" target="_blank">
https://reviews.llvm.org/D34884</a><br>
<br>
Modified:<br>
llvm/trunk/docs/LangRef.rst<br>
llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h<br>
llvm/trunk/include/llvm/IR/IntrinsicInst.h<br>
llvm/trunk/include/llvm/IR/Intrinsics.td<br>
llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp<br>
llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp<br>
llvm/trunk/lib/IR/Verifier.cpp<br>
llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll<br>
llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll<br>
<br>
Modified: llvm/trunk/docs/LangRef.rst<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/LangRef.rst?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/LangRef.rst?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/docs/LangRef.rst (original)<br>
+++ llvm/trunk/docs/LangRef.rst Wed Jul 12 08:25:26 2017<br>
@@ -10282,6 +10282,8 @@ overlap. It copies "len" bytes of memory<br>
to be aligned to some boundary, this can be specified as the fourth<br>
argument, otherwise it should be set to 0 or 1 (both meaning no alignment).<br>
<br>
+.. _int_memmove:<br>
+<br>
'``llvm.memmove``' Intrinsic<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
<br>
@@ -14178,4 +14180,81 @@ In the most general case call to the '``<br>
lowered to a call to the symbol ``__llvm_memcpy_element_unordered_atomic_*``. Where '*'<br>
is replaced with an actual element size.<br>
<br>
+Optimizer is allowed to inline memory copy when it's profitable to do so.<br>
+<br>
+'``llvm.memmove.element.unordered.atomic``' Intrinsic<br>
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
+<br>
+Syntax:<br>
+"""""""<br>
+<br>
+This is an overloaded intrinsic. You can use<br>
+``llvm.memmove.element.unordered.atomic`` on any integer bit width and for<br>
+different address spaces. Not all targets support all bit widths however.<br>
+<br>
+::<br>
+<br>
+ declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* <dest>,<br>
+ i8* <src>,<br>
+ i32 <len>,<br>
+ i32 <element_size>)<br>
+ declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i64(i8* <dest>,<br>
+ i8* <src>,<br>
+ i64 <len>,<br>
+ i32 <element_size>)<br>
+<br>
+Overview:<br>
+"""""""""<br>
+<br>
+The '``llvm.memmove.element.unordered.atomic.*``' intrinsic is a specialization<br>
+of the '``llvm.memmove.*``' intrinsic. It differs in that the ``dest`` and<br>
+``src`` are treated as arrays with elements that are exactly ``element_size``<br>
+bytes, and the copy between buffers uses a sequence of<br>
+:ref:`unordered atomic <ordering>` load/store operations that are a positive<br>
+integer multiple of the ``element_size`` in size.<br>
+<br>
+Arguments:<br>
+""""""""""<br>
+<br>
+The first three arguments are the same as they are in the<br>
+:ref:`@llvm.memmove <int_memmove>` intrinsic, with the added constraint that<br>
+``len`` is required to be a positive integer multiple of the ``element_size``.<br>
+If ``len`` is not a positive integer multiple of ``element_size``, then the<br>
+behaviour of the intrinsic is undefined.<br>
+<br>
+``element_size`` must be a compile-time constant positive power of two no<br>
+greater than a target-specific atomic access size limit.<br>
+<br>
+For each of the input pointers the ``align`` parameter attribute must be<br>
+specified. It must be a power of two no less than the ``element_size``. Caller<br>
+guarantees that both the source and destination pointers are aligned to that<br>
+boundary.<br>
+<br>
+Semantics:<br>
+""""""""""<br>
+<br>
+The '``llvm.memmove.element.unordered.atomic.*``' intrinsic copies ``len`` bytes<br>
+of memory from the source location to the destination location. These locations<br>
+are allowed to overlap. The memory copy is performed as a sequence of load/store<br>
+operations where each access is guaranteed to be a multiple of ``element_size``<br>
+bytes wide and aligned at an ``element_size`` boundary.<br>
+<br>
+The order of the copy is unspecified. The same value may be read from the source<br>
+buffer many times, but only one write is issued to the destination buffer per<br>
+element. It is well defined to have concurrent reads and writes to both source<br>
+and destination provided those reads and writes are unordered atomic when<br>
+specified.<br>
+<br>
+This intrinsic does not provide any additional ordering guarantees over those<br>
+provided by a set of unordered loads from the source location and stores to the<br>
+destination.<br>
+<br>
+Lowering:<br>
+"""""""""<br>
+<br>
+In the most general case call to the<br>
+'``llvm.memmove.element.unordered.atomic.*``' is lowered to a call to the symbol<br>
+``__llvm_memmove_element_unordered_atomic_*``. Where '*' is replaced with an<br>
+actual element size.<br>
+<br>
The optimizer is allowed to inline the memory copy when it's profitable to do so.<br>
<br>
Modified: llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h (original)<br>
+++ llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h Wed Jul 12 08:25:26 2017<br>
@@ -340,6 +340,12 @@ namespace RTLIB {<br>
MEMCPY_ELEMENT_UNORDERED_ATOMIC_8,<br>
MEMCPY_ELEMENT_UNORDERED_ATOMIC_16,<br>
<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_1,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_2,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_4,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_8,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_16,<br>
+<br>
// EXCEPTION HANDLING<br>
UNWIND_RESUME,<br>
<br>
@@ -515,6 +521,11 @@ namespace RTLIB {<br>
/// MEMCPY_ELEMENT_UNORDERED_ATOMIC_* value for the given element size or<br>
/// UNKNOW_LIBCALL if there is none.<br>
Libcall getMEMCPY_ELEMENT_UNORDERED_ATOMIC(uint64_t ElementSize);<br>
+<br>
+ /// getMEMMOVE_ELEMENT_UNORDERED_ATOMIC - Return<br>
+ /// MEMMOVE_ELEMENT_UNORDERED_ATOMIC_* value for the given element size or<br>
+ /// UNKNOW_LIBCALL if there is none.<br>
+ Libcall getMEMMOVE_ELEMENT_UNORDERED_ATOMIC(uint64_t ElementSize);<br>
}<br>
}<br>
<br>
<br>
Modified: llvm/trunk/include/llvm/IR/IntrinsicInst.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/IntrinsicInst.h?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/IntrinsicInst.h?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/IR/IntrinsicInst.h (original)<br>
+++ llvm/trunk/include/llvm/IR/IntrinsicInst.h Wed Jul 12 08:25:26 2017<br>
@@ -296,6 +296,95 @@ namespace llvm {<br>
}<br>
};<br>
<br>
+ class ElementUnorderedAtomicMemMoveInst : public IntrinsicInst {<br>
+ private:<br>
+ enum { ARG_DEST = 0, ARG_SOURCE = 1, ARG_LENGTH = 2, ARG_ELEMENTSIZE = 3 };<br>
+<br>
+ public:<br>
+ Value *getRawDest() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_DEST));<br>
+ }<br>
+ const Use &getRawDestUse() const { return getArgOperandUse(ARG_DEST); }<br>
+ Use &getRawDestUse() { return getArgOperandUse(ARG_DEST); }<br>
+<br>
+ /// Return the arguments to the instruction.<br>
+ Value *getRawSource() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_SOURCE));<br>
+ }<br>
+ const Use &getRawSourceUse() const { return getArgOperandUse(ARG_SOURCE); }<br>
+ Use &getRawSourceUse() { return getArgOperandUse(ARG_SOURCE); }<br>
+<br>
+ Value *getLength() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_LENGTH));<br>
+ }<br>
+ const Use &getLengthUse() const { return getArgOperandUse(ARG_LENGTH); }<br>
+ Use &getLengthUse() { return getArgOperandUse(ARG_LENGTH); }<br>
+<br>
+ bool isVolatile() const { return false; }<br>
+<br>
+ Value *getRawElementSizeInBytes() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_ELEMENTSIZE));<br>
+ }<br>
+<br>
+ ConstantInt *getElementSizeInBytesCst() const {<br>
+ return cast<ConstantInt>(getRawElementSizeInBytes());<br>
+ }<br>
+<br>
+ uint32_t getElementSizeInBytes() const {<br>
+ return getElementSizeInBytesCst()->getZExtValue();<br>
+ }<br>
+<br>
+ /// This is just like getRawDest, but it strips off any cast<br>
+ /// instructions that feed it, giving the original input. The returned<br>
+ /// value is guaranteed to be a pointer.<br>
+ Value *getDest() const { return getRawDest()->stripPointerCasts(); }<br>
+<br>
+ /// This is just like getRawSource, but it strips off any cast<br>
+ /// instructions that feed it, giving the original input. The returned<br>
+ /// value is guaranteed to be a pointer.<br>
+ Value *getSource() const { return getRawSource()->stripPointerCasts(); }<br>
+<br>
+ unsigned getDestAddressSpace() const {<br>
+ return cast<PointerType>(getRawDest()->getType())->getAddressSpace();<br>
+ }<br>
+<br>
+ unsigned getSourceAddressSpace() const {<br>
+ return cast<PointerType>(getRawSource()->getType())->getAddressSpace();<br>
+ }<br>
+<br>
+ /// Set the specified arguments of the instruction.<br>
+ void setDest(Value *Ptr) {<br>
+ assert(getRawDest()->getType() == Ptr->getType() &&<br>
+ "setDest called with pointer of wrong type!");<br>
+ setArgOperand(ARG_DEST, Ptr);<br>
+ }<br>
+<br>
+ void setSource(Value *Ptr) {<br>
+ assert(getRawSource()->getType() == Ptr->getType() &&<br>
+ "setSource called with pointer of wrong type!");<br>
+ setArgOperand(ARG_SOURCE, Ptr);<br>
+ }<br>
+<br>
+ void setLength(Value *L) {<br>
+ assert(getLength()->getType() == L->getType() &&<br>
+ "setLength called with value of wrong type!");<br>
+ setArgOperand(ARG_LENGTH, L);<br>
+ }<br>
+<br>
+ void setElementSizeInBytes(Constant *V) {<br>
+ assert(V->getType() == Type::getInt8Ty(getContext()) &&<br>
+ "setElementSizeInBytes called with value of wrong type!");<br>
+ setArgOperand(ARG_ELEMENTSIZE, V);<br>
+ }<br>
+<br>
+ static inline bool classof(const IntrinsicInst *I) {<br>
+ return I->getIntrinsicID() == Intrinsic::memmove_element_unordered_atomic;<br>
+ }<br>
+ static inline bool classof(const Value *V) {<br>
+ return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));<br>
+ }<br>
+ };<br>
+<br>
/// This is the common base class for memset/memcpy/memmove.<br>
class MemIntrinsic : public IntrinsicInst {<br>
public:<br>
<br>
Modified: llvm/trunk/include/llvm/IR/Intrinsics.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Intrinsics.td?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Intrinsics.td?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/IR/Intrinsics.td (original)<br>
+++ llvm/trunk/include/llvm/IR/Intrinsics.td Wed Jul 12 08:25:26 2017<br>
@@ -873,6 +873,18 @@ def int_memcpy_element_unordered_atomic<br>
ReadOnly<1><br>
]>;<br>
<br>
+// @llvm.memmove.element.unordered.atomic.*(dest, src, length, elementsize)<br>
+def int_memmove_element_unordered_atomic<br>
+ : Intrinsic<[],<br>
+ [<br>
+ llvm_anyptr_ty, llvm_anyptr_ty, llvm_anyint_ty, llvm_i32_ty<br>
+ ],<br>
+ [<br>
+ IntrArgMemOnly, NoCapture<0>, NoCapture<1>, WriteOnly<0>,<br>
+ ReadOnly<1><br>
+ ]>;<br>
+<br>
+<br>
//===------------------------ Reduction Intrinsics ------------------------===//<br>
//<br>
def int_experimental_vector_reduce_fadd : Intrinsic<[llvm_anyfloat_ty],<br>
<br>
Modified: llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp (original)<br>
+++ llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp Wed Jul 12 08:25:26 2017<br>
@@ -4994,6 +4994,44 @@ SelectionDAGBuilder::visitIntrinsicCall(<br>
DAG.setRoot(CallResult.second);<br>
return nullptr;<br>
}<br>
+ case Intrinsic::memmove_element_unordered_atomic: {<br>
+ auto &MI = cast<ElementUnorderedAtomicMemMoveInst>(I);<br>
+ SDValue Dst = getValue(MI.getRawDest());<br>
+ SDValue Src = getValue(MI.getRawSource());<br>
+ SDValue Length = getValue(MI.getLength());<br>
+<br>
+ // Emit a library call.<br>
+ TargetLowering::ArgListTy Args;<br>
+ TargetLowering::ArgListEntry Entry;<br>
+ Entry.Ty = DAG.getDataLayout().getIntPtrType(*DAG.getContext());<br>
+ Entry.Node = Dst;<br>
+ Args.push_back(Entry);<br>
+<br>
+ Entry.Node = Src;<br>
+ Args.push_back(Entry);<br>
+<br>
+ Entry.Ty = MI.getLength()->getType();<br>
+ Entry.Node = Length;<br>
+ Args.push_back(Entry);<br>
+<br>
+ uint64_t ElementSizeConstant = MI.getElementSizeInBytes();<br>
+ RTLIB::Libcall LibraryCall =<br>
+ RTLIB::getMEMMOVE_ELEMENT_UNORDERED_ATOMIC(ElementSizeConstant);<br>
+ if (LibraryCall == RTLIB::UNKNOWN_LIBCALL)<br>
+ report_fatal_error("Unsupported element size");<br>
+<br>
+ TargetLowering::CallLoweringInfo CLI(DAG);<br>
+ CLI.setDebugLoc(sdl).setChain(getRoot()).setLibCallee(<br>
+ TLI.getLibcallCallingConv(LibraryCall),<br>
+ Type::getVoidTy(*DAG.getContext()),<br>
+ DAG.getExternalSymbol(TLI.getLibcallName(LibraryCall),<br>
+ TLI.getPointerTy(DAG.getDataLayout())),<br>
+ std::move(Args));<br>
+<br>
+ std::pair<SDValue, SDValue> CallResult = TLI.LowerCallTo(CLI);<br>
+ DAG.setRoot(CallResult.second);<br>
+ return nullptr;<br>
+ }<br>
case Intrinsic::dbg_declare: {<br>
const DbgDeclareInst &DI = cast<DbgDeclareInst>(I);<br>
DILocalVariable *Variable = DI.getVariable();<br>
<br>
Modified: llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp (original)<br>
+++ llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp Wed Jul 12 08:25:26 2017<br>
@@ -384,6 +384,16 @@ static void InitLibcallNames(const char<br>
"__llvm_memcpy_element_unordered_atomic_8";<br>
Names[RTLIB::MEMCPY_ELEMENT_UNORDERED_ATOMIC_16] =<br>
"__llvm_memcpy_element_unordered_atomic_16";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_1] =<br>
+ "__llvm_memmove_element_unordered_atomic_1";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_2] =<br>
+ "__llvm_memmove_element_unordered_atomic_2";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_4] =<br>
+ "__llvm_memmove_element_unordered_atomic_4";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_8] =<br>
+ "__llvm_memmove_element_unordered_atomic_8";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_16] =<br>
+ "__llvm_memmove_element_unordered_atomic_16";<br>
Names[RTLIB::UNWIND_RESUME] = "_Unwind_Resume";<br>
Names[RTLIB::SYNC_VAL_COMPARE_AND_SWAP_1] = "__sync_val_compare_and_swap_1";<br>
Names[RTLIB::SYNC_VAL_COMPARE_AND_SWAP_2] = "__sync_val_compare_and_swap_2";<br>
@@ -801,6 +811,23 @@ RTLIB::Libcall RTLIB::getMEMCPY_ELEMENT_<br>
default:<br>
return UNKNOWN_LIBCALL;<br>
}<br>
+}<br>
+<br>
+RTLIB::Libcall RTLIB::getMEMMOVE_ELEMENT_UNORDERED_ATOMIC(uint64_t ElementSize) {<br>
+ switch (ElementSize) {<br>
+ case 1:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_1;<br>
+ case 2:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_2;<br>
+ case 4:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_4;<br>
+ case 8:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_8;<br>
+ case 16:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_16;<br>
+ default:<br>
+ return UNKNOWN_LIBCALL;<br>
+ }<br>
}<br>
<br>
/// InitCmpLibcallCCs - Set default comparison libcall CC.<br>
<br>
Modified: llvm/trunk/lib/IR/Verifier.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Verifier.cpp?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Verifier.cpp?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/IR/Verifier.cpp (original)<br>
+++ llvm/trunk/lib/IR/Verifier.cpp Wed Jul 12 08:25:26 2017<br>
@@ -4044,6 +4044,42 @@ void Verifier::visitIntrinsicCallSite(In<br>
"incorrect alignment of the source argument", CS);<br>
break;<br>
}<br>
+ case Intrinsic::memmove_element_unordered_atomic: {<br>
+ auto *MI = cast<ElementUnorderedAtomicMemMoveInst>(CS.getInstruction());<br>
+<br>
+ ConstantInt *ElementSizeCI =<br>
+ dyn_cast<ConstantInt>(MI->getRawElementSizeInBytes());<br>
+ Assert(ElementSizeCI,<br>
+ "element size of the element-wise unordered atomic memory "<br>
+ "intrinsic must be a constant int",<br>
+ CS);<br>
+ const APInt &ElementSizeVal = ElementSizeCI->getValue();<br>
+ Assert(ElementSizeVal.isPowerOf2(),<br>
+ "element size of the element-wise atomic memory intrinsic "<br>
+ "must be a power of 2",<br>
+ CS);<br>
+<br>
+ if (auto *LengthCI = dyn_cast<ConstantInt>(MI->getLength())) {<br>
+ uint64_t Length = LengthCI->getZExtValue();<br>
+ uint64_t ElementSize = MI->getElementSizeInBytes();<br>
+ Assert((Length % ElementSize) == 0,<br>
+ "constant length must be a multiple of the element size in the "<br>
+ "element-wise atomic memory intrinsic",<br>
+ CS);<br>
+ }<br>
+<br>
+ auto IsValidAlignment = [&](uint64_t Alignment) {<br>
+ return isPowerOf2_64(Alignment) && ElementSizeVal.ule(Alignment);<br>
+ };<br>
+ uint64_t DstAlignment = CS.getParamAlignment(0),<br>
+ SrcAlignment = CS.getParamAlignment(1);<br>
+ Assert(IsValidAlignment(DstAlignment),<br>
+ "incorrect alignment of the destination argument", CS);<br>
+ Assert(IsValidAlignment(SrcAlignment),<br>
+ "incorrect alignment of the source argument", CS);<br>
+<br>
+ break;<br>
+ }<br>
case Intrinsic::gcroot:<br>
case Intrinsic::gcwrite:<br>
case Intrinsic::gcread:<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll Wed Jul 12 08:25:26 2017<br>
@@ -62,4 +62,67 @@ define void @test_memcpy_args(i8** %Stor<br>
call void @llvm.memcpy.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %Dst, i8* align 4 %Src, i32 4, i32 4) ret void<br>
}<br>
<br>
+define i8* @test_memmove1(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 1, i32 1)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $1, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_1<br>
+}<br>
+<br>
+define i8* @test_memmove2(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove2<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 2, i32 2)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $2, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_2<br>
+}<br>
+<br>
+define i8* @test_memmove4(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove4<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 4, i32 4)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $4, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_4<br>
+}<br>
+<br>
+define i8* @test_memmove8(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove8<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 8 %P, i8* align 8 %Q, i32 8, i32 8)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $8, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_8<br>
+}<br>
+<br>
+define i8* @test_memmove16(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove16<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 16 %P, i8* align 16 %Q, i32 16, i32 16)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $16, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_16<br>
+}<br>
+<br>
+define void @test_memmove_args(i8** %Storage) {<br>
+ ; CHECK: test_memmove_args<br>
+ %Dst = load i8*, i8** %Storage<br>
+ %Src.addr = getelementptr i8*, i8** %Storage, i64 1<br>
+ %Src = load i8*, i8** %Src.addr<br>
+<br>
+ ; 1st arg (%rdi)<br>
+ ; CHECK-DAG: movq (%rdi), [[REG1:%r.+]]<br>
+ ; CHECK-DAG: movq [[REG1]], %rdi<br>
+ ; 2nd arg (%rsi)<br>
+ ; CHECK-DAG: movq 8(%rdi), %rsi<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $4, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_4<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %Dst, i8* align 4 %Src, i32 4, i32 4) ret void<br>
+}<br>
+<br>
declare void @llvm.memcpy.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
+declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
<br>
Modified: llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll (original)<br>
+++ llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll Wed Jul 12 08:25:26 2017<br>
@@ -22,4 +22,28 @@ define void @test_memcpy(i8* %P, i8* %Q,<br>
ret void<br>
}<br>
declare void @llvm.memcpy.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
+<br>
+define void @test_memmove(i8* %P, i8* %Q, i32 %A, i32 %E) {<br>
+ ; CHECK: element size of the element-wise unordered atomic memory intrinsic must be a constant int<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 1, i32 %E)<br>
+ ; CHECK: element size of the element-wise atomic memory intrinsic must be a power of 2<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 1, i32 3)<br>
+<br>
+ ; CHECK: constant length must be a multiple of the element size in the element-wise atomic memory intrinsic<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 7, i32 4)<br>
+<br>
+ ; CHECK: incorrect alignment of the destination argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* %P, i8* align 4 %Q, i32 1, i32 1)<br>
+ ; CHECK: incorrect alignment of the destination argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 1 %P, i8* align 4 %Q, i32 4, i32 4)<br>
+<br>
+ ; CHECK: incorrect alignment of the source argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* %Q, i32 1, i32 1)<br>
+ ; CHECK: incorrect alignment of the source argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 1 %Q, i32 4, i32 4)<br>
+<br>
+ ret void<br>
+}<br>
+declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
+<br>
; CHECK: input module is broken!<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div></blockquote></div></div>