<div dir="ltr">Sample of the TargetLoweringBase part here: <a href="https://reviews.llvm.org/D35522">https://reviews.llvm.org/D35522</a></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Jul 17, 2017 at 4:44 PM Derek Schuff <<a href="mailto:dschuff@google.com">dschuff@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">It looks mostly straightforward to move the names into the def file, with the initialization of the name list in TargetLoweringBase.cpp reducing mostly to a single inclusion. There are a few quirks (e.g. target-dependent names) that would have to be fixed-up after the inclusion but it doesn't seem too bad.<div><br></div><div>Would you be opposed to adding the signature information to the def file also? (Currently the signature is really just a named descriminator and separate logic in the wasm code knows what to do for each signature). Only the wasm backend would use it right now, but it would make all the uses straightforward. The wasm backend would use the common name definition, and then have an additional big list of functions to whitelist the ones that are supported. Any new additions to the def file would be automatically included in the common name list, and be handled as unsupported by the wasm backend.</div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Jul 17, 2017 at 10:50 AM Daniel Neilson <<a href="mailto:dneilson@azul.com" target="_blank">dneilson@azul.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>There’s a similar thing in InitLibcallNames() within CodeGen/TargetLoweringBase.cpp that would be good to roll into this if it can be made to work.</div></div><div style="word-wrap:break-word">
<div><br>
</div>
<div>-Daniel</div></div><div style="word-wrap:break-word">
<br>
<div>
<blockquote type="cite">
<div>On Jul 17, 2017, at 9:45 AM, Derek Schuff <<a href="mailto:dschuff@google.com" target="_blank">dschuff@google.com</a>> wrote:</div>
<br class="m_-8586201336093946882m_8787684839763540149Apple-interchange-newline">
<div>
<div dir="ltr">Hi Daniel, David,
<div><br>
</div>
<div>Thanks for fixing this up!</div>
<div>I also agree that this current system is too brittle and we should fix it.</div>
<div>Fundamentally what we need is to keep information about each libcall (its signature) in order to generate the wasm type declarations for generated calls. I would guess that the current system attempted to avoid making changes to the shared RuntimeLibCalls
code, but I think it might make more sense to turn the enum in RuntimeLibCalls.h into a .def file similar to what we have in other places (instructions, etc). Then we can reuse it to define the RTLIB::Libcall enumeration and also the wasm signatures.</div>
<div><br>
</div>
<div>I can volunteer to do this if that seems like a good idea (or we can figure something else out).</div>
<div><br>
</div>
<div>thanks,</div>
<div>-Derek</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, Jul 12, 2017 at 12:32 PM David Blaikie via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Wed, Jul 12, 2017 at 12:05 PM Daniel Neilson <<a href="mailto:dneilson@azul.com" target="_blank">dneilson@azul.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>The problem is in lib/Target/WebAssembly/WebAssemblyRuntimeLibcallSignatures.cpp. There’s an array declared:</div>
<div>545 static const char *<br>
546 RuntimeLibcallNames[RTLIB::UNKNOWN_LIBCALL] = {<br>
<br>
</div>
<div> That is defining a runtime lib call name for each entry in the enum RTLIB:Libcall from include/llvm/CodeGen/RuntimeLibcalls.h.</div>
<div><br>
</div>
<div>This patch added entries to the enum, but didn’t add entries to the RuntimeLibcallNames array. In fact, it looks like there are a couple of arrays in that WebAssembly file that are expecting to be changed/updated whenever the RTLIB::Libcall enum
has been changed/updated. Mirrored structures like this are kind of dangerous for exactly this reason, aren’t they?</div>
<div><br>
</div>
<div> What’s the suggested way forward? (Still kind of new to LLVM) Should I try to add the corresponding entries to the WebAssembly arrays? Does someone with some familiarity with those tables want to tackle it?</div>
</div>
</blockquote>
</div>
</div>
<div dir="ltr">
<div class="gmail_quote">
<div><br>
Thanks for committing a fix - and I agree with you that this representation sounds problematic.<br>
<br>
Adding a couple of WebAssembly developers to this thread to bnring this to their attention, see if they have some ideas about how to make this code less brittle.<br>
</div>
</div>
</div>
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div><br>
</div>
<div>-Daniel</div>
</div>
<div style="word-wrap:break-word">
<div><br>
</div>
<div>
<blockquote type="cite">
<div>On Jul 12, 2017, at 11:37 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:</div>
<br class="m_-8586201336093946882m_8787684839763540149m_3161077768360514121m_1278730238680722948Apple-interchange-newline">
<div>
<div dir="ltr">Reverting the patch locally does seem to remove the failure.<br>
<br>
Here's the crash dump from one of the failing tests:<br>
<br>
<div>/usr/local/google/home/blaikie/dev/llvm/build/default/./bin/llc < /usr/local/google/home/blaikie/dev/llvm/src/test/CodeGen/WebAssembly/global.ll -asm-verbose=false -disable-wasm-fallthrough-return-opt -disable-wasm-explicit-locals | /usr/local/google/home/blaikie/dev/llvm/build/default/./bin/FileCheck
/usr/local/google/home/blaikie/dev/llvm/src/test/CodeGen/WebAssembly/global.ll</div>
<div>--</div>
<div>Exit Code: 2</div>
<div><br>
</div>
<div>Command Output (stderr):</div>
<div>--</div>
<div>#0 0x000000000471fcb9 llvm::sys::PrintStackTrace(llvm::raw_ostream&) src/lib/Support/Unix/Signals.inc:398:11</div>
<div>#1 0x000000000471fe69 PrintStackTraceSignalHandler(void*) src/lib/Support/Unix/Signals.inc:462:1</div>
<div>#2 0x000000000471e4c3 llvm::sys::RunSignalHandlers() src/lib/Support/Signals.cpp:0:5</div>
<div>#3 0x00000000047201c4 SignalHandler(int) src/lib/Support/Unix/Signals.inc:252:1</div>
<div>#4 0x00007f770219a330 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10330)</div>
<div>#5 0x00007f7700d7079a /build/eglibc-SvCtMH/eglibc-2.19/string/../sysdeps/x86_64/multiarch/strcmp-sse2-unaligned.S:29:0</div>
<div>#6 0x000000000350622b llvm::GetSignature(llvm::WebAssemblySubtarget const&, char const*, llvm::SmallVectorImpl<llvm::wasm::ValType>&, llvm::SmallVectorImpl<llvm::wasm::ValType>&) src/lib/Target/WebAssembly/WebAssemblyRuntimeLibcallSignatures.cpp:1294:3</div>
<div>#7 0x0000000003501337 llvm::WebAssemblyMCInstLower::GetExternalSymbolSymbol(llvm::MachineOperand const&) const src/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp:100:3</div>
<div>#8 0x0000000003501ce5 llvm::WebAssemblyMCInstLower::Lower(llvm::MachineInstr const*, llvm::MCInst&) const src/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp:229:33</div>
<div>#9 0x00000000034fee64 llvm::WebAssemblyAsmPrinter::EmitInstruction(llvm::MachineInstr const*) src/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp:208:5</div>
<div>#10 0x00000000037a9dd5 llvm::AsmPrinter::EmitFunctionBody() src/lib/CodeGen/AsmPrinter/AsmPrinter.cpp:1037:11</div>
<div>#11 0x00000000021225e8 llvm::AsmPrinter::runOnMachineFunction(llvm::MachineFunction&) src/include/llvm/CodeGen/AsmPrinter.h:277:5</div>
<div>#12 0x00000000034ff8d3 llvm::WebAssemblyAsmPrinter::runOnMachineFunction(llvm::MachineFunction&) src/lib/Target/WebAssembly/WebAssemblyAsmPrinter.h:49:5</div>
<div>#13 0x0000000003a21f61 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) src/lib/CodeGen/MachineFunctionPass.cpp:62:8</div>
<div>#14 0x0000000003e5d92f llvm::FPPassManager::runOnFunction(llvm::Function&) src/lib/IR/LegacyPassManager.cpp:1519:27</div>
<div>#15 0x0000000003e5dc45 llvm::FPPassManager::runOnModule(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1540:16</div>
<div>#16 0x0000000003e5e3da (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1596:27</div>
<div>#17 0x0000000003e5df06 llvm::legacy::PassManagerImpl::run(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1699:16</div>
<div>#18 0x0000000003e5e8f1 llvm::legacy::PassManager::run(llvm::Module&) src/lib/IR/LegacyPassManager.cpp:1730:3</div>
<div>#19 0x00000000020ea715 compileModule(char**, llvm::LLVMContext&) src/tools/llc/llc.cpp:619:42</div>
<div>#20 0x00000000020e883c main src/tools/llc/llc.cpp:358:13</div>
<div>#21 0x00007f7700cf7f45 __libc_start_main build/eglibc-SvCtMH/eglibc-2.19/csu/libc-start.c:321:0</div>
<div>#22 0x00000000020e8029 _start (build/default/./bin/llc+0x20e8029)</div>
<div>Stack dump:</div>
<div>0. Program arguments: build/default/./bin/llc -asm-verbose=false -disable-wasm-fallthrough-return-opt -disable-wasm-explicit-locals</div>
<div>1. Running pass 'Function Pass Manager' on module '<stdin>'.</div>
<div>2. Running pass 'WebAssembly Assembly Printer' on function '@call_memcpy'</div>
<div>FileCheck error: '-' is empty.</div>
<div>FileCheck command line: build/default/./bin/FileCheck src/test/CodeGen/WebAssembly/global.ll</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, Jul 12, 2017 at 11:31 AM Daniel Neilson <<a href="mailto:dneilson@azul.com" target="_blank">dneilson@azul.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>That’s surprising to me; this should basically be NFC since nothing is actually using the introduced intrinsic. Any information that you can share would be appreciated.</div>
</div>
<div style="word-wrap:break-word">
<div><br>
</div>
<div>-Daniel</div>
</div>
<div style="word-wrap:break-word"><br>
<div>
<blockquote type="cite">
<div>On Jul 12, 2017, at 11:26 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:</div>
<br class="m_-8586201336093946882m_8787684839763540149m_3161077768360514121m_1278730238680722948m_-6007959335525949403Apple-interchange-newline">
<div>
<div dir="ltr">This seems to be breaking internal buildbots on WebAssembly tests (test/CodeGen/WebAssembli/{byval,f16,f32,f64,frem,global,i128,legalize,mem-intrinsics}.ll<br>
<br>
I've not reproduced this myself, but working on that now to get more info, including stack traces, etc.</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, Jul 12, 2017 at 8:25 AM Daniel Neilson via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Author: dneilson<br>
Date: Wed Jul 12 08:25:26 2017<br>
New Revision: 307796<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=307796&view=rev" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project?rev=307796&view=rev</a><br>
Log:<br>
Add element atomic memmove intrinsic<br>
<br>
Summary: Continuing the work from <a href="https://reviews.llvm.org/D33240" rel="noreferrer" target="_blank">
https://reviews.llvm.org/D33240</a>, this change introduces an element unordered-atomic memmove intrinsic. This intrinsic is essentially memmove with the implementation requirement that all loads/stores used for the copy are done with unordered-atomic loads/stores
of a given element size.<br>
<br>
Reviewers: eli.friedman, reames, mkazantsev, skatkov<br>
<br>
Reviewed By: reames<br>
<br>
Subscribers: llvm-commits<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D34884" rel="noreferrer" target="_blank">
https://reviews.llvm.org/D34884</a><br>
<br>
Modified:<br>
llvm/trunk/docs/LangRef.rst<br>
llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h<br>
llvm/trunk/include/llvm/IR/IntrinsicInst.h<br>
llvm/trunk/include/llvm/IR/Intrinsics.td<br>
llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp<br>
llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp<br>
llvm/trunk/lib/IR/Verifier.cpp<br>
llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll<br>
llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll<br>
<br>
Modified: llvm/trunk/docs/LangRef.rst<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/LangRef.rst?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/LangRef.rst?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/docs/LangRef.rst (original)<br>
+++ llvm/trunk/docs/LangRef.rst Wed Jul 12 08:25:26 2017<br>
@@ -10282,6 +10282,8 @@ overlap. It copies "len" bytes of memory<br>
to be aligned to some boundary, this can be specified as the fourth<br>
argument, otherwise it should be set to 0 or 1 (both meaning no alignment).<br>
<br>
+.. _int_memmove:<br>
+<br>
'``llvm.memmove``' Intrinsic<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
<br>
@@ -14178,4 +14180,81 @@ In the most general case call to the '``<br>
lowered to a call to the symbol ``__llvm_memcpy_element_unordered_atomic_*``. Where '*'<br>
is replaced with an actual element size.<br>
<br>
+Optimizer is allowed to inline memory copy when it's profitable to do so.<br>
+<br>
+'``llvm.memmove.element.unordered.atomic``' Intrinsic<br>
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
+<br>
+Syntax:<br>
+"""""""<br>
+<br>
+This is an overloaded intrinsic. You can use<br>
+``llvm.memmove.element.unordered.atomic`` on any integer bit width and for<br>
+different address spaces. Not all targets support all bit widths however.<br>
+<br>
+::<br>
+<br>
+ declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* <dest>,<br>
+ i8* <src>,<br>
+ i32 <len>,<br>
+ i32 <element_size>)<br>
+ declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i64(i8* <dest>,<br>
+ i8* <src>,<br>
+ i64 <len>,<br>
+ i32 <element_size>)<br>
+<br>
+Overview:<br>
+"""""""""<br>
+<br>
+The '``llvm.memmove.element.unordered.atomic.*``' intrinsic is a specialization<br>
+of the '``llvm.memmove.*``' intrinsic. It differs in that the ``dest`` and<br>
+``src`` are treated as arrays with elements that are exactly ``element_size``<br>
+bytes, and the copy between buffers uses a sequence of<br>
+:ref:`unordered atomic <ordering>` load/store operations that are a positive<br>
+integer multiple of the ``element_size`` in size.<br>
+<br>
+Arguments:<br>
+""""""""""<br>
+<br>
+The first three arguments are the same as they are in the<br>
+:ref:`@llvm.memmove <int_memmove>` intrinsic, with the added constraint that<br>
+``len`` is required to be a positive integer multiple of the ``element_size``.<br>
+If ``len`` is not a positive integer multiple of ``element_size``, then the<br>
+behaviour of the intrinsic is undefined.<br>
+<br>
+``element_size`` must be a compile-time constant positive power of two no<br>
+greater than a target-specific atomic access size limit.<br>
+<br>
+For each of the input pointers the ``align`` parameter attribute must be<br>
+specified. It must be a power of two no less than the ``element_size``. Caller<br>
+guarantees that both the source and destination pointers are aligned to that<br>
+boundary.<br>
+<br>
+Semantics:<br>
+""""""""""<br>
+<br>
+The '``llvm.memmove.element.unordered.atomic.*``' intrinsic copies ``len`` bytes<br>
+of memory from the source location to the destination location. These locations<br>
+are allowed to overlap. The memory copy is performed as a sequence of load/store<br>
+operations where each access is guaranteed to be a multiple of ``element_size``<br>
+bytes wide and aligned at an ``element_size`` boundary.<br>
+<br>
+The order of the copy is unspecified. The same value may be read from the source<br>
+buffer many times, but only one write is issued to the destination buffer per<br>
+element. It is well defined to have concurrent reads and writes to both source<br>
+and destination provided those reads and writes are unordered atomic when<br>
+specified.<br>
+<br>
+This intrinsic does not provide any additional ordering guarantees over those<br>
+provided by a set of unordered loads from the source location and stores to the<br>
+destination.<br>
+<br>
+Lowering:<br>
+"""""""""<br>
+<br>
+In the most general case call to the<br>
+'``llvm.memmove.element.unordered.atomic.*``' is lowered to a call to the symbol<br>
+``__llvm_memmove_element_unordered_atomic_*``. Where '*' is replaced with an<br>
+actual element size.<br>
+<br>
The optimizer is allowed to inline the memory copy when it's profitable to do so.<br>
<br>
Modified: llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h (original)<br>
+++ llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h Wed Jul 12 08:25:26 2017<br>
@@ -340,6 +340,12 @@ namespace RTLIB {<br>
MEMCPY_ELEMENT_UNORDERED_ATOMIC_8,<br>
MEMCPY_ELEMENT_UNORDERED_ATOMIC_16,<br>
<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_1,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_2,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_4,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_8,<br>
+ MEMMOVE_ELEMENT_UNORDERED_ATOMIC_16,<br>
+<br>
// EXCEPTION HANDLING<br>
UNWIND_RESUME,<br>
<br>
@@ -515,6 +521,11 @@ namespace RTLIB {<br>
/// MEMCPY_ELEMENT_UNORDERED_ATOMIC_* value for the given element size or<br>
/// UNKNOW_LIBCALL if there is none.<br>
Libcall getMEMCPY_ELEMENT_UNORDERED_ATOMIC(uint64_t ElementSize);<br>
+<br>
+ /// getMEMMOVE_ELEMENT_UNORDERED_ATOMIC - Return<br>
+ /// MEMMOVE_ELEMENT_UNORDERED_ATOMIC_* value for the given element size or<br>
+ /// UNKNOW_LIBCALL if there is none.<br>
+ Libcall getMEMMOVE_ELEMENT_UNORDERED_ATOMIC(uint64_t ElementSize);<br>
}<br>
}<br>
<br>
<br>
Modified: llvm/trunk/include/llvm/IR/IntrinsicInst.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/IntrinsicInst.h?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/IntrinsicInst.h?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/IR/IntrinsicInst.h (original)<br>
+++ llvm/trunk/include/llvm/IR/IntrinsicInst.h Wed Jul 12 08:25:26 2017<br>
@@ -296,6 +296,95 @@ namespace llvm {<br>
}<br>
};<br>
<br>
+ class ElementUnorderedAtomicMemMoveInst : public IntrinsicInst {<br>
+ private:<br>
+ enum { ARG_DEST = 0, ARG_SOURCE = 1, ARG_LENGTH = 2, ARG_ELEMENTSIZE = 3 };<br>
+<br>
+ public:<br>
+ Value *getRawDest() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_DEST));<br>
+ }<br>
+ const Use &getRawDestUse() const { return getArgOperandUse(ARG_DEST); }<br>
+ Use &getRawDestUse() { return getArgOperandUse(ARG_DEST); }<br>
+<br>
+ /// Return the arguments to the instruction.<br>
+ Value *getRawSource() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_SOURCE));<br>
+ }<br>
+ const Use &getRawSourceUse() const { return getArgOperandUse(ARG_SOURCE); }<br>
+ Use &getRawSourceUse() { return getArgOperandUse(ARG_SOURCE); }<br>
+<br>
+ Value *getLength() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_LENGTH));<br>
+ }<br>
+ const Use &getLengthUse() const { return getArgOperandUse(ARG_LENGTH); }<br>
+ Use &getLengthUse() { return getArgOperandUse(ARG_LENGTH); }<br>
+<br>
+ bool isVolatile() const { return false; }<br>
+<br>
+ Value *getRawElementSizeInBytes() const {<br>
+ return const_cast<Value *>(getArgOperand(ARG_ELEMENTSIZE));<br>
+ }<br>
+<br>
+ ConstantInt *getElementSizeInBytesCst() const {<br>
+ return cast<ConstantInt>(getRawElementSizeInBytes());<br>
+ }<br>
+<br>
+ uint32_t getElementSizeInBytes() const {<br>
+ return getElementSizeInBytesCst()->getZExtValue();<br>
+ }<br>
+<br>
+ /// This is just like getRawDest, but it strips off any cast<br>
+ /// instructions that feed it, giving the original input. The returned<br>
+ /// value is guaranteed to be a pointer.<br>
+ Value *getDest() const { return getRawDest()->stripPointerCasts(); }<br>
+<br>
+ /// This is just like getRawSource, but it strips off any cast<br>
+ /// instructions that feed it, giving the original input. The returned<br>
+ /// value is guaranteed to be a pointer.<br>
+ Value *getSource() const { return getRawSource()->stripPointerCasts(); }<br>
+<br>
+ unsigned getDestAddressSpace() const {<br>
+ return cast<PointerType>(getRawDest()->getType())->getAddressSpace();<br>
+ }<br>
+<br>
+ unsigned getSourceAddressSpace() const {<br>
+ return cast<PointerType>(getRawSource()->getType())->getAddressSpace();<br>
+ }<br>
+<br>
+ /// Set the specified arguments of the instruction.<br>
+ void setDest(Value *Ptr) {<br>
+ assert(getRawDest()->getType() == Ptr->getType() &&<br>
+ "setDest called with pointer of wrong type!");<br>
+ setArgOperand(ARG_DEST, Ptr);<br>
+ }<br>
+<br>
+ void setSource(Value *Ptr) {<br>
+ assert(getRawSource()->getType() == Ptr->getType() &&<br>
+ "setSource called with pointer of wrong type!");<br>
+ setArgOperand(ARG_SOURCE, Ptr);<br>
+ }<br>
+<br>
+ void setLength(Value *L) {<br>
+ assert(getLength()->getType() == L->getType() &&<br>
+ "setLength called with value of wrong type!");<br>
+ setArgOperand(ARG_LENGTH, L);<br>
+ }<br>
+<br>
+ void setElementSizeInBytes(Constant *V) {<br>
+ assert(V->getType() == Type::getInt8Ty(getContext()) &&<br>
+ "setElementSizeInBytes called with value of wrong type!");<br>
+ setArgOperand(ARG_ELEMENTSIZE, V);<br>
+ }<br>
+<br>
+ static inline bool classof(const IntrinsicInst *I) {<br>
+ return I->getIntrinsicID() == Intrinsic::memmove_element_unordered_atomic;<br>
+ }<br>
+ static inline bool classof(const Value *V) {<br>
+ return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));<br>
+ }<br>
+ };<br>
+<br>
/// This is the common base class for memset/memcpy/memmove.<br>
class MemIntrinsic : public IntrinsicInst {<br>
public:<br>
<br>
Modified: llvm/trunk/include/llvm/IR/Intrinsics.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Intrinsics.td?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Intrinsics.td?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/IR/Intrinsics.td (original)<br>
+++ llvm/trunk/include/llvm/IR/Intrinsics.td Wed Jul 12 08:25:26 2017<br>
@@ -873,6 +873,18 @@ def int_memcpy_element_unordered_atomic<br>
ReadOnly<1><br>
]>;<br>
<br>
+// @llvm.memmove.element.unordered.atomic.*(dest, src, length, elementsize)<br>
+def int_memmove_element_unordered_atomic<br>
+ : Intrinsic<[],<br>
+ [<br>
+ llvm_anyptr_ty, llvm_anyptr_ty, llvm_anyint_ty, llvm_i32_ty<br>
+ ],<br>
+ [<br>
+ IntrArgMemOnly, NoCapture<0>, NoCapture<1>, WriteOnly<0>,<br>
+ ReadOnly<1><br>
+ ]>;<br>
+<br>
+<br>
//===------------------------ Reduction Intrinsics ------------------------===//<br>
//<br>
def int_experimental_vector_reduce_fadd : Intrinsic<[llvm_anyfloat_ty],<br>
<br>
Modified: llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp (original)<br>
+++ llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp Wed Jul 12 08:25:26 2017<br>
@@ -4994,6 +4994,44 @@ SelectionDAGBuilder::visitIntrinsicCall(<br>
DAG.setRoot(CallResult.second);<br>
return nullptr;<br>
}<br>
+ case Intrinsic::memmove_element_unordered_atomic: {<br>
+ auto &MI = cast<ElementUnorderedAtomicMemMoveInst>(I);<br>
+ SDValue Dst = getValue(MI.getRawDest());<br>
+ SDValue Src = getValue(MI.getRawSource());<br>
+ SDValue Length = getValue(MI.getLength());<br>
+<br>
+ // Emit a library call.<br>
+ TargetLowering::ArgListTy Args;<br>
+ TargetLowering::ArgListEntry Entry;<br>
+ Entry.Ty = DAG.getDataLayout().getIntPtrType(*DAG.getContext());<br>
+ Entry.Node = Dst;<br>
+ Args.push_back(Entry);<br>
+<br>
+ Entry.Node = Src;<br>
+ Args.push_back(Entry);<br>
+<br>
+ Entry.Ty = MI.getLength()->getType();<br>
+ Entry.Node = Length;<br>
+ Args.push_back(Entry);<br>
+<br>
+ uint64_t ElementSizeConstant = MI.getElementSizeInBytes();<br>
+ RTLIB::Libcall LibraryCall =<br>
+ RTLIB::getMEMMOVE_ELEMENT_UNORDERED_ATOMIC(ElementSizeConstant);<br>
+ if (LibraryCall == RTLIB::UNKNOWN_LIBCALL)<br>
+ report_fatal_error("Unsupported element size");<br>
+<br>
+ TargetLowering::CallLoweringInfo CLI(DAG);<br>
+ CLI.setDebugLoc(sdl).setChain(getRoot()).setLibCallee(<br>
+ TLI.getLibcallCallingConv(LibraryCall),<br>
+ Type::getVoidTy(*DAG.getContext()),<br>
+ DAG.getExternalSymbol(TLI.getLibcallName(LibraryCall),<br>
+ TLI.getPointerTy(DAG.getDataLayout())),<br>
+ std::move(Args));<br>
+<br>
+ std::pair<SDValue, SDValue> CallResult = TLI.LowerCallTo(CLI);<br>
+ DAG.setRoot(CallResult.second);<br>
+ return nullptr;<br>
+ }<br>
case Intrinsic::dbg_declare: {<br>
const DbgDeclareInst &DI = cast<DbgDeclareInst>(I);<br>
DILocalVariable *Variable = DI.getVariable();<br>
<br>
Modified: llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp (original)<br>
+++ llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp Wed Jul 12 08:25:26 2017<br>
@@ -384,6 +384,16 @@ static void InitLibcallNames(const char<br>
"__llvm_memcpy_element_unordered_atomic_8";<br>
Names[RTLIB::MEMCPY_ELEMENT_UNORDERED_ATOMIC_16] =<br>
"__llvm_memcpy_element_unordered_atomic_16";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_1] =<br>
+ "__llvm_memmove_element_unordered_atomic_1";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_2] =<br>
+ "__llvm_memmove_element_unordered_atomic_2";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_4] =<br>
+ "__llvm_memmove_element_unordered_atomic_4";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_8] =<br>
+ "__llvm_memmove_element_unordered_atomic_8";<br>
+ Names[RTLIB::MEMMOVE_ELEMENT_UNORDERED_ATOMIC_16] =<br>
+ "__llvm_memmove_element_unordered_atomic_16";<br>
Names[RTLIB::UNWIND_RESUME] = "_Unwind_Resume";<br>
Names[RTLIB::SYNC_VAL_COMPARE_AND_SWAP_1] = "__sync_val_compare_and_swap_1";<br>
Names[RTLIB::SYNC_VAL_COMPARE_AND_SWAP_2] = "__sync_val_compare_and_swap_2";<br>
@@ -801,6 +811,23 @@ RTLIB::Libcall RTLIB::getMEMCPY_ELEMENT_<br>
default:<br>
return UNKNOWN_LIBCALL;<br>
}<br>
+}<br>
+<br>
+RTLIB::Libcall RTLIB::getMEMMOVE_ELEMENT_UNORDERED_ATOMIC(uint64_t ElementSize) {<br>
+ switch (ElementSize) {<br>
+ case 1:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_1;<br>
+ case 2:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_2;<br>
+ case 4:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_4;<br>
+ case 8:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_8;<br>
+ case 16:<br>
+ return MEMMOVE_ELEMENT_UNORDERED_ATOMIC_16;<br>
+ default:<br>
+ return UNKNOWN_LIBCALL;<br>
+ }<br>
}<br>
<br>
/// InitCmpLibcallCCs - Set default comparison libcall CC.<br>
<br>
Modified: llvm/trunk/lib/IR/Verifier.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Verifier.cpp?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Verifier.cpp?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/IR/Verifier.cpp (original)<br>
+++ llvm/trunk/lib/IR/Verifier.cpp Wed Jul 12 08:25:26 2017<br>
@@ -4044,6 +4044,42 @@ void Verifier::visitIntrinsicCallSite(In<br>
"incorrect alignment of the source argument", CS);<br>
break;<br>
}<br>
+ case Intrinsic::memmove_element_unordered_atomic: {<br>
+ auto *MI = cast<ElementUnorderedAtomicMemMoveInst>(CS.getInstruction());<br>
+<br>
+ ConstantInt *ElementSizeCI =<br>
+ dyn_cast<ConstantInt>(MI->getRawElementSizeInBytes());<br>
+ Assert(ElementSizeCI,<br>
+ "element size of the element-wise unordered atomic memory "<br>
+ "intrinsic must be a constant int",<br>
+ CS);<br>
+ const APInt &ElementSizeVal = ElementSizeCI->getValue();<br>
+ Assert(ElementSizeVal.isPowerOf2(),<br>
+ "element size of the element-wise atomic memory intrinsic "<br>
+ "must be a power of 2",<br>
+ CS);<br>
+<br>
+ if (auto *LengthCI = dyn_cast<ConstantInt>(MI->getLength())) {<br>
+ uint64_t Length = LengthCI->getZExtValue();<br>
+ uint64_t ElementSize = MI->getElementSizeInBytes();<br>
+ Assert((Length % ElementSize) == 0,<br>
+ "constant length must be a multiple of the element size in the "<br>
+ "element-wise atomic memory intrinsic",<br>
+ CS);<br>
+ }<br>
+<br>
+ auto IsValidAlignment = [&](uint64_t Alignment) {<br>
+ return isPowerOf2_64(Alignment) && ElementSizeVal.ule(Alignment);<br>
+ };<br>
+ uint64_t DstAlignment = CS.getParamAlignment(0),<br>
+ SrcAlignment = CS.getParamAlignment(1);<br>
+ Assert(IsValidAlignment(DstAlignment),<br>
+ "incorrect alignment of the destination argument", CS);<br>
+ Assert(IsValidAlignment(SrcAlignment),<br>
+ "incorrect alignment of the source argument", CS);<br>
+<br>
+ break;<br>
+ }<br>
case Intrinsic::gcroot:<br>
case Intrinsic::gcwrite:<br>
case Intrinsic::gcread:<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/element-wise-atomic-memory-intrinsics.ll Wed Jul 12 08:25:26 2017<br>
@@ -62,4 +62,67 @@ define void @test_memcpy_args(i8** %Stor<br>
call void @llvm.memcpy.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %Dst, i8* align 4 %Src, i32 4, i32 4) ret void<br>
}<br>
<br>
+define i8* @test_memmove1(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 1, i32 1)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $1, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_1<br>
+}<br>
+<br>
+define i8* @test_memmove2(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove2<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 2, i32 2)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $2, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_2<br>
+}<br>
+<br>
+define i8* @test_memmove4(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove4<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 4, i32 4)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $4, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_4<br>
+}<br>
+<br>
+define i8* @test_memmove8(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove8<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 8 %P, i8* align 8 %Q, i32 8, i32 8)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $8, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_8<br>
+}<br>
+<br>
+define i8* @test_memmove16(i8* %P, i8* %Q) {<br>
+ ; CHECK: test_memmove16<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 16 %P, i8* align 16 %Q, i32 16, i32 16)<br>
+ ret i8* %P<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $16, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_16<br>
+}<br>
+<br>
+define void @test_memmove_args(i8** %Storage) {<br>
+ ; CHECK: test_memmove_args<br>
+ %Dst = load i8*, i8** %Storage<br>
+ %Src.addr = getelementptr i8*, i8** %Storage, i64 1<br>
+ %Src = load i8*, i8** %Src.addr<br>
+<br>
+ ; 1st arg (%rdi)<br>
+ ; CHECK-DAG: movq (%rdi), [[REG1:%r.+]]<br>
+ ; CHECK-DAG: movq [[REG1]], %rdi<br>
+ ; 2nd arg (%rsi)<br>
+ ; CHECK-DAG: movq 8(%rdi), %rsi<br>
+ ; 3rd arg (%edx) -- length<br>
+ ; CHECK-DAG: movl $4, %edx<br>
+ ; CHECK: __llvm_memmove_element_unordered_atomic_4<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %Dst, i8* align 4 %Src, i32 4, i32 4) ret void<br>
+}<br>
+<br>
declare void @llvm.memcpy.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
+declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
<br>
Modified: llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff" rel="noreferrer" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll?rev=307796&r1=307795&r2=307796&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll (original)<br>
+++ llvm/trunk/test/Verifier/element-wise-atomic-memory-intrinsics.ll Wed Jul 12 08:25:26 2017<br>
@@ -22,4 +22,28 @@ define void @test_memcpy(i8* %P, i8* %Q,<br>
ret void<br>
}<br>
declare void @llvm.memcpy.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
+<br>
+define void @test_memmove(i8* %P, i8* %Q, i32 %A, i32 %E) {<br>
+ ; CHECK: element size of the element-wise unordered atomic memory intrinsic must be a constant int<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 1, i32 %E)<br>
+ ; CHECK: element size of the element-wise atomic memory intrinsic must be a power of 2<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 1, i32 3)<br>
+<br>
+ ; CHECK: constant length must be a multiple of the element size in the element-wise atomic memory intrinsic<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 4 %Q, i32 7, i32 4)<br>
+<br>
+ ; CHECK: incorrect alignment of the destination argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* %P, i8* align 4 %Q, i32 1, i32 1)<br>
+ ; CHECK: incorrect alignment of the destination argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 1 %P, i8* align 4 %Q, i32 4, i32 4)<br>
+<br>
+ ; CHECK: incorrect alignment of the source argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* %Q, i32 1, i32 1)<br>
+ ; CHECK: incorrect alignment of the source argument<br>
+ call void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* align 4 %P, i8* align 1 %Q, i32 4, i32 4)<br>
+<br>
+ ret void<br>
+}<br>
+declare void @llvm.memmove.element.unordered.atomic.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i32) nounwind<br>
+<br>
; CHECK: input module is broken!<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div></blockquote></div></blockquote></div>